Please, Translate the following English into Korean:
 Hi, I'm <PRESIDIO_ANONYMIZED_PERSON> from Carnegie Mellon University and I'll be presenting P-OPT, a practical optimal cache replacement policy for graph analytics workloads. This work was done in collaboration with Neil Grego and Amar Jareel from NVIDIA and my advisor Brandon Lucia. Graph analytics has many important applications ranging from the traditional use cases such as path planning and social network analysis to more modern use cases such as representation learning and graph neural networks. In the past, large graphs were primarily processed using distributed systems where the graph would have to be partitioned across multiple machines. However, the increasing main memory capacity of server-closed processors allows us to analyze large graphs using just a single machine. To illustrate that point, I'm showing this table highlighting a few of the popular input graphs used in the evaluation of graph processing systems. The last column of this table shows the memory requirement of a typical application analysis. The last column of this table shows the memory requirement of a typical application analysis. We can see from this table that for the graphs highlighted in red once we can change theуск is to read this value each time with a combination of like停 of priority and delay. However, these graphs can't control the lead as needed by a single LeuteQ-4071 iThe This shows that there are a lot of functions that need to be partitioned into multipleобibtive and Lemurist tools to monitor model processing. However, for graphs highlighted in green, the memory requirements for dis So while single-node graph analytics can be more efficient than distributed graph processing, I'm going to show that single-node graph processing performance is still suboptimal. The primary reason for that is single-node graph analytics performance is heavily DRAM latency bound. So as we can see from this chart, graph applications can spend up to 80% of their execution time just waiting for requests to come back from DRAM. And the main reason graph application performance is so DRAM latency bound is because graph applications tend to have poor cache locality. The chart on the left shows the last-level cache miss rate for a range of graph applications. And we can see that the high last-level cache miss rate of graph applications leads to many DRAM accesses which ends up dominating execution time. So if we can improve locality of graph applications, we can reduce DRAM accesses and improve performance. So with that, I just covered the primary bottleneck of single-node graph analytics, which is poor cache locality. For the rest of the talk, I'll be following this outline. I'll first discuss this. So let's get started. So let's start with the reasons for poor cache locality in single-node graph analytics. Then I will be covering the main insight of this work, which is that the ladies optimal replacement policy is viable for graph processing workloads. And finally, I'll talk about P-OCT, which is a practical design for performing optimal cache replacement. So next up is the reasons for poor cache locality. In this section, I'll be talking about the fundamental reason for poor cache locality in graph processing workloads, which is the irregular memory access pattern. And finally, I'll be covering about why existing replacement policies are insufficient for graph analytics workloads. Before that, just a quick overview of single-node graph analytics workloads. So shown on the slide is the input graph and its corresponding adjacency matrix representation. Whenever we have an edge connecting two vertices, then the corresponding entry of the adjacency matrix is a non-zero. For the typical graphs that we're going to be dealing with, the adjacency matrix is going to be extremely sparse. And because of the high sparsity value, the adjacency matrix is going to be extremely sparse. And because of the high sparsity value, the adjacency matrix is going to be extremely sparse. So we can't just restore the graph using the adjacency matrix format. We need a compressed representation. A popular compressed representation is the compressed-pass-row or CSR format. One way to think of the CSR is basically all the non-zero coordinates of the adjacency matrix sorted by source IDs. A CSR is an efficient encoding of the outgoing neighbors of each vertex. Alternatively, we could store all the non-zero coordinates sorted by destination IDs, and that would be the adjacency matrix. So we can use the compressed-pass-column or the CSC format. A CSC format is an efficient encoding of all the incoming neighbors of any vertex. And now that we have seen the common representations for graph workloads, which are CSR and CSC, we can take a look at the common computation patterns in graph analytics. All graph analytics workloads basically reduce to two computation patterns. The first pattern is where we process all the outgoing neighbors for every vertex. And that's what is known as a push execution as shown here. Basically, for each edge in the graph, we are basically accessing certain data structures based on the destination and source coordinates. In this code, we can see the concept of a frontier, which is basically used to refer to executions where we only have to process a subset of the vertices, not all the vertices in the graph. And because a push execution is a row-by-row traversal of the adjacency matrix, it is a CSR traversal. Alternatively, we can use the C-SR traversal. This is a C-SR traversal. Alternatively, graph applications could analyze all the incoming neighbors for every vertex, and that would be a pull execution. A pull execution is a column-by-column traversal, hence it's a CSC traversal. And graph applications are iterative, where in each iteration they may switch between push and pull executions depending on the frontier density. And because graph applications switch between push and pull executions, they tend to store both the CSR as well as the CSC. So with that quick overview, let's move on to the next step. We will be looking at the C-SR and the CSC traversal. So let's start with the C-SR. So in this quick overview, we are now ready to see the fundamental reason for poor cache locality in graph workloads, which is the irregular memory access pattern. And to see that in detail, we are going to be looking at the pull execution, which was introduced previously. And recall, a pull execution is a CSC traversal. So the second line of the pull execution basically tries to find out the source neighbors for each destination vertex. And when we are trying to find the source neighbors, we are going to consult the entries in this compress-pass column. The problem here is that the pull execution is not a C-SR traversal. The problem here is that the contents of the compress-pass column can be arbitrarily ordered and it's further dependent on the graph structure as well as the vertex ordering. Therefore, all data structures which are indexed based on the source IDs tend to have irregular memory accesses. And this is a problem for graph analytics workloads because for the input graphs that we typically deal with, the irregular data footprint is much larger than the last available cache capacity. For example, if we consider a typical graph with 32 million vertices and a zero-sum graph, we can see that the index is much larger than the last available cache capacity. So we can see that the index is much larger than the last available cache capacity. And if we assume 4 bytes per element, the irregular data footprint comes out to be 128 megabytes, which is much larger than the last available cache capacity. So we saw that graph applications have a high irregular data footprint. And whenever we are talking about workloads where the working set size is larger than the cache capacity, we have to talk about cache replacement policy. So we performed an experiment where we tried to find out the effect of different cache replacement policies on the PageRank graph application. So on this chart, the x-axis is different last-level cache replacement policies, and the y-axis is the Mises-Perkins instruction or the MPKI metric. In this chart, we have included state-of-the-art replacement policies. For example, a version of DRIP has been shown to be used in server-class processes, whereas the Hawkeye policy won the cache replacement championship a few years back. But even these state-of-the-art replacement policies only provide a marginal benefit over the simple LRU replacement. And the primary reason for that is because all these state-of-the-art replacement policies are based on heuristics. And unfortunately the heuristics that they use are insufficient to capture the complex reuse patterns of graph analytics workloads. So to take a look at the complex reuse patterns of graph analytics workloads, let's once again look at the pull execution where the source data accesses were irregular. The reuse patterns of graph analytics workloads is very similar to the pull execution for the source data accesses. The reason for this is that the data reuse pattern is unique for every vertex. So any one-size-fits-all solution used by replacement policies is unlikely to work for graph analytics workloads. And the second challenging reuse property is that even for a single vertex, the reuse pattern is dynamically varying depending on where we are, which phase we are in the graph execution. So the expansion of these two complex reuse patterns basically causes the state-of-the-art replacement policies to perform not much better than the LRU replacement policy. So with that, we just saw the reasons for poor cache locality in graph applications. And we saw that the complex reuse patterns of graph analytics workloads basically causes existing replacement policies to not perform much better than the very simple LRU policy. In the next section of the talk, I'll be talking about Beledi's optimal replacement policies. So this is one of the reasons why graph analytics workloads are so important. So let's go over the reasons for poor cache locality in graph applications. And particularly, we are going to be seeing that Beledi's optimal replacement policy is actually viable for graph processing. And because Beledi's optimal replacement policy is not a heuristics-based solution, it is able to successfully capture all the complex reuse patterns for graph application data. So once again, the main insight of our work is that Beledi's optimal replacement policy is viable for graph processing. So what exactly is Beledi's optimal replacement policy? Beledi's replacement policy is a theoretical replacement policy. It is a theoretical upper bound on cache replacement. And essentially what it says is that if we have a cache which is completely filled with elements and we need to insert a new element, then Beledi's optimal replacement policy would say that we should be evicting the element that's going to be accessed furthest in the execution or furthest in the future. And just by this definition, it's probably clear that Beledi's optimal replacement is an upper bound and it's often impractical because it relies on oracular knowledge of the future. But a key observation here is that the optimal replacement policy is not a heuristic-based solution. The key observation of our work was that the graph structure, particularly the graph's transpose, contains all the necessary information required to perform Beledi's optimal replacement. This one key graph application property that actually allows us to use the graph's transpose to perform Beledi's optimal replacement. And we are going to see that application property with the help of the pull execution introduced previously, which was a CSC traversal. Shown on the bottom right of the slide is the irregularization of the graph. And this is the irregularization of the graph. And this is the irregular data stream that would be generated by the pull execution. And recall that in a pull execution, it is the source data accesses which are irregular. So the key graph application property that actually allows us to perform Beledi's optimal replacement for graph analytics is that in a pull execution, the destination IDs are kind of like timestamps for when an irregular data element is accessed. To see that point in action, consider the source data is two element. So source data is two is first accessed. When we are processing the neighbors of vertex D zero. Next, it is accessed when we are processing the neighbors of vertex D one. And finally, it is accessed when we are processing the vertex neighbors of vertex D three. So in this fashion, we can see that the destination ID at which source data elements are accessed are kind of like timestamps with a larger destination ID indicating a later reference in the graphs execution. This property, we are now ready to see how the graphs transpose can be used for optimal cache replacement. And to do that, we are going to be looking at a very simple cache simulation. For the sake of presentation, we are going to be making two simplifying assumptions for the cache shown in the bottom left. So the first assumption is that each line of this cache is only going to contain one source data element. And the second assumption is that we are only going to insert the irregular data elements into this cache. So with those assumptions out of the way, let's start the cache simulation. So we first make an access to source data is one. The cache is empty. We can easily insert it into the cache. That's a cold miss. Next we access source data is two. Again, we have an empty entry. We can insert it into the cache. And now we have entered an interesting scenario. The cache is completely full and we need to insert source data is four. So the cache replacement question that we need to answer is which of these two lines should we be evicting? Source data is one or source data is two. And the bellini solution, of course, is that we should be evicting the line which is going to be accessed later in the execution or which will be accessed further in the execution. So now recall the key graph application property, which was that the destination ID at which a source data element is accessed is kind of like a timestamp. So to find out when source data is one will be accessed next, we essentially need to scan the row corresponding to vertex S1 to find out that it's going to be accessed next while processing the neighbors of vertex D4. And recall that whenever we are looking at the source data element, we are looking at row information. The most efficient encoding is the compress pass row. So we can consider the CSR entry corresponding to vertex S1 to find out that it's only going to be accessed next at point D4. Similarly for vertex S1, we can consult its CSR entry to find out that it's going to be accessed next at point D1. And now that we have the next references for both of these cache lines, we can say that source data S1 is the one that's going to be accessed later in the execution and hence it should be evicted. As for the ladies optimal replacement. So just as a quick recap, in this example, we saw that in a pull execution, which was a CSC traversal, the transpose, which is the CSR contained all the necessary information for us to be able to perform the ladies optimal replacement. And this example is not restricted to a pull execution. For example, if we had a push execution, then this time the transpose would be a CSC and we could still do the ladies optimal replacement. So the general principle is that for a graph application, the transpose contains all the necessary information for optimal replacement. And such a transpose based optimal replacement policy could provide significant benefits. And we are showing that from, and here's the chart from before, but we've added the results for topt, a transpose based optimal replacement in the end. And we can see that compared to the state of the art solutions topt provides significant LLC misreduction. And this is a very significant improvement. So, we can see that the optimal replacement is not a good result. And we can see that the optimal replacement is not a good result. So, we can see that the optimal replacement is not a good result. We found that on average topt reduces LLC misses by 1.7x compared to LRU. When these results are impressive, unfortunately, the topt results are an idealized result because they don't include the overheads of topt. So as far as we are concerned with the practical replacement policy, we are not done yet. So let's take a look at the overheads of transpose based optimal replacement or topt. All the overheads are the same. The overheads of transpose based optimal replacement are basically for finding the next references using the gross transpose. And there are two key overheads that transpose based optimal replacement incurs. The first overhead is a DRAM access overhead. So recall that depending on the contents of the cache at replacement time, we may have to scan arbitrary rows of the adjacency matrix. Or essentially what that means is that depending on the contents of the cache, we may have to find out arbitrary neighborhoods in the cache. So we can't fit them in the CSR. And because these CSR accesses are irregular, we can't expect to fit them in cache. So during cache replacement time, we would be doing additional DRAM accesses. So that's an overhead. But even after we find out the vertex neighborhood for any given vertex, it's not a simple operation because we need to do a traversal to find out when the vertex is going to be accessed next. So there's also a runtime overhead associated with finding the next differences even after having... finding the neighborhoods. So essentially the main question that we answered in this paper is how do we retrieve the next reference information that is encoded in a graph transpose, but do so without incurring these two overheads of topt. So with that, I just covered the main insight of our work, which was that the graph transpose allows us to perform optimal cache replacement, but we can't directly use that solution because it incurs overheads. In the final part of the presentation, I'm going to be introducing our work Popt, which is a practical optimal cache replacement solution, and it relies on quantization to reduce overheads. So the main problem in transpose-based optimal replacement is that when it consults the transpose, we are tracking next references at the granularity of destination IDs. For example, for source data S2, we are going to look at the CSR and find out that its source data S2 will be in the next reference. So we can see that the source data S2 is in the next reference. One way to reduce overheads is to divide the execution into coarse-grained regions called epochs and only track next references at the granularity of epochs. So, for example, if we divide the execution into epochs where each epoch spans the processing of two vertices, now the source data next references are basically quantized into epochs 0 and 1. We can see that by controlling the number of epochs, each epoch is quantizing itself epochs, we can control the amount of quantization in the next reference value and this kind of exposes an accuracy versus replacement metadata size straight up. For all our evaluation, we found that choosing 256 epochs or using 8 bits to represent the next references turns out to be a pretty good balance point between accuracy and metadata size. The entire reason we broke up the execution into coarse-grained epochs is to be able to come up with a new data structure called the re-reference matrix. The re-reference matrix is a quantized representation of the next reference values that is encoded in a graph's transpose and it is the re-reference matrix that allows us to perform practical optimal replacement. So I'll just quickly go over how the re-reference matrix is constructed and how it helps us to go towards a practical design. So for the epoch execution model shown on the left, the re-reference matrix is shown on the right. The re-reference matrix is very simple. It is a very simple model. It is a very simple model. It is a very simple model. It is basically constructed using the graph's transpose before the graph execution. So it's a pre-processing step. And the re-reference matrix has the dimensions of number of epochs and the number of cache lines spanning the irregular data elements. Each entry of the re-reference matrix basically stores for each cache line at each epoch what is the distance to the next reference in epochs. So to see that in action, consider vertex S1 at epoch 0. We know that vertex S1 at epoch 0 is going to be accessed in two epochs. So the corresponding re-reference matrix entry is 2. Next, when vertex S1 is at epoch 1, we know that it's going to be accessed in the very next epoch. So the re-reference matrix is 1. And finally, vertex S1 is accessed at epoch 2. So the corresponding entry for the cache line at epoch 2 is 0. So in this fashion, for each irregular data cache line and at each epoch, we can pre-compute the quantized next reference values. And the benefit of arranging next references in this re-reference matrix format is that at any point in time, we are concerned with only one column of the re-reference matrix. For example, when the pull execution is in epoch 1, then all the elements of interest are in the E1 column of the re-reference matrix. And because the re-reference matrix is quantized, the footprint of the re-reference matrix column is really small. For example, when we plugged in a typical values for the quantization level of cache line sizes, we found that a 32-medium vertex graph contributes to a 2-megabyte re-reference matrix footprint. And 2 megabytes can reasonably fit within a typical server-class processor's last-level cache. So we can completely eliminate DRAM access overhead during cache replacement because all the necessary information can fit within the cache. Another benefit of the re-reference matrix is that the quantized next references are pre-computed. So we can re-reference the re-reference matrix within the cache. So we can re-reference the re-reference matrix within the cache. So we don't need to perform any complicated traversal operations. It's a constant time lookup to find out what is the next reference for any given vertex. So in this fashion, we can see that the re-reference matrix eliminated both the overheads of transpose-based optimal replacement. And it is the re-reference matrix which allows us to come up with practical optimal cache replacement, which we call P-opt. So the way P-opt works is that we reserve a portion of the LLC to store a column of the re-reference matrix column. So we can store the re-reference matrix column within the LLC. And we rely on existing support for way-based cache partitioning to do this. And once we have stored a portion, or once we have stored the re-reference matrix column within the LLC, then during an epoch, we use these quantized next reference values to drive an approximate version of the ladies optimal replacement. Let's just take a look at how P-opt actually performs. So we're going to be looking at LLC misintroductions normalized to LRU. In this chart, we are showing the results for the DRIP and LRU policies for the PageRank application. If we add the result for P-opt, we can see that P-opt provides significantly better LLC misintroduction compared to both LRU and DRIP. And the interesting fact here is that the P-opt results are even after including the fact that a few ways of the LLC had to be reserved for storing the re-reference matrix column. Another interesting fact is that the P-opt results is pretty close to the ideal case scenario, where we are able to access the precise replacement information and pay zero cost for doing so. Basically, the T-opt results from before. So this just goes to show that our quantization trick basically does not lose significant does not have a significant impact on the quality of replacement information. We can still get close to ideal benefits. And these trends translate across a range of graph applications, and we found that on average P-opt is able to reduce LLC misses by 35% over LRU and 24% compared to DRIP. And the P-opt's LLC misreductions are just 12% away from the ideal case scenario. And all these LLC misreductions of P-opt basically translate almost one to one into speedups, which just goes to show how DRAM latency bound graph applications are. Any reduction in LLC misses translates into speedups. We found that P-opt is able to provide up to 1.66 speedup over LRU replacement. In the interest of time, I didn't cover a lot more details that were there in the paper. So in the paper, we looked at modified reference matrix design that allows us to offset the quality loss that comes from quantization. We showed how P-opt could be generalized to support NUCA caches, handle multiple irregular data types. We also showed how P-opt is complementary to tiding optimization, software tiding optimizations, such as 1D tiding and propagation blocking. And finally, we were able to show that P-opt's benefits are not dependent on any graph structure or the T-opt. So we were able to show that P-opt's benefits are not dependent on any graph structure or the T-opt. So we were able to show that P-opt's benefits are not dependent on any graph structure or the T-opt. But therefore, P-opt is able to provide more consistent gains compared to prior graph optimizations. And we have released the cache simulators that we use for a subset of our evaluations of P-opt. So in summary, in this work, we presented two key ideas. The first key idea was that we are actually able to perform Billady's optimal cache replacement for graph analytics workloads. So we're able to get this theoretical upper bound replacement policy by using the information encoded in a graph. And we also showed that we can perform a very similar cache replacement for graph analytics workloads. So we're able to get this theoretical upper bound replacement policy by using the information encoded in a graph. And we also showed that we are actually able to perform a very similar cache replacement for graph analytics workloads. So we're able to get this theoretical upper bound replacement policy by using the information encoded in a graph. And the second key idea was that we can quantize the transposed data to allow us to come up with a practical implementation of optimal cache replacement. And with the quantization, even after including all overheads, we are just 12% away from the ideal case scenario. So with that, I've reached the end of this presentation. Please check out our paper and check out our cache simulators. Thanks a lot for watching.