SORS: Navigating heterogeneity and scalability in modern chip design
Channel: BSC CNS
Date: 27/06/2023

🗣Speaker: Speaker: Marcelo Vera, PhD candidate in the Department of Computer Science at Princeton University, US.

Abstract: Due to the growing size of AI models, computationally-intensive workloads, such as neural networks, are increasingly adopting sparse data structures to reduce memory footprints. Similar sparse structures are also central to graph analytics, recommendation systems, and computational chemistry. Traversing sparse data leads to irregular, data-dependent memory accesses, which frequently miss in the cache hierarchy due to limited spatial or temporal reuse. Consequently, fetching data from main memory results in processor stalls and increased memory access contention in multi-processor systems. This talk focuses on how computers process sparse data structures by introducing innovations across the hardware-software stack.

First, I present the hardware design of a memory-access engine, MAPLE, to which off-the-shelf processing elements (PEs) can offload irregular memory accesses and consume the data via software-managed queues. When used in conjunction with software pipelining or prefetching, MAPLE effectively mitigates memory latency, providing 2x speedups over software-only and hardware-only techniques.

Second, to scale sparse applications to thousands of PEs, my work introduces Dalorex, an execution model where a program is split at pointer indirection into tasks that only access a small range of the address space. Tasks then execute at the processor with dedicated access to that memory range, which creates a pipeline of task executions for each independent exploration of the data structure. My work also proposes a novel integration of memory technology and PEs, which enables creating Dalorex at scale. I present results with parallelizations of six sparse workloads scaling up to a million PEs, leading to faster execution times than the top entries of the Graph500 ranking for the studied problem sizes.

In an era of big data, sparse data structures are crucial for a wide range of applications. My work demonstrates that by migrating the execution to the PE closest to a memory region as the sparse structure is traversed, we can enable massive improvements in performance and energy efficiency. While different memory integrations involve trade-offs in performance, power, and cost, my proposed chiplet integration enables cost-effective decision-making after silicon production. Moreover, a section of the talk will be devoted to two formal verification methodologies developed to aid the design of safe and secure hardware modules in the context of the Cambrian explosion of accelerators and specialized hardware.

Further information here: https://www.bsc.es/research-and-development/research-seminars/sors-navigating-heterogeneity-and-scalability-modern-chip-design