Becker et al. Next generation sequencing (NGS) is the driving force behind precision medicine and is revolutionizing most, if not all, areas of the life sciences. Particularly when targeting the major common diseases, an exponential growth of NGS data is foreseen for the next decades. This enormous increase of NGS data and the need to process the data quickly for real-world applications requires to rethink our current compute infrastructures. Here we provide evidence that memory-driven computing (MDC), a novel memory-centric hardware architecture, is an attractive alternative to current processor-centric compute infrastructures. To illustrate how MDC can change NGS data handling, we used RNA-seq assembly and pseudoalignment followed by quantification as two first examples. Adapting transcriptome assembly pipelines for MDC reduced compute time by 5.9-fold for the first step (SAMtools). Even more impressive, pseudoalignment by near-optimal probabilistic RNA-seq quantification (kallisto) was accelerated by more than two orders of magnitude with identical accuracy and indicated 66% reduced energy consumption. One billion RNA-seq reads were processed in just 92 seconds. Clearly, MDC simultaneously reduces data processing time and energy consumption. Together with the MDC-inherent solutions for local data privacy, a new compute model can be projected pushing large scale NGS data processing and primary data analytics closer to the edge by directly combining high-end sequencers with local MDC, thereby also reducing movement of large raw data to central cloud storage. We further envision that other data-rich areas will similarly benefit from this new memory-centric compute architecture. Next generation sequencing (NGS) and its manifold applications are transforming the life and medical sciences 1,2,3,4 as reflected by an exponential 860-fold increase in published data sets during the last 10 years (Fig. 1a) . Considering only human genome sequencing, the increase in genomic data over the next decade has been estimated to reach more than three Exabyte (3x10 18 byte) 5 . NGS data are projected to become on par either with data collections such as in astronomy or YouTube or the most demanding in terms of data acquisition, storage, distribution, and analysis . In addition, since one of the cornerstones of genomics research and application is comparative analysis between different samples, long-term storage of genomic data will be mandatory. To cope with this data avalanche, so far, compute power is simply increased by building larger high-performance computing (HPC) clusters (more processors/cores), and by scaling out and centralizing data storage (cloud solutions) 6, 7 . While currently still legitimate, it can already be foreseen that this approach is not sustainable. Centralized storage and computing leads to increased data movement, data duplication, and additional risks to data security and privacy. This is of particular interest given recent data privacy laws that have been put in place (e.g. the General Data Protection Regulation (GDPR) in the European Union) 8 . Another important aspect that requires re-thinking algorithms and the required compute infrastructure to cope with the growing requirements of genomic research is the enormous energy consumption of data centers providing the respective cloud services. Moreover, since many current algorithms are not scaling well with increasing data volumes, just increasing (scaling out) classical HPC infrastructures might not suffice. A solution to these emerging limitations requires reduction of 1) compute time, 2) data movement, 3) data duplications, and 4) energy foot print to a minimum, while increasing 5) data and privacy protection at the same time. We addressed these issues by combining a novel computer hardware architecture -namely memory-driven computing (MDC) -with optimized algorithms in genomic data analytics. All classical computers, from laptops to super computers, are based on the von Neumann architecture from Memory-driven computing 3 in CPU-Cores (up to 224) and memory (up to 12TB). Servers (standalone, rackmount or blade systems) may be clustered over high-performance interconnects to scale CPU-resources and workloads may be deployed on the cluster using a scheduler. Specialized HPC systems may reduce space requirements further by higher integration. Nowadays, scale up interconnected systems can provide up to 48TB of main memory acting as a single server instance. In contrast to the von Neumann architecture (Extended Data Fig. 1a) , MDC can pool large, shared and persistent (storage class) memory and connects this memory and compute resources (CPU) via a fabric following the Gen-Z protocol ('Fabric Attached Memory', FAM) (Extended Data Fig.  1b) 10 . While traditional systems are processor-centric (Extended Data Fig. 1a) , in MDC the FAM is the central component (Extended Data Fig. 1b ) thereby avoiding I/O (Extended Data Fig. 1c ) by allowing all processors to directly access the FAM (Extended Data Fig. 1d ). The MDC architecture enables all processors in the cluster to operate on shared data concurrently and the combination of MDC and the Gen-Z protocol provides the necessary byte-addressable scaling up to 4096 Yottabyte (2 92 byte) of FAM (Extended Data  Fig. 1e ). Yet, long-term storage or archiving can still be accessed via traditional I/O-handling (Extended Data Fig. 1f) . MDC has been realized as a prototype system called the Memory Fabric Testbed (MFT) with 160 TB of memory, but the concept can also be emulated on classic architecture using a FAM Emulation (FAME) software 11 . It has been hypothesized that processing time and compute power can be substantially reduced by eliminating the need for I/O handling within the application . The use of so-called 'in-memory' processing systems has been a first attempt towards this direction, however, unlike MDC, these systems do not offer large persistent memory and still require fetching and storing data for processing from I/O-based storage systems. Genomic data processing pipelines often consist of serial tasks. Using classical hardware infrastructure, each task loads small parts of a large data set from storage to local volatile memory (RAM) and writes its results back (Fig. 1c) . Subsequent tasks need to fetch the data again, incurring high file access overhead 13 . With MDC, all data can be held in memory (Fig. 1d ) and accessed as necessary throughout the pipeline, thus avoiding the penalty associated with moving data to disk between pipeline stages. Two main adaptations are needed to move applications to MDC: per-application optimization (Extended Data Fig. 2a ) and pipeline-wide optimization (Extended Data Fig. 2b) . First, individual applications can be modified for more efficient data access, and the abundance of memory enables in-memory data structures to be optimized for processing speed. Second, data exchange between parts of the pipeline can be optimized by not only reading from memory but also using in-memory data formats that reduce costly data transformations and parsing across the processing pipeline. Irrespective of source, all NGS data require lengthy pre-processing by either alignment (including pseudoalignment) to an annotated reference or de novo assembly prior RNA-seq quantification and further data analysis 14, 15 . To demonstrate the benefits of MDC for RNA-seq alignment and quantification, we used one of the fastest methods introduced as near-optimal probabilistic RNA-seq quantification (kallisto) 16 . Currently, kallisto reads small chunks of data in a serial fashion and processes them in parallel (Fig. 2a , orig/host). In contrast, the modified version of kallisto overcomes the data access bottleneck by allowing random parallel access and processing of larger data packages (Fig. 2a , mdc/FAME, mdc/host). The modified version of kallisto can be executed directly on classical systems with sufficiently large memory (RAM) or via the FAME emulation environment (for further technical details see Methods). We examined our modifications using the RNA-seq dataset (ERR188140) with 140 x 10 6 reads introduced recently 16 (Fig.  2b, c, d ) on two systems for the classical architecture (Blades, Superdome X (SDX) , Fig 2b, c) and the MFT for the MDC architecture (Fig. 1d, Extended Data Fig. 3) . Our results clearly show that the modifications of kallisto significantly reduced the runtime within each of the tested systems (Fig. 2b, c, d ). Not Becker et al. Memory-driven computing 4 unexpectedly, emulation does not produce the same gain as native hardware, but even under these conditions MDC-modified code showed a significant gain over the original. To determine general applicability and scaling, we collected 1,181 publicly available RNA-seq data sets derived from blood donors with different diseases from a total of 19 studies (Extended Data Fig. 4) . To rule out any influence of the underlying hardware, we used both the Blades and the SDX environment. All the RNA-seq data sets were processed with both the original kallisto version and the MDC-modified version either within the FAME environment or directly on the system (Fig. 2e, f) . Independent of the size of the individual dataset, the MDC-modified algorithm always outperformed the original version both within FAME and directly on hardware resulting in reduced runtime of up to 13.4-fold. Even more surprising the gain of performance was disproportionally higher with larger data size. We further assessed kernel and user times (Methods) and found kernel time to be negligible (data not shown) while user time revealed a reduction of 66% by the MDC-modified version of kallisto when executed directly on the Blades or the SDX (Extended Data Fig. 5a, b) . Also considering parallelism of data processing we observed an up to 112-fold reduction of compute time in comparison to the original hardware and software settings. To determine accuracy between the different versions of the algorithm, we compared the RNA-seq results derived by the original version of kallisto, the MDC-modified version and two alternative aligners (HiSat2, STAR) (Fig. 3a) . MDC modification to kallisto only showed very minor differences (correlation 1-7.5x10 -9 ), while we observed lower correlations with HiSat2 and STAR, a finding that was previously reported 16 . We have chosen transcriptome assembly to demonstrate the effect of pipeline wide optimizations (for detailed technical description see methods). We followed the steps proposed by Pertea et al. 17 and have focused on SAMtools 18 and StringTie 19 that run serially with I/O based data exchange (Extended Data Fig.  2b ). Prior work by Hajj et al. 20 demonstrated that in-memory processing can accelerate SAMtools. We modified the sorting function to load and sort data in parallel without any temporary files and to keep the results in memory for StringTie to avoid disk-based data exchange. Our initial findings on SAMtools show a significant acceleration of up to 5.9-fold on the Blades and SDX for the ERR188140 data set (Fig. 4a, b) . These finding were supported by the larger 1181 sample data set (Fig. 4c, d) . Future work will focus on addressing the current bottlenecks in StringTie. Collectively, genomic research and usage of genomic data for precision medicine require significant reduction of compute time, data movement, data duplication and energy consumption. We provide compelling evidence that these goals can be reached by moving genomic data processing to MDC, as exemplified for RNA-seq assembly and pseudoalignment. Considering the increase of memory also allowing for an increase of parallelism of data processing we demonstrate an up to 112-fold reduction in compute time in comparison to the previously used single core system for probably one of the most sophisticated RNA-seq data quantification strategies available to date (kallisto) 16 . Within one computer system, the modifications of kallisto resulted in up to 13.4-fold reduction of runtime and up to 66% reduction of user time (Methods), which is an indicator for reduced energy consumption. Similarly, a significant runtime reduction was also observed for the first step of the assembly of RNA-seq data. We predict that similar results for data processing can be obtained for all other NGS-based data (e.g. genome and epigenome alignment and assembly) even up to the single cell level. Our results clearly indicate that we can gain major benefits by investing in both optimized data analytics algorithms as well as the underlying hardware and software infrastructure. MDC seems to be an ideal architecture model since it accommodates the requirements of future computing in biomedical research. MDC is centered on data rather than processors (Extended Data Figure 1b) ; is modular due to the memory fabric specifications (Gen-Z) (Extended Data  Fig. 6a) ; allows the integration of specialized analysis hardware 21
Memory-driven computing REF , in this research a huge pool of different types of memories created and connected to the processing resources through Gen-Z communication protocol.
Memory-driven computing accelerates genomic data processing
bioRxiv
Biology