# PLORC: A Pipelined Lossless Reference-Free Compression Architecture for FASTQ Files
## 1 Introduction  
The rapid advancement of next-generation sequencing (NGS) technologies has led to an exponential increase in the volume of genomic data, creating significant challenges for genomic analysis systems [1]. The collection and analysis of genomic data require the cooperation between multiple computation nodes [2, 3, 4, 5, 6]. Genomic data are transmitted between these nodes through a network. Recent research has been focused on the acceleration of genomic analysis [2, 3, 4, 7, 8, 9, 10, 11]. However, the performance of data transmission and storage has not kept up pace with the increase in the genomic data volume in recent years [12, 13, 14, 15]. As a result, storage and transmission have become bottlenecks in genomic analysis systems. Compression algorithms can address these challenges by reducing the volume of genomic data (an example is shown in Figure 1). The compression result is transmitted by the network, while decompression is used to recover the files when genomic data analysis is requested by computation nodes. Currently, the volume of genomic data generated by high-throughput sequencing platforms or those downloaded from open-source genomic databases has reached GB or even TB level [16, 17, 18]. Furthermore, a NovaSeq X NGS platform can generate 8 Tbase output in about 17-48 h [18]. It takes minutes, even hours [19, 20, 21], for the current available CPU-based compression algorithms to compressand decompress. For example, the algorithm in [20] used 10 min to compress a 7708 MB genomic file. The long processing time reduces the overall analysis system performance. Therefore, the acceleration of compression and decompression would be beneficial. Online compression and decompression [22; 23; 24] can process data streams simultaneously with their input. FPGA (field-programmable gate array) is known for its support of high parallelism, flexible programming, and excellent real-time processing. These features make FPGA an ideal platform for implementing online compression algorithms. High parallelism helps increase the processing speed. Flexible programming enables special designs for genomic data to process input data faster and consume fewer power and hardware resources, which can help FPGA-based implementations easily embed in edge devices with limited resources and collaborate with existing FPGA-based data analysis algorithms [7; 8; 9; 10; 25; 26; 27]. FASTQ is a commonly used file format [17; 19; 20; 21; 28; 29], with '.fastq' as its suffix, for genomic data. Compression algorithms for FASTQ files can be categorized as either lossless or lossy. Although lossy compression can achieve higher compression ratios, it loses information after compression. This paper focuses on lossless compression for FASTQ files to maintain all the valid information. LZ4 [30] and Zstd [31] are general lossless compression algorithms designed for fast compression, and they can be used to compress FASTQ files. The basic idea of such algorithms is to use contextual redundancy in a sliding window to achieve compression. However, the size of the sliding window is limited. The structural characteristics of FASTQ files determine that sliding windows often fail to capture much of the contextual redundancy. Several lossless compression methods specifically designed for FASTQ files have been proposed. A reference probabilistic de Bruijn Graph, built de novo from a set of reads and stored in a Bloom filter, was used to compress sequencing data in [19]. [PERSON] designed SCALCE, a 'boosting' scheme based on the Locally Consistent Parsing technique, to achieve FASTQ compression in [20]. [PERSON] developed a FASTQ compressor with multiple compression modes called SPRING in [21]. These algorithms are performed by CPU and can achieve a good compression ratio. However, their compression processes suffer from low throughput rates and high CPU occupation. It usually takes minutes (even hours), large memory, and multiple CPU cores for them to compress GB-level FASTQ files. Therefore, these algorithms can not achieve online compression.