Comparison of state-of-the-art error-correction coding for sequence-based DNA data storage

Domain: biology
Published: 2026-03-14
Editorial outcome: major_revision

Abstract
Many codecs with different error-correction approaches have been implemented for DNA data storage to date. However, no studies have systematically benchmarked codec implementations to establish their current state-of-the-art. Here, we use in silico and in vitro experiments to compare the performance of six representative codecs from literature. In isolation, these codecs can tolerate error rates up to 14% and a sequence loss of 65%. Under realistic conditions, we further establish that storage densities as high as 117 EB g-1 are feasible using existing codecs and current synthesis and sequencing technologies. Verifying our results experimentally, we demonstrate data storage at 43 EB g-1 using synthesis by material deposition and 13 EB g-1 using electrochemical synthesis, employing existing codecs from literature. Besides closing in on the physical limits of DNA data storage, this study thus demonstrates the maturity of error-correction coding, defines its current state-of-the-art, and establishes best practices for codec benchmarking.

Introduction
Since the inception of DNA data storage, a major challenge has been defining the rules for reversibly converting between digital information and DNA sequences. This challenge, which falls within the realm of coding theory for DNA data storage, mainly involves the design and implementation of effective encoders and decoders (“codecs”). For this, early demonstrations of DNA data storage focused on source coding (i.e., compression) to efficiently encode the mostly text-based data used at the time,1 e.g., by run-length encoding2 or Huffmann codes.3,4 Then, with the advent of array-based DNA synthesis and next-generation sequencing, channel coding came into focus to enable error-free recovery of binary data and overcome the necessity for manual intervention during decoding.5–7 Still today, the majority of codecs employ linear block codes such as Reed-Solomon7–9, Fountain10,11, or repetition6,12 codes with an inner/outer code separation strategy for error correction (see Supplementary Table 2 for an overview).1

For sequence-based DNA data storage with Illumina sequencing, which has been used for the largest demonstrations of DNA data storage to date,9,13,14 the use of channel coding is necessitated by the peculiar challenges posed by this workflow. First, array-based DNA synthesis is limited to sequence lengths of only a few hundred nucleotides (nt), requiring data segmentation across many individual sequences in an oligonucleotide pool.13,15 Secondly, all biochemical steps of the DNA data storage workflow (see Fig. 1a) introduce errors into sequences and affect their distribution, potentially leading to sequence loss.16,17 Thus, codecs must simultaneously compensate for nucleotide errors and sequence dropout, motivating the widely-used code separation strategy with an inner and outer code.1,13 Beyond these basic considerations however, the breadth of intended applications and available technologies for DNA data storage each present individual challenges and requirements. Accordingly, many codecs are designed to support specific applications’ error profiles (e.g., photolithographic synthesis,18 aging-induced decay,19,20 or nanopore sequencing9,21,22) or fulfill specific sequence constraints (e.g., GC content,6,10 homopolymers,5,7,10 k-mer frequency11,23, motifs11, or free energy24).

Besides the logical redundancy introduced by codecs, the inherent presence of many sequence copies during biochemical processing provides an additional layer of redundancy in DNA data storage, termed physical redundancy.13,16 However, achieving DNA’s extreme storage densities (theoretically up to 227 EB g-1 for double-stranded DNA, see Supplementary Note 2) requires minimizing both logical and physical redundancy simultaneously, while maintaining sufficient redundancy to facilitate error-free decoding.25 As a result, while logical redundancy is directly related to synthesis cost, only the product of logical and physical redundancy is relevant for the storage system’s data density. Complicating matters further, sequencing also yields multiple reads per sequence, providing another source of redundancy through sequencing depth.13,16 Evidently, there exist a trade-off between these forms of redundancy which is not reflected in the encoder’s code rate (i.e., data bits stored per nucleotide). Thus, any attempt at isolating the performance of a codec from these other sources of redundancy is futile without sufficient standardization of experimental conditions.

Nonetheless, codecs are still commonly evaluated across studies by simply comparing the level of logical redundancy used (i.e., the encoder’s code rate), despite the often vastly different experimental conditions (e.g., synthesis provider, physical redundancy, sequencing depth).11,24,26 In recent years, codecs have also been increasingly compared through in silico simulations that artificially vary the error rate in their input data.11,23,24 However, these comparisons lack an established baseline and the extent of their standardization (e.g., code rate, sequence length, file size) often remains unclear. As a notable exception, Ping et al.24 were the first to evaluate two codecs in a standardized experiment, using a serial dilution as a benchmark. Nonetheless, the absence of a generally accepted state-of-the-art and standardized benchmarks with experimental relevance currently impedes objective assessment of codec performance.

In this study, we systematically benchmark codecs for DNA data storage both in silico and in vitro to establish the current state-of-the-art. For this, six representative codecs selected from the literature in October 2023 (see Table 1) were tested in multiple standardized scenarios across a range of experimental conditions centered around the most common, sequence-based data storage workflow with Illumina sequencing.13,14 For our in silico benchmarks, we first employ simple synthetic error scenarios (i.e., applying fixed error rates) commonly used in the literature. Then, we extend our analysis with comprehensive benchmarks considering the full spectrum of non-idealities present in real-world sequencing data (using DT4DDS17), and verify our results with standardized in vitro experiments. In doing so, we demonstrate the benefits of read clustering on codec performance, assess the transferability of synthetic benchmarks to realistic scenarios, and evaluate the limits of common experimental benchmarks. We establish and verify an experimental benchmark for codec performance, demonstrating the capabilities of existing codecs to achieve record-breaking storage densities. Our work presents an unbiased and standardized assessment of the current state-of-the-art in error-correction coding for DNA data storage, thereby providing both a suitable baseline and a framework for benchmarking for future studies.

Results
Selecting and standardizing codecs for benchmarking
For this study, we limited our benchmarking of codec performance to representative examples, using the availability of an open-source implementation with sufficient documentation, the presence of in vitro experiments in the original publication, the prominence in literature, and considerations for covering a broad range of approaches as selection criteria (see Methods and Supplementary Table 2, cut-off date October 2023). As a result, we selected DNA-Aeon by Welzel et al.11, DNA Fountain by Erlich and Zielinski10,27, DNA-RS by Heckel18,28,29, an implementation of the codec used by Goldman et al.6 (“Goldman”), HEDGES by Press et al.30, and Yin-Yang by Ping et al.24 (see Table 1 for an overview, and Supplementary Table 2 for a list of candidate codecs). All codecs were standardized to facilitate impartial performance comparisons by choosing their parameters – as far as possible – such that they yield similar code rates (0.50, 1.00, and 1.50 bit nt-1) and sequence lengths (around 150 nt).

Importantly, codec parameters were not optimized specifically for this study. Instead, the parameters selected by the codecs’ original authors were only adjusted to achieve standardized code rates and similar sequence lengths. This effectively changed only each codec’s overall redundancy level (via the code rate), whereas other codec features – such as the balance between inner and outer redundancy – were kept at approximately the level selected by the original authors. While this approach might lead to suboptimal codec configurations in our benchmarks, this approach mimics the use case for experimentalists and matches each codec’s authors’ selected implementation most closely. This was deemed important, as the codecs’ original authors were assumed to have estimated their ideal parameters based on their specific use cases in DNA data storage. These selected parameters are listed in Supplementary Tables 5-9. Minor changes to some codec implementations were also required to facilitate automated testing and ensure representative performance (see Supplementary Note 1).

For an initial comparison, a basic error scenario was implemented, analogous to the most common synthetic performance benchmarks in the literature.11,23,24,30 This scenario randomly introduces single-nucleotide errors to create 30 erroneous copies of each sequence generated by a codec, at a variable rate and specified composition (53% substitutions, 45% deletions, and 2% insertions, resembling the error pattern in Ref. 17). Using this approach, a broad range of error rates were iteratively sampled in order to identify the error rate at which decoding started to fail (for a total of 30 decoding attempts per condition, see Fig. 1b and Methods). Then, as our standard metric of error tolerance, we report the error rate at which decoding still succeeded with 95% probability, based on a logistic regression of all 30 decoding outcomes (solid lines in Fig. 1b).

Notably, decoding was constrained to one hour, 8 GB of memory, and one CPU core per attempt, using an input file of 19 kB. The choice of time constraint and file size were based on limitations of the employed computing environment, and thus indirectly enforced a decoding speed of at least 5.4 bytes per second. While this minimum enforced decoding throughput was thus not based on practical considerations, it is nonetheless sufficiently permissive such that any codec which is limited by this constraint would be unsuitable for most applications of DNA data storage (e.g., cold storage of large archives). Accordingly, the time constraint did not limit decoding in the vast majority of cases, and we show the decoding performance relative to the runtime in Supplementary Figs. 4, 6, 9, 11, 12, and 13. Moreover, the constraint to one CPU core negated any undue disadvantage due to a lack of parallelization in a codec’s implementation. However, this constraint thus overlooks inherent incompatibilities with parallel processing possibly present in some decoding algorithms, which represent a considerable disadvantage to their large-scale implementation.

Fig. 1d shows the error tolerance of all codecs in this basic error scenario, without clustering and consensus generation by alignment (i.e., the set of all 30 erroneous reads per sequence are supplied to the decoder directly, following the procedure by Erlich et al.10; shown as “Naïve”, grey bars). Unsurprisingly, lower code rates (i.e., higher redundancy) led to increased error tolerances for all codecs. However, the time constraint limited decoding performance in several runs, especially for the DNA-Aeon codec (symbolized by ⧖ in Fig. 1d, see also Supplementary Fig. 4). As a result, DNA-Aeon failed to decode the data within the time limit at error rates above 0.3%, irrespective of code rate. In contrast, the HEDGES codec, at 0.63 bit nt-1, exhibited the highest error tolerance at 7.7%, more than double that of the runner-up, DNA-RS with 3.3% at 0.50 bit nt-1. Thus, based only on this commonly used synthetic performance benchmark, the HEDGES codec would be considered the best-performing codec of our selection from the literature.

Testing the benefits of read clustering on codecs’ error tolerance
A codec’s decoding capability is directly related to the balance between available redundancy and error frequency in the data. Contrary to most traditional coding channels however, the sequencing data used for decoding in DNA data storage is inherently replicated, i.e., multiple erroneous reads of each sequence are available. This inherent repetition code can be exploited by generating a less erroneous consensus sequence from the individual reads via clustering (see Fig. 1a). Therefore, read clustering promises to increase a codec’s error tolerance without the need for additional logical redundancy, which is surprisingly seldomly exploited in the literature (see Table 1).

To assess the benefits of clustering on codec performance, we selected both established clustering algorithms from bioinformatics (CD-HIT, MMseqs2, Starcode)31–33 as well as specialized DNA data storage clustering algorithms (LSH, Clover)18,34 from the literature. First, parameters for all clustering algorithms were selected which maximized their sensitivity, accuracy, and specificity (see Methods and Supplementary Table 1). Then, each codec was paired with each clustering algorithm to assess compatibility and quantify improvements in error-correction capabilities. For this, the basic error scenario – the introduction of random errors into 30 sequence copies – was reused.

Comparing the previously established error tolerances without clustering (“Naïve”, grey bars in Fig. 1d) to the best-performing clustering algorithm for each codec (colored bars in Fig. 1d) revealed that clustering and consensus generation improved codec performance in all cases. On average, tolerated error rates increased by 6.5±2.5% in absolute terms, effectively more than doubling the error tolerance of most codecs. As expected, this positive effect on error tolerance was caused by a drastic reduction in the error rate within the clustered reads (as low as 0% up to an error rate of 5%, see Supplementary Figs. 1 and 2), thereby considerably reducing the effective error rate for the decoder. Our results match previous results on the error-correction capability of consensus generation,35 and thus highlight the benefits of exploiting the inherent redundancy in sequencing data. The latter is best illustrated by the performance of the Yin-Yang codec, an optimal bit-to-base coding scheme without error-correction capabilities:24 it exhibited an error tolerance of 4.2% while relying solely on the indirect error correction conveyed by clustering. Specifically, the clustering step already completely eliminates all errors in the consensus reads up to this error rate (see also Supplementary Fig. 1), removing the need for any codec-level error correction for error-free data recovery. Thereby, its performance represents the baseline for every codec with additional error-correction capabilities in this basic error scenario.

Across all codecs, the tolerated error rate with clustering often exceeded 5%, well above the error rates commonly encountered after commercial synthesis and in common workflows.16,17,25 Additional tests considering each error type in isolation (see Supplementary Figs. 8-10) also did not expose any major differences in error tolerance between error types. Notably, the performance of DNA-Aeon improved most drastically with clustering, achieving an error tolerance of 7.7% at 1.50 bit nt-1. Evidently, the reduced workload conveyed by clustering lifted its limitation by the time constraint drastically (see Supplementary Fig. 4). This highlights another benefit to read clustering besides the exploitation of sequencing data’s inherent redundancy: the acceleration of decoding pipelines through the reduction of codec workload by 1-2 orders of magnitude (depending on sequencing depth).

Interestingly, nine of the thirteen tested codecs and code rates performed best with the established general-purpose clustering algorithm CD-HIT (red bars in Fig. 1d). Unsurprisingly, this general preference for CD-HIT was caused by this clusterer’s superior performance, yielding the fewest and least erroneous clusters across all tested clustering algorithms up to an overall error rate of around 10% (see Supplementary Figs. 1-3). In addition, several combinations of clustering algorithms and codecs failed completely (see Supplementary Table 3 for full results). Further investigation suggests incompatibilities exist between these clustering algorithms and the sequence features generated by some codecs (e.g., indexing regions, overlapping sections in the Goldman codec). Specifically, some codec-clustering pairings exacerbate sequence loss by clustering reads from different design sequences together. In some cases, up to 92% of sequences present in the sequencing data no longer appeared in the clustered reads, explaining the incompatibility of such codec-clustering pairings. Thus, the ideal clustering algorithms must reduce the error rate in consensus reads efficiently, thereby reducing the workload for the inner decoder, without causing sequence loss themselves, which drastically burdens codecs’ outer decoders. Given the universal benefits of clustering identified above, the pairing between codecs and clustering algorithms established in Fig. 1d were used for all further in silico studies and in vitro experiments.

Evaluating codecs’ tolerance to errors and sequence dropout simultaneously
Errors within the DNA sequence are not the only type of fault occurring in the DNA data storage channel. Also sequence dropout, i.e., the absence of reads from some sequences in the sequencing data, requires correction by a codec.13,16 However, codecs’ tolerance to sequence dropout is rarely quantified in the literature, especially in combination with variable error rates. Thus, we extended the aforementioned basic error scenario with another variable, the fraction of sequences lost. As such, Fig. 1c shows the rate of sequence loss each codec tolerates in the absence of errors. The large differences in tolerated sequence loss between codecs and code rates – from 0% (Goldman and Yin-Yang) to 64% (DNA Fountain at 0.5 bit/nt) – highlight that codecs are not equally equipped to compensate for this unique problem in DNA data storage.

The addition of sequence dropout as a second error source also enabled the simultaneous quantification of codecs’ tolerance towards errors and sequence dropout, uncovering any considerable tradeoffs. Fig. 1e shows the resulting Pareto fronts, delimiting each codec’s feasible region for the simultaneous correction of errors and sequence dropout.

In accordance with the previous results, lower code rates (i.e., higher redundancy) considerably extended each codec’s feasible regions in Fig. 1e, by increasing their tolerances to both errors and sequence dropout. However, the extent to which each codec tolerated sequence dropout, even at minimal error rates, differed considerably. Especially HEDGES, previously identified as best-performing codec based on raw error-correction capability, only tolerated up to 7.8% and 10.5% sequence dropout at 1.07 bit nt-1 and 0.63 bit nt-1, respectively. This improves only slightly on the 7.9% sequence dropout tolerated by the Goldman codec, with its basic repetition code at 0.34 bit nt-1. In contrast, DNA-Fountain – including only a small RS code within each sequence to detect rather than correct an erroneous sequence10 – previously exhibited a low error-correction capability (up to around 5% nt-1) across all code rates. However, its tolerance to sequence dropout increased drastically from 3% at 1.50 bit nt-1 up to 63% at 0.50 bit nt-1. Evidently, each codec’s tolerances to errors and sequence dropout are balanced differently, such that neither metric in isolation can be considered as a robust performance benchmark.

The runtime constraint also limited codec performance of DNA-Aeon, DNA-RS, and HEDGES in this scenario (see Supplementary Fig. 11). Interestingly, both DNA-Aeon and HEDGES were only limited by time at high error rates while operating unconstrained at high sequence dropout, suggesting their implementations of within-sequence redundancy caused excessive runtimes. As previously, memory use did not limit any codec in this scenario.

In showcasing each codec’s balance between correction of errors and compensation for sequence loss, Fig. 1e also highlights the benefits of a balanced approach to redundancy. The DNA-RS codec – and DNA-Aeon to a lesser extent – dominated the feasible regions of DNA Fountain and HEDGES, unless extreme sequence dropout (> 50%) or error rates (> 12%) occurred. Considering additionally that especially DNA-Aeon (at all code rates), but also DNA-RS (at 0.50 bit nt-1), were partially limited by the time constraint (see Supplementary Fig. 11) at these extremes, these two codecs emerge as broadly superior in this synthetic analysis of errors and sequence dropout.

Benchmarking codecs with common literature experiments
While the synthetic benchmarks used above clearly showcased each codec’s theoretical capabilities under synthetic conditions, they are only a simplified representation of DNA data storage’s true error channel. Despite the common use of these simple synthetic benchmarks for codec benchmarking in the literature, they overlook the non-ideal error patterns and biases present in experimental sequencing data – from error runs to skewed coverage distributions.16,17,36 Thus, for all further benchmarks, we implemented realistic experimental workflows using the models implemented in the simulation software DT4DDS17 (see Methods). This simulation software takes into account the aforementioned non-idealities which are missing from the synthetic benchmarks previously performed in the literature.

As a first realistic benchmark, three common experiments used by Erlich and Zielinski10, Organick et al.9,25, and Ping et al.24 were replicated in silico. The three chosen scenarios covered the most common experiments in DNA data storage: a serial dilution to assess maximum storage density, a serial amplification to demonstrate copyability, and read down-sampling to quantify minimum sequencing depth (see Fig. 2).

In accordance with the original studies’ results (gray lines in Fig. 2), our standardized, in silico replications of these literature benchmarks showcased the possibility of data recovery at high storage density and after deep replication. However, the serial dilution and serial amplification benchmarks exposed only minor performance differences between codecs at best (blue and orange bars in Fig. 2). In the serial dilution benchmark, the iterative 10-fold dilutions only partially resolved any differences between code rates of the same codec. This is expected, given that this benchmark uses oligo pools synthesized by material deposition (i.e., Twist Bioscience), whose high homogeneity and low error rate pose few challenges to codecs even at physical redundancies as low as 10x.25 In the serial amplification benchmark, deep replication of these oligo pools by a high-fidelity (i.e., low-error) polymerase thus expectedly also did not lead to any discernable difference between codecs. Evidently, all codec implementations – with the exception of the Goldman codec – are similarly performant in these two literature benchmarks under equal and controlled conditions. This renders these two literature benchmarks ill-suited for comparing codec performance experimentally.

In contrast to the other two literature benchmarks, the down-sampling of sequencing data did expose considerable differences between codecs (green bars in Fig. 2). Given the low error rates associated with synthesis by material deposition,17 this benchmark relies mostly on tolerance to sequence dropout rather than error correction, thereby it is expected to favor codecs such as DNA Fountain. However, both DNA-Aeon and DNA-RS outperform DNA Fountain considerably across all code rates, requiring as few as only one sequencing read per sequence on average at 0.50 bit nt-1 (i.e., a sequencing depth of 1). Ostensibly, the low sequencing depth in this benchmark negated the benefits from clustering (as shown by the poor performance of the Yin-Yang codec) and thus enforced efficient use of the available read data. These results suggest read down-sampling is the only experimental benchmark sufficiently informative for codec comparisons out of the three literature experiments tested.

Benchmarking codecs across common experimental workflow choices
From an experimental perspective, three parameters matter most for any DNA data storage workflow: the synthesis provider, the number of oligos per sequence present during storage (“physical redundancy”), and the number of sequencing reads per sequence (“sequencing depth”).13,16,17 While the choice of synthesis provider is usually well-documented in literature studies and its effect on error rate well-established, the choices of physical redundancy and sequencing depth vary considerably across studies and affect only the homogeneity of sequencing reads.13,16,17 Specifically, both low physical redundancies and low sequencing depths increase the likelihood of sequence dropout and diminish the diversity of the oligos due to the stochastic sampling of only few oligo copies. Importantly, physical redundancy directly affects the data’s storage density (i.e., it is inversely proportional to storage density), whereas sequencing depth directly influences sequencing costs.

While the literature benchmarks presented above offer straightforward workflows, they still consider only the latter two parameters in isolation. Thus, we set out to compare the selected codecs across all three experimental parameters directly. First, we implemented two scenarios in DT4DDS (see Methods and Fig. 3b), centered around the two most-commonly used array-based synthesis technologies: material deposition / printing (i.e., Agilent, Twist Biosciences) and electrochemical synthesis (i.e., Genscript/CustomArray).13 The former is included in the high-uniformity, low-error scenario (“high-fidelity”) using a high-fidelity polymerase for amplification and yielding an error rate of around 0.1% with minor sequence loss (<1% without dilution).17 In contrast, the low-uniformity, high-error scenario (“low-fidelity”) includes electrochemical synthesis and amplification by an error-prone polymerase, yielding an error rate of around 1.5% and considerable sequence dropout (>2% without dilution).17 Within these two scenarios, we then varied the physical redundancy and sequencing depth akin to the synthetic scenario in Fig. 1e, thereby simultaneously optimizing for storage density and reading cost.

The Pareto fronts illustrating the tradeoff between physical redundancy and sequencing depth in the two scenarios are shown in Fig. 3a. Focusing first on the high-fidelity scenario (top half of Fig. 3a, average error rate 0.1%), a considerable separation of codecs was observed across all code rates. The Yin-Yang codec, as the baseline without additional error-correction capabilities, showcases how clustering and the inherent repetition in sequencing reads suffice to yield a storage density of 6.6 EB g-1 with a sequencing depth of 30x (32x physical redundancy at 1.85 bit nt-1 code rate, see Fig. 3c). Both DNA Fountain (15 EB g-1 at 7.6x and 1.00 bit nt-1) as well as HEDGES (38 EB g-1 at 3.2x and 1.07 bit nt-1) considerably improved upon this baseline, despite their limited ability to tolerate considerable sequence dropout and large error rates simultaneously. In contrast, both DNA-Aeon and DNA-RS tolerated physical redundancies and sequencing depths below 10x even at a code rate of 1.50 bit nt-1. At a code rate of 0.50 bit nt-1, these codecs required only around one physical copy and one sequencing read per reference sequence on average, in-line with the results in Fig. 2. As a result, both DNA-Aeon and DNA-RS achieved storage densities well above the current state-of-the-art (28 EB g-1, see Supplementary Note 2 and Organick et al.25) in this scenario (about 117 EB g-1 at 30x sequencing depth, see Fig. 3c).

Expectedly, the low-fidelity scenario (bottom half of Fig. 3a, average error rate 1.5%) and the significantly increased error load under these conditions, challenged codecs considerably more. Thus, required sequencing depths and physical redundancies were about 1-2 order of magnitude larger than in the high-fidelity scenario. Surprisingly, the DNA Fountain, Goldman, and Yin-Yang codecs were unable to decode the data in the low-fidelity scenario at all. As these codecs tolerated much larger average error rates in the synthetic benchmarks (see Fig. 1e, dotted line), this observation further questions the transferability of synthetic results to experimental workflows. The other codecs – DNA-Aeon, DNA-RS, and HEDGES – achieved generally similar performance, albeit with slight advantages for DNA-Aeon at 1.00 bit nt-1 and DNA-RS at 0.50 bit nt-1. Nonetheless, the overall performance falls drastically short compared to the storage densities obtained in the high-fidelity scenario, peaking at 17 EB g-1 for DNA-Aeon at 1.00 bit nt-1 (6.7x physical coverage, see Fig. 3c). Additionally, both DNA-Aeon and DNA-RS were partially limited by the time constraint, as shown in Supplementary Figs. 12+13. This suggests that also in this benchmark, high error-correction capability is in a direct trade-off with computational power. Expectedly, memory use did not limit codec performance in either scenario.

Notably, most Pareto fronts in both scenarios feature a clear symmetry with respect to the extremes of physical redundancy and sequencing depth. This suggests down-sampling sequencing data to assess minimum sequencing depth – one of the literature benchmarks tested above – is a good proxy for the minimum physical redundancy required during storage. However, the Pareto fronts in Fig. 3a demonstrate clearly that the validity of this approach relies upon the other parameter being non-limiting. For example, while DNA-RS at 1.00 bit nt-1 exhibited a minimum sequencing depth of 1x in the read down-sampling (see Fig. 2), recovering data with this codec after storage at a 1x physical redundancy would require a sequencing depth exceeding 20x.

Interestingly, the highest storage densities of DNA-Aeon (140 EB g-1, 0.50 bit nt-1, physical redundancy 0.49x) and DNA-RS (125 EB g-1, 1.00 bit nt-1, physical redundancy 0.97x) at 30x sequencing depth were not achieved at the highest code rate tested (i.e., 1.50 bit nt-1). This suggests the major benefit of additional logical redundancy – enabling further reduction in physical redundancy – considerably outweighs the loss in code rate. However, choosing lower code rates increases synthesis costs proportionally, as the price of synthesis scales directly with the number of sequences to be synthesized resp. the amount of data stored (see Supplementary Fig. 20a+b). Given the standardized code rates used in this study, all codecs feature similar costs per bit stored, at 14 USD/kB on average in the high-fidelity scenario and 7.9 USD/kB in the low-fidelity scenario (at a 1 MB scale, see Supplementary Fig. 20c). Ultimately, synthesis cost is not directly defined by the choice of codec, but purely by the employed code rate. To this end, the choice of code rate must ultimately compromise between achieving the highest storage densities at lower code rates and reducing the synthesis costs at higher code rates. Thus, given the direct scaling of synthesis costs with code rate, our performance analysis comparing three standardized code rates also translates directly to anticipated synthesis costs.

Experimental replications demonstrating state-of-the-art storage densities
Experimental replications of the high- and low-fidelity scenarios in vitro were performed to establish the accuracy of our in silico analysis and confirm the validity of our observations regarding storage density. For this, adjusted codec parameters were used to store files with 5 kB, 17 kB, and 19 kB into a total of 11 293 sequences using all six selected codecs. These parameters (detailed in Supplementary Tables 5-9) were chosen to target code rates of 1.00 bit nt-1, 1.50 bit nt-1, and the highest code rate supported by each codec, while adhering to a length limit of 126 nt (a constraint of electrochemical synthesis, see Methods). After synthesis by Genscript and Twist Biosciences, both oligo pools were amplified, diluted, re-amplified, and sequenced as outlined in Fig. 3b for the synthetic scenarios. Figure 4 shows the decoding results of these in vitro replications of the low- and high-fidelity workflows, using five different physical redundancies at a fixed sequencing depth of 30x. The decoding results predicted from our in silico analysis are also shown for comparison (green shading).

Overall, the experimental results (see Fig. 4) closely follow the trends observed in our in silico analysis of the workflows (see Fig. 3a). Expectedly, the low-fidelity scenario proved more challenging than the high-fidelity scenarios across all codecs. Nonetheless, DNA-Aeon and DNA-RS performed best, both capable of recovering the data stored at physical redundancies as low as 2x in the high-, and 5x in the low-fidelity scenario (at a code rate of 1.0 bit nt-1). Taking into account the physical redundancies measured by qPCR (see Fig. 4 and Supplementary Tables 13+14), this equals storage densities of 57 EB g-1 and 17 EB g-1, respectively (43 EB g-1 and 13 EB g-1 including adapters, see Supplementary Table 11). These results not only validate the accuracy of our benchmarks in Fig. 3, but also considerably improve upon previously demonstrated achievable data densities for DNA data storage (26 EB g-1 using high-fidelity workflow by Organick et al.25, 0.033 EB g-1 using low-fidelity workflow by Grass et al.7, see Supplementary Table 11).

High-density data storage in the high-fidelity scenario was also achieved with HEDGES (physical redundancy of 2x, equivalent to 56 EB g-1). However, the HEDGES codec was less capable in the low-fidelity scenario, maxing out at a physical redundancy of 50x (52x by qPCR, equivalent to 2.2 EB g-1). More generally, most codecs failed to decode the data in the low-fidelity scenario at code rates above 1.0 bit nt-1, even at 1000x physical redundancy (see Fig.4, right). The DNA-RS codec is the only exception in our test, achieving successful decoding with 1.50 bit nt-1 down to 10x physical redundancy (12.5x by qPCR, 14 EB g-1). This highlights the ability of the low-fidelity scenario to differentiate codecs by testing their tolerance to both errors and sequence dropout. In contrast, all codecs with error-correction capabilities worked reliably at 1.5 bit nt-1 and 1000x physical redundancy in the high-fidelity scenario.

Figure 4 also shows the probability of successful decoding derived for the experimental conditions from in silico simulations. These decoding probabilities show good agreement with the experimental results, highlighting the overall accuracy of the in silico simulations. However, systematic deviations to experimental results were present in some cases. In the high-fidelity scenario (Fig. 4, left), the capabilities of the DNA-Aeon, DNA Fountain and Yin-Yang codecs were overestimated. In contrast, these simulations were too pessimistic with respect to the HEDGES and Goldman codecs. The predictions in the low-fidelity scenario (Fig. 4, right) – while generally more accurate – tended to slightly overestimate the capabilities of the DNA-Aeon and HEDGES codecs.

To elucidate the source of these systematic deviations, the homogeneity of error rates and sequence dropout in the experimental datasets was assessed. While the rates of errors were similar across codecs, the rate of sequence dropout varied drastically between codecs at the same physical redundancy (see Supplementary Fig. 15). Upon closer inspection, these variations were caused by systematic differences in the coverage homogeneity of each codec’s sequences (see Supplementary Figs. 16+17). Evidently, the sequences generated by some codecs were less homogeneously represented in the oligo pools than others, with coefficients of variation at 1000x physical coverage ranging from 0.38 to 1.15. As all sequences were synthesized on the same chips in a randomized order, this inhomogeneity does not impact the fairness of the experiments. Instead, the differences in pool homogeneity likely resulted from biases during amplification, potentially caused by certain sequence features.37,38 For example, the sequences generated by the Goldman codec were least abundant in both scenarios, at half the mean sequencing depth (see Supplementary Fig. 15). Here, the repetitive elements introduced by the codec could inhibit amplification by enabling secondary structures.39 In-line with this reasoning, its sequences are underrepresented in the sequencing data (see Supplementary Fig. 15), and thus less likely to be sampled during dilution. This explains the increased rates of sequence dropout from these codecs, which cause the systematic deviations to the simulation results (additional discussion is provided in Supplementary Note 3). However, these deviations in copy numbers in the physical pool do not explain the much larger differences between codec performances across physical redundancies in Figure 4.

Extension of benchmarking pipeline to additional literature codecs
Finally, to highlight the utility of the developed benchmarking pipeline for codec comparisons, the scope of literature codecs was extended to cover additional codecs by Song et al. (DBGPS)19, Chandak et al. (LDPC)40 and Zan et al. (Modulation)41. Figure 5 shows the results of the in silico benchmarks for these three additional codecs, with the results by the best-performing codecs from the main study – DNA-Aeon by Welzel et al.11 and DNA-RS by Heckel et al.18,28 – also shown for comparison. In line with the results from Fig. 1d, clustering by CD-HIT drastically improved error tolerance across these additional codecs, see Fig. 5a. Moreover, these codecs’ ability to correct individual error types did not differ from the results of DNA-Aeon and DNA-RS (see Fig. 5b).

Importantly, the Pareto fronts across code rates in Fig. 5c+d show that DBGPS by Song et al.19 performed similarly to DNA-Aeon and DNA-RS in most cases. Thus, its performance is on-par with these state-of-the-art codecs, to within the experimental error associated with the lack of parameter optimization in this study. In contrast, both LDPC by Chandak et al.40 and the modulation-based codec by Zan et al.41 fall behind the performance of DNA-Aeon, DNA-RS, and DBGPS in our in silico benchmarks.

In the case of the modulation-based codec by Zan et al.41, its lack of data-level error correction renders it comparable to the Yin-Yang codec by Ping et al.24 investigated in our initial codec benchmark. Reassuringly, both of these codecs relying solely on clustering and sequence alignment for error correction achieved similar tolerated error rates of around 4-5% in our test (compare Fig. 1d and 5a). However, contrary to the Yin-Yang codec by Ping et al.24, the implementation of the modulation-based codec provided in the repository by Zan et al.41 does not include sequence indexing. As a result, it cannot recover the stored data if sequences are recovered out of order, unless sequence indexing is implemented separately from the codec. Naturally, features such as sequence indexing and an outer code for tolerance to sequence loss can be retrofitted to existing codecs, using established methods like Fountain or Reed-Solomon codes. However, such adaptions to existing codec implementations require additional programming, testing and optimization efforts, which is not realistic for most use cases and was therefore not in scope of this study. These limitations of the modulation-based codec cause the discrepancy in observed performance between Zan et al.41 and the benchmark in Fig. 5.

Taken together, the extension of our in silico pipeline to three additional codecs from the literature demonstrates its simplicity and utility for codec benchmarking, without requiring any experimental effort. In addition, the results obtained for the DBGPS codec by Song et al.19 further support the observation that a well-executed, balanced inner/outer code separation strategy is more important for codec performance than the specific choice of employed error-correction code. Thus, these results showcase both how error-correction codecs for DNA data storage are optimally constructed and how their performance is realistically assessed.

Methods
Selection of codecs and clustering algorithms
The selection of codecs for this study was based on the availability of an open-source implementation with sufficient documentation, the presence of in vitro experiments in the original publication, the prominence in literature, and considerations for covering a broad range of approaches. We therefore selected DNA-Aeon by Welzel et al.11, DNA Fountain by Erlich and Zielinski10,27, DNA-RS by Heckel18,28,29, an implementation of the codec used by Goldman et al.6 (“Goldman”), HEDGES by Press et al.30, and Yin-Yang by Ping et al.24 (without additional error-correction elements). The installation and usage for encoding and decoding of each codec followed the documentation as far as possible. Nonetheless, some minor changes to the implementations of the DNA-Aeon, Goldman, HEDGES, and Yin-Yang codecs were required to either facilitate automated testing or ensure representative performance. These changes are described in Supplementary Note 1 and are available in the code associated with this study.

The selection of clustering algorithms for this study was based on the same criteria as for the codecs. However, due to the sparse availability of suitable clustering algorithms specific to DNA data storage, we mainly considered general purpose clustering algorithm s. We therefore selected CD-HIT by Li et al.31,52,53, Clover by Qu et al.34, clustering based on Locality-Sensititve Hashing by Darestani and Heckel18,54 (“LSH”), MMseqs2 by Steinegger et al.32,55, and Starcode by Zorita et al.33,56. In addition, we implement the naïve clustering approach used for DNA Fountain10 which simply uses the unique sequencing reads sorted by their abundance.

Selection of codec parameters
All of the selected codecs which support adjusting the sequence design and/or the level of error-correction were harmonized with respect to code rate, sequence length, and constraint choice. If supported, three sets of parameters yielding code rates of 1.50 bit nt-1, 1.00 bit nt-1, and 0.50 bit nt-1 at a sequence length of around 150 nt were created for the in silico studies. For the in vitro experiment, due to the constraint on sequence length imposed by one of the synthesis providers (Genscript, 170 nt including adapters), three sets of parameters yielding code rates of 1.50 bit nt-1, 1.00 bit nt-1, and the highest code rate possible, using at most 126 nt, were selected. A detailed list of codec parameters used in this study is provided in Supplementary Tables 5-9.

Benchmarking of clustering algorithms
To compare the selected clustering algorithms individually and select their optimal parameters, two experimental sequencing datasets from a previous study17 (PRJEB65931) were used. To preclude any impact of sequence design on clustering performance, these sequencing datasets were obtained from randomly generated sequences. To test both a high- and a low-fidelity scenario, both an experiment with synthesis by material deposition (ERR12033821) and one by electrochemical synthesis (ERR12033820) were used. For more details, see Ref. 17. Prior to clustering, the paired sequencing reads were merged with NGmerge57 (v0.3) and subsampled to a sequencing coverage of 20 sequencing reads per reference sequence using seqtk58 (v1.4). After clustering, individual clusters were aligned using Kalign59 (v3.4.0) to yield consensus sequences.

Besides clustering speed, three other performance metrics were calculated by comparing the consensus sequences after clustering to the reference sequences. To do so, each consensus sequence was assigned to a reference sequence by minimizing their Levenshtein distance. First, sensitivity was defined as the fraction of reference sequences which still had at least one associated consensus sequence after clustering. Second, accuracy was defined as the mean Levenshtein similarity of the closest match to each reference sequence. Third, specificity was defined as the ratio of the number of reference sequences with at least one associated consensus sequence relative to the total number of consensus sequences.

Setup of the simulation pipelines for codec evaluation
Each simulation pipeline consisted of the steps outlined in Fig. 1a: encoding, workflow, clustering, and decoding. For encoding, a set of fixed parameters (see below) for each codec was used to encode a 19 kB binary file with random content into DNA sequences (“reference sequences”). The workflow, either a script to introduce random errors at fixed rates or a workflow implemented in DT4DDS17,60 (v1.1, see below), then generated simulated sequencing reads from the reference sequences. These sequencing reads were then clustered by a specified clustering algorithm (see above), and the clusters were aligned individually using Kalign59 (v3.4.0) to yield consensus sequences. Finally, the consensus sequences were provided to the codec together with any supplemental data if needed, in order to attempt decoding. Decoding success was assessed by byte-by-byte comparison to the original input file, with only complete recovery of the data being considered as successful decoding.

In all cases, the consensus sequences generated after clustering were padded or trimmed to the length of the reference sequences. This was necessary as several codec implementations were incompatible with sequences shorter or longer than those originally designed. Moreover, workflows using DT4DDS17 (see below) – thus yielding paired sequencing reads – employed NGmerge57 (v0.3) for read merging prior to clustering.

To automate the process of preparing and running the simulation pipelines, management tools and wrapper scripts were written in Python (v3.11) using BioPython (v1.84), scipy (v1.14.1), statsmodels (v0.14.1), numpy (v2.0.2), pandas (v2.2.3), plotly (v5.24.0), psutil (v6.0.0), RapidFuzz (v3.10.0), bamboost (v0.8.0), and h5py (v3.12.1) under Ubuntu 22.04 LTS.

Computational constraints
All simulation pipelines were run on the Euler cluster operated by the High-Performance Computing group at ETH Zürich, using the Slurm workload manager. Each pipeline was constrained to one core of an AMD EPYC 7763 CPU (2.45 GHz nominal, 3.50 GHz peak), 8 GB RAM (DDR4, 3200 MHz), and 2 GB of temporary disk space. Each individual step of a pipeline was further limited to one hour of runtime programmatically. If any constraint was violated throughout the pipeline, decoding was considered unsuccessful.

Definition and estimation of decoding probability
In order to assess the resilience of a chosen selection of codec, parameters, and clustering algorithm towards any of the workflow’s experimental parameters, a one-dimensional sensitivity analysis was performed (see below). Using the binary outcomes of this analysis (i.e., decoding success at each parameter choice) as dependent variables, a logit model was fitted to estimate the parameter value at which the probability of successful decoding would equal 95%. This value was then considered the performance threshold for the chosen selection of codec, parameters, and clustering algorithm.

One-dimensional sensitivity analysis
Each one-dimensional sensitivity analysis was performed for a single parameter of a workflow and over a specified parameter range. To increase the accuracy of the estimated performance threshold, the sensitivity analysis was performed in three stages. In the first stage, ten logarithmically-spaced points were chosen across the full parameter range and tested. In the second stage, the ten outcomes of the first stage were used for a rough estimation of the performance threshold 𝑡 (see above), and ten additional logarithmically-spaced points were selected from the range [𝑡/2, 2𝑡] and tested. In the third stage, the procedure of the second stage was repeated using the points from both the first and the second stage. The complete set of thirty binary outcomes were then used to generate the final estimate of the performance threshold (see above).

Generation of Pareto fronts
In order to enable sensitivity analyses across two parameters of a workflow, two sets of one-dimensional sensitivity analyses were performed. In each set, one of the workflow parameters was fixed to one of ten logarithmically-spaced values selected from the specified parameter range, while the other was varied according to the three-stage process outlined above. From the resulting combinations of feasible parameter thresholds, only the Pareto efficient points were used to generate Pareto fronts.

In silico experiments
Pairing of codecs with clustering algorithms
For the pairing with codecs, only the best-performing parameter sets of each clustering algorithm from the individual benchmarking (see above) were considered. Each clustering algorithm was then tested with each codec in the basic error scenario, as described in detail in the following section. The clustering algorithm with the best performance threshold for each codec was then used throughout the rest of the study. The pairings are provided in Supplementary Table 3.

Evaluation of codec performance in basic error scenarios
A workflow was implemented which introduced specific error types at specified rates randomly throughout the reference sequences, generating 30 erroneous reads per sequence. Considered error types were substitutions (with equal probability for all substituting nucleobases), insertions (with equal probability for all inserted nucleobases), deletions, mixed-errors (at a fixed ratio of 53:45:2 substitutions:deletions:insertions, resembling the error pattern in Ref. 17), and sequence dropout (i.e., the removal of a fixed proportion of reference sequences from the reads). All error types were considered in a range of 0.1-40% nt-1, and sequence dropout used a range of 0.5-99%. This workflow was used for the one-dimensional sensitivity analyses and the Pareto fronts in Fig. 1.

Evaluation of codec performance in literature experiments
The serial dilution and serial PCR experiments used in the study by Erlich and Zielinski10, as well as the read down-sampling experiment by Organick et al.9, were replicated as workflows using DT4DDS17. All workflows assumed high-fidelity synthesis by material deposition, and the use of a high-fidelity polymerase. Sequencing depths were adapted to reflect the average sequencing depths reported in the original studies. Contrary to the other workflows, the comparison of codecs in these replicated literature experiments did not use the sensitivity analysis approach described above. Instead, each iteration of the workflow was tested ten times and the highest iteration count with which all ten tests were successfully decoded was reported.

Evaluation of codec performance in high- and low-fidelity scenarios
Both a high- and a low-fidelity version of a generic data storage workflow, following the definitions for the best- and worst-case in Gimpel et al.17, were implemented using DT4DDS17. These scenarios are outlined in Fig. 3b. In short, the simulated workflows include synthesis by material deposition or electrochemical synthesis, amplification for 15 cycles with either a high- (Q5) or low-fidelity (Taq) polymerase, dilution to a specified mean physical coverage, re-amplification for 25 cycles, and sequencing at a specified sequencing depth using paired-end, 150 nt reads of an iSeq 100. For both the high- and low-fidelity scenario, the mean physical redundancy after dilution and the sequencing depth were varied (see above) to yield the Pareto fronts in Fig. 3a.

Replication of the in vitro workflows
To compare the experimental data (see below) with the simulated results from the in silico analysis, the high- and low-fidelity scenarios used previously (see above) were adapted to use the adjusted codec parameters (see above) and match the workflow employed in the in vitro experiment. Specifically, the deletion rate during synthesis in the low-fidelity scenario was decreased to 0.0044 nt-1, the insertion rate during synthesis increased to 0.0010 nt-1, and the number of PCR cycles increased to 23 and 29 in the first and second round of amplification, respectively. These adapted workflows were used as described above in a one-dimensional sensitivity analysis, varying the physical redundancy after dilution to yield the decoding probability as a function of physical redundancy.

Experimental replication of high- and low-fidelity scenarios
The high- and low-fidelity scenarios outlined above were recreated experimentally with identically composed oligo pools synthesized by Twist Biosciences (South San Francisco, CA, United States) and Genscript (Piscataway, NJ, United States), using established protocols for DNA data storage.28 As Genscript only supported a total sequence length of 170 nt including PCR adapters, the parameters of all codecs were adjusted (see above) to limit the reference sequences to 126 nt, leaving 41 nt for truncated Illumina TruSeq adapters and 3 nt for a codec-specific suffix (see Supplementary Fig. 14). In addition, to limit the number of sequences to be synthesized, the parameter sets yielding code rates of 0.50 bit nt-1 were replaced with parameter sets yielding the maximum code rate supported by each codec. Multiple versions of a compressed image of ETH Zürich’s main building (created by ETH Zürich / Gian Marco Castelberg) were used as input files, with either 5 kB (Goldman codec), 17 kB (all codecs at 1.00 bit nt-1), or 19 kB (all others). As the sequence length varied between codecs, shorter reference sequences were padded with random nucleotides (see Supplementary Table 10 and Supplementary Fig. 14). In total, 11 293 sequences across all codecs were created, padded and indexed (up to 129 nt), supplied with PCR adapters (for a total of 170 nt), and their order randomized prior to being ordered for synthesis.

Pool preparation
The oligo pools ordered from Twist Biosciences and Genscript were handled and amplified individually. The oligo pool by Twist Biosciences, received dry, was resuspended to 10 ng µL-1 with ultrapure water. To create a master pool, 1 µL of a 5000x dilution of the oligo pool was then amplified with 10 µL Q5 High-Fidelity polymerase master mix (New England Biolabs, M0492S), 1 µL of 10 µM 0F and 0R primers each (Microsynth, Balgach, Switzerland, see Supplementary Table 12), and 7 µL ultrapure water, replicated in a total of 96 wells. Thermocycling followed established protocols,28 using an initial denaturation at 95 °C for 3 min, followed by 15 cycles at 95 °C for 15 s, 54 °C for 30 s, and 72 °C for 30 s.

The oligo pool by Genscript, received as a solution, was diluted to 5 ng µL-1 with ultrapure water. To create a master pool, 1 µL of the diluted oligo pool was then amplified with 10 µL KAPA SYBR FAST polymerase master mix (Sigma-Aldrich), 1 µL of 10 µM 0F and 0R primers each (Microsynth, see Supplementary Table 12), and 7 µL ultrapure water, replicated in a total of 96 wells. Thermocycling followed the aforementioned protocol for 23 cycles.

For both oligo pools, each pool’s wells were then combined and purified (DNA Clean & Concentrator-5, ZymoResearch). To increase purity further, each pool was then run on an agarose gel (E-Gel EX Agarose Gels 2%, Invitrogen) and the appropriate bands excised and purified (ZymoClean Gel DNA Recovery Kit, ZymoResearch). Finally, the pools were dialyzed (0.025 µm, 25 mm VSWP membrane, MF-Millipore) for 4 hours against ultrapure water. Concentration was measured by fluorescence (Qubit dsDNA HS Kit, Invitrogen) and by spectrophotometry (NanoDrop, Thermo Scientific).

The concentration of the master pool prepared from Twist Biosciences was measured as 49.4 ng µL-1 (Qubit) and 51.5 ng µL-1 (NanoDrop) respectively. Its concentration was therefore averaged to 50.45 ng µL-1 for all further experiment. The concentration of the master pool prepared from Genscript was measured as 20.6 ng µL-1 (Qubit) and 18.2 ng µL-1 (NanoDrop) respectively. Its concentration was therefore averaged to 19.40 ng µL-1 for all further experiments.

Dilution and quantification by qPCR
Dilution to specified physical coverages (1000x, 50x, 25x, 10x, 5x, and 2x) was performed starting from the master pool. To convert from specified physical redundancy to required concentration, 11 293 sequences of dsDNA with 170 nt were assumed, yielding a physical redundancy of 509074x per ng. All dilutions were prepared such that 5 µL contained the required mass corresponding to the specified physical redundancy 𝑟, e.g., the concentration 𝑐 was selected as 𝑐 = 𝑟 / (5 μL ⋅ 509074 ng−1). Dilutions were performed serially while limiting the dilution factor to below 100x in each step, thereby maximizing dilution accuracy.

Calibration curves for qPCR were set up for each master pool individually, using serial dilutions of the master pools spanning a range from 0.05 ng µL-1 (equivalent to a coverage of around 130 000 per 5 µL) to 5·10-8 ng µL-1 (coverage of 0.13 per 5 µL). qPCR used the same thermocycling settings as above, using 5 µL sample with 10 µL KAPA SYBR FAST polymerase master mix (Sigma-Aldrich), 1 µL of 10 µM 0F and 0R primers each (Microsynth, see Supplementary Table 12), and 3 µL ultrapure water, measured in duplicates. The calibration curves are shown in Supplementary Fig. 18. The qPCR results of all prepared dilutions, calibrated against a standard from the calibration curve measured in parallel, are given in Supplementary Tables 13 and 14.

Sequencing
Sequencing preparation followed established protocols.17,28 In short, 5 µL of each dilution was amplified for 20 cycles (samples of Genscript and Twist pool with 1000x physical redundancy), 26 cycles (only Genscript pool, 50x and 25x physical redundancy), or 29 cycles (all others) with 10 µL KAPA SYBR FAST polymerase master mix (only Genscript pool, Sigma-Aldrich) or 10 µL Q5 High-Fidelity polymerase master mix (only Twist pool, New England Biolabs, M0492S), 1 µL of 10 µM 2FUF and indexed 2RIF primers each (Microsynth, see Supplementary Table 11), and 3 µL ultrapure water. Amplified samples were run on an agarose gel (E-Gel EX Agarose Gels 2%, Invitrogen) and the appropriate band excised and purified (ZymoClean Gel DNA Recovery Kit, ZymoResearch) prior to quantification by fluorescence (Qubit dsDNA HS Kit, Invitrogen). All purified samples were individually diluted to 1 nM and pooled. The combined samples, diluted to 50 pM, were added to an iSeq 100 i1 Reagent v2 cartridge (Illumina) for 150 nt paired-end sequencing.

Analysis of error rates and coverage biases
Sequence coverage in all sequencing datasets was assessed by read mapping with BBMap61 (v39.01). Error analysis of the sample with 1000x coverage was performed as outlined in Gimpel et al.17 using the tools implemented in DT4DDS17. Error analysis of the sets of sequences belonging to a single codec at a specified code rate were performed by using only the reference sequences of that codec for mapping with BBMap.

Evaluation of codec performance in the in vitro experiment
The sequencing reads corresponding to each sample with a specified physical coverage were separated into subsets for each codec and code rate by filtering with BBMap61. After separation, each set of sequencing reads was randomly downsampled to a sequencing depth of 30 reads per reference sequence ten times. These sampled sequencing reads were used as input to the decoding pipeline consisting of a codec-specific clustering algorithm (see above) and the decoding step of the codec itself. Decoding success was assessed by byte-by-byte comparison to the original input file used for encoding.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "DNA storage is emerging as a potential alternative to the challenge of exponentially increasing data. One major topic is how to effectively retrieve information form the IDS error caused by DNA synthesis, PCR and sequencing process. and the dropout issue in the random sampling process is also a critical issue to be solved. Up to now, many methods have proposed to tackle these problems. this paper intended to make a comprehensive comparison of some famous codec to evaluate their performance. Although this work is very meaningful for us to understand the real performance of these methods, there exist some issues need to be solved:\n1. the dropout issue is a unique problem in DNA storage in the random sampling process, it is relative easy to solve by the fountain code which is very mature in tradition communication society. And any codec could use it to solve this issue. therefore, the key issue should be the IDS error correction.\nmy suggestion is to separate the evaluation of IDS error correction capability and dropout issue.\n2. what is the criterion of the six SOTA methods? I suggest the author should select more representative methods:\n(1) The modulation-based codec which can tolerate high IDS errors(https://pubs.acs.org/doi/abs/10.1021/acs.jcim.3c00629)\n(2) deep learning based methods(https://onlinelibrary.wiley.com/doi/abs/10.1002/smtd.202400959,\nhttps://www.nature.com/articles/s41598-024-83806-5)\n3. the Yin-yang codec is actually designed to obtain a balanced encoding, it has no error correction capability in theory.\n4. although the authors make many simulation study, I want to know what the important insight can we get from your study?\n5. To take a comprehensive comparison, the author should compare based on logical density, error capability, time complexity, cost etc to give us a general view of these method.\n6. Fig 1c shows the tolerance of individual error type which I think is meaningless, as the combination of them is the reality and more challenging.\n7. concerning the clustering algorithm, obviously, the quality of clustering have great influence on the recovering process of any codec.", "reviewer_2": "The authors present a detailed comparative study of state-of-the-art codecs for DNA data storage under unified synthetic and experimental frameworks. The analysis focuses on the error-correction capabilities of existing codes, addressing both base-level errors and sequence dropouts. This has important implications for synthesis and sequencing costs, as well as storage density. The contribution is highly valuable to the DNA data storage community and provides clear benchmarks for future research. All claims made by the authors are well-supported by thorough evidence, and the methodology is clearly described. I do, however, have a number of concerns and general comments detailed below.\n*Major comments:\n- One of the main conclusions of the paper, as stated in the abstract “We find synthetic benchmarks commonly used in literature to be unsuitable indicators of codec performance…”; does not strike me as particularly surprising or intriguing, given the simplistic methodology adopted for synthetic benchmarking. For example, the setup in Fig. 1(e) assumes exactly 30 reads for a subset of the strands and zero reads for the rest (controlled via dropout rate). It is evident that this “30 or zero” setup is far from realistic, and the resulting outcomes are not suitable indicators of real-world performance. In fact, prior work by the authors and others has shown that the coverage distribution is positively skewed and can be well-approximated by a log-normal distribution. Such a model could have been adopted for synthetic benchmarking, making the comparison between synthetic and real settings more meaningful. With relatively little effort, the synthetic setup could have been made more realistic, and any observed discrepancies between the synthetic and real cases would then have carried more weight. As it stands, I believe this is the main weakness of the study, and I provide further comments on this point in the sequel. \n- In the “basic error scenario,” the meaning of “without clustering” is unclear. Does this imply that only a single reads of each strand is available for decoding? If multiple copies are available, are they ordered, indexed, or otherwise identifiable? If the decoder receives 30 noisy, unordered copies per oligo, it is unclear how decoding can proceed without a clustering step. Please clarify the assumptions and the exact decoding procedure in this scenario. \n- While I understand that restricting the experiments to a single core eliminates the influence of parallelization and provides a uniform basis for comparison, this setting may also be unfair for certain codecs. Specifically, the “lack of parallelization in a codec’s implementation” is not always just an implementation limitation. In some cases, it reflects an inherent property of the decoding algorithm itself, which may not be amenable to parallel processing. This is a meaningful disadvantage of the codec that should not be overlooked in a comparative evaluation. I recommend clarifying and discussing this point. \n- The imposed time constraint (5.4 bytes/second) should be justified. Is this threshold based on practical considerations, or is it arbitrary? Would the conclusions change significantly if the threshold were set to, say, 4 bytes/second? Presenting decoding performance as a function of runtime, rather than relying on a single cutoff (whose value may be difficult to justify), would offer a more informative and flexible comparison. \n- Fig. 1 (d) shows the effect of clustering + MSA + consensus on error tolerance. This improvement is due to a reduction in the effective error rate that the codec must handle. For instance, when we say a codec tolerates x% errors with 30 reads, it is actually tolerating a residual error rate y% after consensus. However, the relationship between raw and residual error rates is not reported. I strongly recommend including these values, as they provide insight into how much of the burden is handled by the codec versus the consensus step. \n- Given that cost is a central limitation in DNA storage, I was expecting a more thorough discussion of this aspect. Specifically, which codecs are best positioned to minimize cost per information bit while maintaining reliable decoding? For instance, the codec that allows the highest information density (bits/nt) under realistic error rates likely enables the lowest synthesis cost. Anchoring the comparison in terms of potential cost-per-bit would add significant value to the work. \n- The paper frequently refers to “physical redundancy,” but its role in performance is not thoroughly discussed. If measured before amplification, and assuming relatively uniform physical coverage, why is physical redundancy critical? Is it because low physical redundancy can lead to synthesis errors being systematically propagated across all reads of a given strand? Or is it because low redundancy exacerbates coverage bias, increasing the likelihood of strand dropout? Clarifying this point would improve the discussion. \n*Some minor comments:\n- There is a missing punctuation mark in line 133. \n- The terms “accuracy” and “similarity” appear to be used interchangeably in the main text and supplementary materials when describing clustering metrics. It would be helpful to adopt consistent terminology.", "reviewer_3": "The paper \"Comparison of state-of-the-art error-correction coding for sequence-based DNA data storage\" by Gimpel and colleagues is well written, and the objectives are clearly presented. The goal of the manuscript is to demonstrate how DNA storage algorithms (i.e., error-correcting codes) can be realistically analyzed and compared with respect to their performance for long-term storage. The study presents a benchmark of six representative DNA data storage codes, using both in silico simulations and in vitro experiments under standardized conditions.\nHowever, the paper has several limitations that reduce the interpretability of its conclusions.\n1. The selection of only six codecs: While justified by implementation availability and literature prominence, several recent codes are excluded, potentially limiting the claim of establishing the true state-of-the-art. Including a broader or more diverse codec set would provide a more comprehensive view of the field.\n2. Simplified error models used for synthetic benchmarking: This is very similar to what has been done in the literature and raises the question of novelty. This approach, while standardized, potentially overlooks critical edge-case behaviors that would be revealed under more diverse or adaptive error distributions, e.g., specific motifs that affect sequencing or synthesis.\n3. Fixed parameter sets without optimization: The study relies on fixed parameter sets for each code without performing any code-specific parameter optimization. As a result, some codes may appear to underperform not due to intrinsic limitations but because their configurations were suboptimal for the tested conditions. This could be addressed by including a sensitivity analysis or an optimization run to assess how performance scales with parameter tuning.\n4. Lack of computational performance analysis: A detailed analysis of runtime and memory usage is also required. A comprehensive benchmark should include not only error-correction capabilities but also computing performance measures. For real-world DNA storage systems, the speed and scalability of decoding pipelines are crucial performance metrics, and omitting them leaves a gap in the overall assessment.\n5. Codec–clustering algorithm pairing: The pairing between codes and clustering algorithms significantly impacts performance. While the paper tests various clustering strategies and selects the best-performing one for each code, it does not fully explore whether these pairings are theoretically optimal or practically justified. In some cases, general-purpose tools like CD-HIT outperformed specialized algorithms, which raises questions about code-specific biases or incompatibilities with clustering strategies. A deeper investigation into why certain pairings fail, or how code design interacts with clustering, would have added depth to the analysis.\n6. Importance of motifs in realistic evaluation: It is also worth mentioning that motifs are highly important for any realistic evaluation. There are known sequence motifs that interact with enzymes during sequencing and also some affecting DNA synthesis. PCR and other processes are also influenced by sequence motifs (or more precisely, by how these motifs interact with enzymes). This important fact should be more prominently highlighted, as only one code in the tested list can handle motifs. This directly relates to my first point: the selection of codes. There are other published codes that can handle sequence motifs. A fair, and more importantly, a realistic comparison would include motif-aware codes. A code that is not able to handle motifs will inevitably fail from time to time in practice, especially when large amounts of data are encoded.\nIn conclusion, the paper provides a valuable and ambitious framework for standardized code benchmarking in DNA data storage. However, its conclusions are somewhat constrained by the limited code selection, oversimplified error models, lack of parameter optimization, and absence of computational performance metrics. Expanding the scope and granularity of the evaluation would further enhance the robustness and applicability of the findings."}, "author_response": "Response to Referees\nReviewer comments in italics, author replies in red with actions bolded.\nLine numbers refer to the revised manuscript, in which changes have been highlighted in yellow.\nAs a general comment we would like to state that it is/was not the goal of this work to identify one\ncodec as being better than another. Rather, it was our goal to analyse trade-offs in existing coding\nstrategies and ensure that future development of error correction codes for DNA data storage has a\nstrong fundament to compare to. This is also nicely exemplified in the results, as current codes based\non similar principles of inner and outer redundancy converge to a common state of the art (see Fig. 3\nand new Fig. S21). We have adapted the text to make this intention more clear to the readers.\nComments by Referee #1\nDNA storage is emerging as a potential alternative to the challenge of exponentially increasing data.\nOne major topic is how to effectively retrieve information form the IDS error caused by DNA synthesis,\nPCR and sequencing process. and the dropout issue in the random sampling process is also a critical\nissue to be solved. Up to now, many methods have proposed to tackle these problems. this paper\nintended to make a comprehensive comparison of some famous codec to evaluate their performance.\nAlthough this work is very meaningful for us to understand the real performance of these methods,\nthere exist some issues need to be solved:\nWe would like to thank the reviewer for critically assessing our manuscript and providing valuable\ncomments.\n1. the dropout issue is a unique problem in DNA storage in the random sampling process, it is relative\neasy to solve by the fountain code which is very mature in tradition communication society. And any\ncodec could use it to solve this issue. therefore, the key issue should be the IDS error correction.\nmy suggestion is to separate the evaluation of IDS error correction capability and dropout issue.\nWe thank the reviewer for their suggestion of separating the evaluation of errors and sequence\ndropout. While we agree that fountain codes are mature, these also compete with other codes in\ntraditional communications technologies and storage, including Reed Solomon codes (RAID-6 storage),\nBCH codes (SSD discs), Low Density Parity Check codes (Ethernet), and Turbo codes (3G/4G mobile\ncommunications). Consequently, it is not established that Fountain codes are the best solution for\ndropout errors specifically. Still, we agree that separating IDS errors and sequence dropout is\ninteresting. Thus, following the reviewer's suggestion, we have added new results on the tolerance to\nsequence dropout as Fig. 1c. With this change, the separate evaluation of IDS error correction\ncapability (via Fig. 1d) and sequence dropout (via Fig. 1c) is now possible. We have also changed the\ncorresponding section of the results section (ll. 237-242) to highlight this distinction:\n“Thus, we extended the aforementioned basic error scenario with another variable, the\nfraction of sequences lost. As such, Fig. 1c shows the rate of sequence loss each codec tolerates\nin the absence of errors. The large differences in tolerated sequence loss between codecs and\ncode rates – from 0% (Goldman and Yin-Yang) to 64% (DNA Fountain at 0.5 bit/nt) – highlight\nthat codecs are not equally equipped to compensate for this unique problem in DNA data\nstorage.”\nFigure 1: Overview of the scope of this work and the evaluations of clustering algorithms and codecs. (a) Overview of the\ndata storage workflow considered in this work, including encoding of the data with a codec, an in silico or in vitro data storage\nworkflow, post-processing by read clustering and generation of consensus sequences, and decoding of the data with a codec. (b) Exemplary outcome of a basic error scenario with naïve (grey) or CD-HIT clustering (red) upon variation of the overall error\nrate (see Methods). Individual points denote the outcomes of 30 individual iterations of the scenario. The solid lines represent\nthe logistic regression performed to estimate the error rate at which data recovery succeeds with 95% probability (dashed\ngrey line). The corresponding error rate is then used as performance metric in this work. (c) Tolerance of all codecs in the\nbasic error scenario to sequence loss. All codecs use the best-performing clustering algorithm denoted in panel d. Conditions\nindicated with an hourglass (⧖) were limited by the time constraint (see Supplementary Figs. 6-7). (d) Performance of each\ncodec at its supported code rates in the basic error scenario, using both naïve clustering (grey bars) and the best-performing\nclustering algorithm for each codec (colored bars, see Supplementary Table 3 for full data). Conditions indicated with an\nhourglass (⧖) were limited by the time constraint (see Supplementary Figs. 4-5). Dark shading of the bars indicates the\neffective error rate remaining after the clustering step. (e) Pareto fronts of codec performance in a scenario combining errors\nat a fixed ratio of 53% substitutions, 45% deletions, and 2% insertions with sequence dropout. Feasible regions lie below the\nindicated pareto fronts, with the mean error rates of the high-fidelity (dashed line) and low-fidelity scenario (dotted line)\nindicated. The Yin-Yang codec, which did not include independent error-correction capabilities, was unable to decode the\ndata in this scenario at all, given that neither it nor the clustering step was capable of compensating for any sequence\ndropout.\n( conical microtube, pipette, and thermocycler) were provided by Labicons (www.labicons.net). Panel a) partially created in\nBioRender. Gimpel, A. (2025) https://BioRender.com/9wrsaez.\n2. what is the criterion of the six SOTA methods? I suggest the author should select more representative\nmethods:\n(1) The modulation-based codec which can tolerate high IDS errors\n(https://pubs.acs.org/doi/abs/10.1021/acs.jcim.3c00629)\n(2) deep learning based methods (https://onlinelibrary.wiley.com/doi/abs/10.1002/smtd.202400959,\nhttps://www.nature.com/articles/s41598-024-83806-5)\nAs described in the methods of the manuscript, we selected the six codecs based on “availability of an\nopen-source implementation with sufficient documentation, the presence of in vitro experiments in\nthe original publication, the prominence in literature, and considerations for covering a broad range\nof approaches”, with October 2023 as the cut-off date (due to lead time for experimental work). We\nextended our description of selection criteria at the beginning of the results section (ll. 109-112) to\nbetter document our selection:\n“For this study, we limited our benchmarking of codec performance to representative\nexamples, using the availability of an open-source implementation with sufficient\ndocumentation, the presence of in vitro experiments in the original publication, the\nprominence in literature, and considerations for covering a broad range of approaches as\nselection criteria (see Methods and Supplementary Table 2, cut-off date October 2023).”\nThe first method suggested by the reviewer, from Zan et al.1, was initially excluded because its study\ndid not feature any vitro experiments (see Supplementary Table 2). Both deep learning-based\nmethods2,3 suggested by the reviewer were published after our cut-off date (both in December 2024)\nand do not provide an open-source code repository. Nonetheless, we have now added new results\ncomparing the performance of the state-of-the-art with additional literature methods, specifically\nthose by Zan et al. (Modulation)1 proposed by the reviewer, as well as Chandak et al. (LDPC)4, and\nSong et al. (DBGPS)5. The results are shown in Supplementary Fig. 21 and discussed in ll. 404-412 of\nthe results:\n“To demonstrate the versatility of the established high- and low-fidelity scenarios for codec\nbenchmarking, we extended our benchmarking with three additional codecs from the\nliterature, namely Modulation by Zan et al.1, LDPC by Chandak et al.4, and DBGPS by Song et\nal.5 Supplementary Fig. 21 shows the performance of these additional codecs compared to the\nbest-performing codecs from Fig. 3, DNA-Aeon by Welzel et al.6 and DNA-RS by Heckel et al.7,8\nInterestingly, DBGPS by Song et al.5 achieved a similar performance to DNA-Aeon and DNA-RS\nin most scenarios. This highlights the value of standardized benchmarks for codec comparisons\nand suggests using a balanced inner/outer code separation strategy is more important than\nchoosing a specific error-correction code and implementation.”\nSupplementary Figure 21: Extension of the in-silico analysis to additional literature codecs by Song et al. (DBGPS)5, Chandak\net al. (LDPC)4 and Zan et al. (Modulation)1. (a) The effect of clustering on the additional literature codecs, as measured in\ntolerated error rate, between naïve clustering (grey) and the best-performing clustering algorithm (green, CD-HIT). (b)\nComparison of each codec’s error tolerance against individual error types (deletions, blue; insertions, green; substitutions,\norange). The best-performing codecs from the main text, DNA-Aeon by Welzel et al.6 and DNA-RS by Heckel et al.7,8 are shown\nfor comparison. (c) Pareto fronts of the additional codecs upon simultaneous consideration of errors and sequence dropout,\ngrouped by code rate. The best-performing codecs from the main text, DNA-Aeon by Welzel et al.6 and DNA-RS by Heckel et\nal.7,8 are shown in lighter colours for comparison. (d) Pareto fronts of the additional codecs in the high- (top row) and low-\nfidelity scenarios (bottom row) upon variation of the initial coverage and the sequencing depth. In the high-fidelity scenario,\nsynthesis by material deposition and amplification with a high-fidelity polymerase is assumed, whereas the low-fidelity\nscenario includes electrochemical synthesis and amplification with Taq polymerase. The best-performing codecs from the\nmain text, DNA-Aeon by Welzel et al.6 and DNA-RS by Heckel et al.7,8 are shown in lighter colours for comparison.\n3. the Yin-yang codec is actually designed to obtain a balanced encoding, it has no error correction\ncapability in theory.\nWe fully agree with the reviewer. The Yin-Yang codec was included in our analysis to showcase the\npossibility of error-free data decoding without any codec-level error-correction. We have highlighted\nthe lack of error-correction capability for the Yin-Yang codec at multiple locations throughout the\nmanuscript (e.g., ll. 105, 186, 215, 344, 567, Table 1).\n4. although the authors make many simulation study, I want to know what the important insight can\nwe get from your study?\nWe thank the reviewer for the opportunity to improve the description of the main outcomes of our\nstudy. We rephrased the abstract (ll. 23-36) and the conclusions (ll. 493-494, 498-501, 507-513) as\nfollows:\n“[…]. Using synthetic benchmarks, we find that existing codecs can tolerate error rates up to\n14% and sequence loss as high as 65% in isolation. However, codecs weigh redundancy to\nerrors and sequence loss differently, calling the simple metrics commonly used for assessing\ncodec performance in literature into question. To improve upon these simple metrics, we\nimplement comprehensive error scenarios covering major experimental parameters to assess\ncodec performance under realistic conditions. In testing codecs using both high- and low-\nfidelity workflows, a balanced inner/outer code separation strategy emerges as superior error-\ncorrection scheme, independently of the implementation. Moreover, storage densities as high\nas 117 EB g-1 are feasible using existing codecs and current synthesis and sequencing\ntechnologies. Verifying our results with fair, standardized in vitro experiments, we\ndemonstrate data storage at 43 EB g-1 using synthesis by material deposition and 13 EB g-1\nusing the more error prone electrochemical synthesis, employing only existing codecs from the\nliterature. Besides closing in on the physical limits of DNA data storage, this study thus\ndemonstrates the maturity of error-correction coding, defines its current state-of-the-art, and\nestablishes best practices for codec benchmarking.”\n“Notably, we find that existing codecs can tolerate error rates up to 14% and sequence loss as\nhigh as 65% in isolation, but codecs’ abilities to correct both simultaneously varies drastically.\n[…]. Taken together, our results thus challenge common practice in the DNA data storage\nliterature, as both synthetic error benchmarks and serial dilution/amplification experiments\nare widely-used standards to showcase codec performance,6,9–11 but fail to differentiate codec\nperformance in our standardized setup. As an alternative, rigorous benchmark, we propose\nand implement both a high- and low-fidelity scenario, thereby systematically assessing codecs\nacross three critical experimental factors: synthesis technology, physical redundancy, and\nsequencing depth. […]. Extension of our synthetic benchmark suite to three additional\nliterature codecs (see Supplementary Fig. 21) further highlighted that state-of-the-art\nperformance depended more on a well-executed, balanced inner/outer code separation\nstrategy than on the specific choice of error-correction code. This suggests existing codecs are\nsufficiently close to the theoretical limit, such that minor implementation details (e.g.,\noptimality of parameter choices, balance of inner and outer redundancy) cause the remaining,\nminor gaps in codec performance in most cases.”\n5. To take a comprehensive comparison, the author should compare based on logical density, error\ncapability, time complexity, cost etc to give us a general view of these method.\nWe appreciate the reviewer’s suggestion regarding important aspects of a comprehensive comparison.\nWe’d like to point out that our study now includes all of the mentioned aspects:\n- we systematically adjust codec parameters to achieve three standardized logical densities\n(referred to as code rates in the manuscript) of 0.5, 1.0, and 1.5 bit/nt where possible,\n- we quantify error capability with both simple error models for error and dropout tolerance\n(see Fig. 1e), as well as realistic error models which additionally address sequencing depth and\nphysical coverage (see Fig. 3a),\n- we include time complexity in our results by imposing a compute and time constraint, thereby\npenalizing poorly scaling methods (see e.g., Supplementary Figs. 4-5),\n- we have now also added a cost comparison using the cost structures of both Twist and\nGenscript as Supplementary Fig. 20.\n6. Fig 1c shows the tolerance of individual error type which I think is meaningless, as the combination\nof them is the reality and more challenging.\nWe thank the reviewer for their suggestion. We have now replaced the panel in Fig. 1c, showing the\ntolerance to individual error types, with a panel showing the tolerance to sequence dropout, as\nsuggested in this reviewer’s first comment. The tolerance to individual error types is now shown as\nSupplementary Fig. 8 instead. We have amended the main text to reflect this change (ll. 237-242).\n7. concerning the clustering algorithm, obviously, the quality of clustering have great influence on the\nrecovering process of any codec.\nWe agree with the reviewer’s assessment of the relevance of clustering quality on decoding\nperformance. For this reason, we tested all selected codecs with five different clustering algorithms\n(see Supplementary Tables 1 and 3) before selecting the best-performing clustering algorithm for each\ncodec. Indeed, as proposed by the reviewer, the choice of clustering algorithm had remarkable impact\non the codec performance.\nAs Reviewer 3 also suggested a detailed investigation into the impact of the selected codec-clustering\npairings, we performed additional analyses to quantify the reduction in effective error rate for all\ncodec-clustering pairings, see Supplementary Figs. 1 and 3. We added a paragraph outlining the\nfindings to the results section (see ll. 181-183):\n“As expected, this positive effect on error tolerance was caused by a drastic reduction in the\nerror rate within the clustered reads (as low as 0% up to an error rate of 5%, see Supplementary\nFigs. 1 and 2), thereby considerably reducing the effective error rate for the decoder.”\nSupplementary Figure 1: Relationship between the effective error rate in clustered reads and the applied error rate during\nthe basic error scenario for each codec-clustering pairing. For each investigated codec (rows), code rate (columns), and\nclustering algorithm (color), the effective error rate in the clustered reads as determined by mapping to the reference\nsequences is shown. In all cases, an increase in the error rate in the sequencing reads yields an increase in the error rate in\nthe clustered reads.\nComments by Referee #2\nThe authors present a detailed comparative study of state-of-the-art codecs for DNA data storage\nunder unified synthetic and experimental frameworks. The analysis focuses on the error-correction\ncapabilities of existing codes, addressing both base-level errors and sequence dropouts. This has\nimportant implications for synthesis and sequencing costs, as well as storage density. The contribution\nis highly valuable to the DNA data storage community and provides clear benchmarks for future\nresearch. All claims made by the authors are well-supported by thorough evidence, and the\nmethodology is clearly described. I do, however, have a number of concerns and general comments\ndetailed below.\nWe thank the reviewer for their positive evaluation of our work and their great suggestions.\n*Major comments:\n- One of the main conclusions of the paper, as stated in the abstract “We find synthetic benchmarks\ncommonly used in literature to be unsuitable indicators of codec performance…”; does not strike me as\nparticularly surprising or intriguing, given the simplistic methodology adopted for synthetic\nbenchmarking. For example, the setup in Fig. 1(e) assumes exactly 30 reads for a subset of the strands\nand zero reads for the rest (controlled via dropout rate). It is evident that this “30 or zero” setup is far\nfrom realistic, and the resulting outcomes are not suitable indicators of real-world performance. In fact,\nprior work by the authors and others has shown that the coverage distribution is positively skewed and\ncan be well-approximated by a log-normal distribution. Such a model could have been adopted for\nsynthetic benchmarking, making the comparison between synthetic and real settings more meaningful.\nWith relatively little effort, the synthetic setup could have been made more realistic, and any observed\ndiscrepancies between the synthetic and real cases would then have carried more weight. As it stands,\nI believe this is the main weakness of the study, and I provide further comments on this point in the\nsequel.\nWe thank the reviewer for pointing out the perceived simplicity of our synthetic benchmarks. We’d\nlike to clarify that we use two different error models for our synthetic benchmarks throughout our\nmanuscript, precisely to support our conclusion regarding the inadequacy of simplistic error models.\nThe simple model referred to by the reviewer (i.e., exactly 30 reads with fixed error rates and sequence\ndropout) represents the status-quo in the literature, and is used only for the initial analyses in Fig. 1.\nThe subsequent analyses in Figs. 2 and 3 use a complex error model based on exactly the previous\nwork12 by us the reviewer mentioned. This model includes:\n- skewed coverage distributions, as highlighted by the reviewer,\n- sequence-specific efficiency bias during PCR amplification,\n- consecutive deletions (“deletion runs”) from synthesis,\n- positional dependence of the error rate during sequencing,\namong other non-idealities common in real-world sequencing data.\nWe’d also like to point out our experimental confirmation of the results obtained with the complex\nmodel (see Fig. 4). The good agreement between experiments and our model results highlights how\nrealistically our complex model can represent experimental sequencing data.\nTo better differentiate the two models used throughout the study and to highlight the complexity of\nour synthetic benchmarks, we rephrased the corresponding paragraphs in the introduction and\nresults (ll. 89-93 and ll. 274-280):\n“For our in silico benchmarks, we first employ simple synthetic error scenarios (i.e., applying\nfixed error rates) commonly used in the literature. Then, we extend our analysis with\ncomprehensive benchmarks considering the full spectrum of non-idealities present in real-\nworld sequencing data (using DT4DDS12), and verify our results with standardized in vitro\nexperiments”\n“Despite the common use of these simple synthetic benchmarks for codec benchmarking in\nthe literature, they overlook the non-ideal error patterns and biases present in experimental\nsequencing data – from error runs to skewed coverage distributions.12–14 Thus, for all further\nbenchmarks, we implemented realistic experimental workflows using the models\nimplemented in the simulation software DT4DDS12 (see Methods). This simulation software\ntakes into account the aforementioned non-idealities which are missing from the synthetic\nbenchmarks previously performed in the literature.”\n- In the “basic error scenario,” the meaning of “without clustering” is unclear. Does this imply that only\na single reads of each strand is available for decoding? If multiple copies are available, are they ordered,\nindexed, or otherwise identifiable? If the decoder receives 30 noisy, unordered copies per oligo, it is\nunclear how decoding can proceed without a clustering step. Please clarify the assumptions and the\nexact decoding procedure in this scenario.\nWe appreciate the reviewer highlighting this unclear point. “Without clustering” refers to the situation\nin which the set of all 30 reads per sequence are supplied to the decoder (i.e., all unique reads). This\nmeans the decoder has multiple erroneous, unordered copies available for decoding. All encoders\ninvestigated in this study include indexes in their sequences, so that decoding can proceed without\nrequiring the ordering of copies during a separate clustering step. We have improved the explanation\nof the assumptions and the decoding procedure in this situation (ll. 151-153):\n“Fig. 1d shows the error tolerance of all codecs in this basic error scenario, without clustering\nand consensus generation by alignment (i.e., the set of all 30 erroneous reads per sequence\nare supplied to the decoder directly, following the procedure by Erlich et al.9; shown as\n“Naïve”, grey bars).”\n- While I understand that restricting the experiments to a single core eliminates the influence of\nparallelization and provides a uniform basis for comparison, this setting may also be unfair for certain\ncodecs. Specifically, the “lack of parallelization in a codec’s implementation” is not always just an\nimplementation limitation. In some cases, it reflects an inherent property of the decoding algorithm\nitself, which may not be amenable to parallel processing. This is a meaningful disadvantage of the\ncodec that should not be overlooked in a comparative evaluation. I recommend clarifying and discussing this\npoint.\nWe agree with the reviewer’s point about potentially disadvantaging parallelized codecs by\nconstraining codecs to one core. For our evaluation, we preferred to compare codecs using comparable\ncompute power, rather than to introduce the additional dimension of parallelizability. As noted by the\nreviewer, some codecs might not be parallelized due to inherent limitations of the algorithm, while\nother codecs might be parallelizable in theory but weren’t implemented as such. We decided that\nenabling parallelization would misrepresent the latter group, while skewing the overall results based\non the larger amounts of compute power available to parallelized codecs.\nWe now clarify our decision and discuss this point early in the results (ll. 147-150):\n“Moreover, the constraint to one CPU core negated any undue disadvantage due to a lack of\nparallelization in a codec’s implementation. However, this constraint thus overlooks inherent\nincompatibilities with parallel processing possibly present in some decoding algorithms, which\nrepresent a considerable disadvantage to their large-scale implementation.”\n- The imposed time constraint (5.4 bytes/second) should be justified. Is this threshold based on practical\nconsiderations, or is it arbitrary? Would the conclusions change significantly if the threshold were set\nto, say, 4 bytes/second? Presenting decoding performance as a function of runtime, rather than relying\non a single cutoff (whose value may be difficult to justify), would offer a more informative and flexible\ncomparison.\nThe reviewer raises an important point about the impact of our time constraint on our results. We\nchose the time constraint based on practical considerations for our computing environment, rather\nthan based on anticipated decoding throughput. Specifically, the time constraint of one hour per\ndecoding attempt ensures that all decoding attempts can finish within the time limit for a single\ncomputing job on our computing cluster. The specified decoding throughput of 5.4 bytes/second thus\narises indirectly from the time limit (1h) and the file size (19 kB). While the value of this minimum\nthroughput is thus arbitrary, it is sufficiently low such that any codec which is limited by this constraint\nwould be unsuitable for most applications of DNA data storage (e.g., cold storage of large archives).\nWe show the decoding performance relative to the runtime in Supplementary Figs. 4, 6, 9, 11, 12,\nand 13. Evidently, the time constraint is not the limiting factor in the vast majority of cases. In the few\ncases in which time is constraining codec performance, the dependence on the exact time limit is\nminor. We now discuss the dependence of decoding performance on our choice of time limit in the\nresults (ll. 140-146):\n“The choice of time constraint and file size were based on limitations of the employed\ncomputing environment, and thus indirectly enforced a decoding speed of at least 5.4 bytes\nper second. While this minimum enforced decoding throughput was thus not based on\npractical considerations, it is nonetheless sufficiently permissive such that any codec which is\nlimited by this constraint would be unsuitable for most applications of DNA data storage (e.g.,\ncold storage of large archives). Accordingly, the time constraint did not limit decoding in the\nvast majority of cases, and we show the decoding performance relative to the runtime in\nSupplementary Figs. 4, 6, 9, 11, 12, and 13.”\n- Fig. 1 (d) shows the effect of clustering + MSA + consensus on error tolerance. This improvement is\ndue to a reduction in the effective error rate that the codec must handle. For instance, when we say a\ncodec tolerates x% errors with 30 reads, it is actually tolerating a residual error rate y% after consensus.\nHowever, the relationship between raw and residual error rates is not reported. I strongly recommend\nincluding these values, as they provide insight into how much of the burden is handled by the codec\nversus the consensus step.\nWe agree that analysing the reduction in effective error rate afforded by clustering is an important\naspect of understanding the effects of clustering on decoding performance. We have performed a\nnew, in-depth investigation into the effects on error rates for all codec-clustering pairings and report\nthem in Supplementary Fig. 2. Moreover, we have revised Fig. 1d in the main text to show the\neffective error rate after clustering to facilitate a direct comparison.\nTo highlight this effect of clustering, we included more details in the corresponding two paragraphs\nof the results section (see ll. 181-183, 217-221):\n“As expected, this positive effect on error tolerance was caused by a drastic reduction in the\nerror rate within the clustered reads (as low as 0% up to an error rate of 5%, see Supplementary\nFigs. 1 and 2), thereby considerably reducing the effective error rate for the decoder. […].\nInterestingly, nine of the thirteen tested codecs and code rates performed best with the\nestablished general-purpose clustering algorithm CD-HIT (red bars in Fig. 1d). Unsurprisingly,\nthis general preference for CD-HIT was caused by this clusterer’s superior performance,\nyielding the fewest and least erroneous clusters across all tested clustering algorithms up to\nan overall error rate of around 10% (see Supplementary Figs. 1-3).”\nSupplementary Figure 2: Relationship between the effective error rate in the closest clustered reads and the applied error\nrate during the basic error scenario for each codec-clustering pairing. For each investigated codec (rows), code rate\n(columns), and clustering algorithm (color), the effective error rate in the clustered reads as determined by mapping to the\nreference sequences is shown. In contrast to Suppl. Fig. 1, only the closest cluster to each reference sequence is considered.\nThis is most representative of the effective error rate encountered by the decoder. In all cases, an increase in the error rate\nin the sequencing reads yields an increase in the error rate in the clustered reads.\n- Given that cost is a central limitation in DNA storage, I was expecting a more thorough discussion of\nthis aspect. Specifically, which codecs are best positioned to minimize cost per information bit while\nmaintaining reliable decoding? For instance, the codec that allows the highest information density\n(bits/nt) under realistic error rates likely enables the lowest synthesis cost. Anchoring the comparison\nin terms of potential cost-per-bit would add significant value to the work.\nWe thank the reviewer for their great suggestion regarding the comparison of synthesis costs.\nImportantly, the cost structures of commercial synthesis providers are not linear, meaning the cost of\nsynthesizing one base decreases drastically as the overall size of the library grows (see Supplementary\nFig. 20a+b). As a result, the cost per information bit depends much more strongly on the data volume\nthan on the codec choice.\nWith this in mind, we have added a comparison using the cost structures of both Genscript and Twist,\ncomparing the cost per information bit for storing one megabyte of data using all investigated codecs\nand code rates (see Supplementary Fig. 20c). As predicted by the reviewer, the codecs with the highest\ninformation density (i.e., 1.5 bit/nt) have the lowest synthesis cost per bit of information.\nWe now also highlight these findings in the results (ll. 394-403):\n“However, choosing lower code rates increases synthesis costs proportionally, as the price of\nsynthesis scales directly with the number of sequences to be synthesized resp. the amount of\ndata stored (see Supplementary Fig. 20a+b). Given the standardized code rates used in this\nstudy, all codecs feature similar costs per bit stored, at 14 USD/kB on average in the high-\nfidelity scenario and 7.9 USD/kB in the low-fidelity scenario (at a 1 MB scale, see\nSupplementary Fig. 20c). Ultimately, synthesis cost is not directly defined by the choice of\ncodec, but purely by the employed code rate. To this end, the choice of code rate must\nultimately compromise between achieving the highest storage densities at lower code rates\nand reducing the synthesis costs at higher code rates. Thus, given the direct scaling of synthesis\ncosts with code rate, our performance analysis comparing three standardized code rates also\ntranslates directly to anticipated synthesis costs.”\n- The paper frequently refers to “physical redundancy,” but its role in performance is not thoroughly\ndiscussed. If measured before amplification, and assuming relatively uniform physical coverage, why is\nphysical redundancy critical? Is it because low physical redundancy can lead to synthesis errors being\nsystematically propagated across all reads of a given strand? Or is it because low redundancy\nexacerbates coverage bias, increasing the likelihood of strand dropout? Clarifying this point would\nimprove the discussion.\nWe define physical redundancy as the mean number of copies per reference sequence during storage\n(i.e., after synthesis, amplification, and dilution; but before re-amplification and sequencing). As\ncorrectly identified by the reviewer, physical redundancy is critical as an experimental parameter\nbecause a low physical redundancy increases the risk that sequences are missing in the sequencing\ndata, and because the remaining reads feature similar errors (thereby reducing the effectiveness of\nclustering). Importantly, physical redundancy is also an important metric in itself, as it is inversely\nproportional to storage density, one of DNA’s main advantages as a storage medium.\nWe have now extended our discussion of physical redundancy as an experimental parameter,\nhighlighting its effects on sequencing output (ll. 321-328):\n“While the choice of synthesis provider is usually well-documented in literature studies and its\neffect on error rate well-established, the choices of physical redundancy and sequencing depth\nvary considerably across studies and affect only the homogeneity of sequencing reads.12,13,15\nSpecifically, both low physical redundancies and low sequencing depths increase the likelihood\nof sequence dropout and diminish the diversity of the oligos due to the stochastic sampling of\nonly few oligo copies. Importantly, physical redundancy directly affects the data’s storage\ndensity (i.e., it is inversely proportional to storage density), whereas sequencing depth directly\ninfluences sequencing costs.”\nSupplementary Figure 20: Cost comparison of codecs by synthesis provider and scenario. (a) The cost structure of the two\ncommercial providers analysed in this study depends strongly on the number of synthesized sequences (assuming a fixed\nsequence length of 150 nt). (b) Using a fixed code rate of 1.5 bit/nt for synthesis by Twist Biosciences and 1.0 bit/nt for\nGenscript yields comparable prices per kilobyte by data volume. (c) Comparing the costs for synthesis across all codecs and\ncode rates tested in this study, in both the high- (blue) and low-fidelity scenarios (orange) and at a fixed data volume of one\nmegabyte, reveals only minor differences. However, the costs for low-fidelity synthesis are about half as high as the high-\nfidelity synthesis across most codecs.\n*Some minor comments:\n- There is a missing punctuation mark in line 133.\nThanks for pointing out this mistake. This has been fixed.\n- The terms “accuracy” and “similarity” appear to be used interchangeably in the main text and\nsupplementary materials when describing clustering metrics. It would be helpful to adopt consistent\nterminology.\nWe thank the reviewer for bringing up this inconsistency. The use of these terms has now been\nharmonized.\nReviewer #2 (Remarks on code availability):\nI did not try running the code, but the provided Github repository seems to include all the necessary\nsource files.\nWe thank the reviewer for evaluating the completeness of our code repository.\nComments by Referee #3\nThe paper \"Comparison of state-of-the-art error-correction coding for sequence-based DNA data\nstorage\" by Gimpel and colleagues is well written, and the objectives are clearly presented. The goal of\nthe manuscript is to demonstrate how DNA storage algorithms (i.e., error-correcting codes) can be\nrealistically analyzed and compared with respect to their performance for long-term storage. The study\npresents a benchmark of six representative DNA data storage codes, using both in silico simulations\nand in vitro experiments under standardized conditions.\nWe appreciate the reviewer’s positive evaluation of our manuscript and its goals. We also thank the\nreviewer for their constructive comments.\nHowever, the paper has several limitations that reduce the interpretability of its conclusions.\n1. The selection of only six codecs: While justified by implementation availability and literature\nprominence, several recent codes are excluded, potentially limiting the claim of establishing the true\nstate-of-the-art. Including a broader or more diverse codec set would provide a more comprehensive\nview of the field.\nWe agree with the reviewer, and have now extended our analysis to three additional codecs from the\nliterature. While we would have liked to use even more codecs from the literature, we observed that\nalso many recent codecs are either lacking an open-source implementation altogether, or available\nimplementations are poorly reusable, thus requiring undue reprogramming to allow for benchmarking.\nMoreover, many recent studies focus on image storage and compression, and are thus beyond the\nscope of our work on general-purpose methods.\nNonetheless, we have now added new results comparing the performance of the state-of-the-art\nwith additional literature methods, specifically those by Zan et al. (Modulation)1, Chandak et al.\n(LDPC)4, and Song et al. (DBGPS)5. The results are shown in Supplementary Fig. 21 and discussed in ll.\n404-412 of the results:\n“To demonstrate the versatility of the established high- and low-fidelity scenarios for codec\nbenchmarking, we extended our benchmarking with three additional codecs from the\nliterature, namely Modulation by Zan et al.1, LDPC by Chandak et al.4, and DBGPS by Song et\nal.5 Supplementary Fig. 21 shows the performance of these additional codecs compared to the\nbest-performing codecs from Fig. 3, DNA-Aeon by Welzel et al.6 and DNA-RS by Heckel et al.7,8\nInterestingly, DBGPS by Song et al.5 achieved a similar performance to DNA-Aeon and DNA-RS\nin most scenarios. This highlights the value of standardized benchmarks for codec comparisons\nand suggests using a balanced inner/outer code separation strategy is more important than\nchoosing a specific error-correction code and implementation.”\n2. Simplified error models used for synthetic benchmarking: This is very similar to what has been done\nin the literature and raises the question of novelty. This approach, while standardized, potentially\noverlooks critical edge-case behaviors that would be revealed under more diverse or adaptive error\ndistributions, e.g., specific motifs that affect sequencing or synthesis.\nReviewer 2 also remarked on the perceived simplicity of our synthetic benchmarks in their first\ncomment above. As discussed in more detail in our response there, we’d like to clarify that we use two\ndifferent error models for our synthetic benchmarks. A simple model, as commonly used in literature\n(i.e., fixed error rates and sequence dropout), is used only for the initial analyses in Fig. 1. The\nsubsequent analyses in Figs. 2 and 3 use a complex error model based on our previous comprehensive\nerror characterization in Ref. 12. This includes diverse and complex error patterns which have not been\nused previously for codec comparisons, such as error runs, skewed coverage distributions, PCR biases,\nand positional biases.\nMoreover, we have verified the results obtained from the complex error model with experiments,\nshowing good agreement. This highlights how closely our complex model represents experimental\nsequencing data. This confirmation of benchmark results using experiments covering multiple codecs\nis also novel in itself.\nTo better highlight the complexity of our synthetic benchmarks and underline the novelty in their use\nfor codec benchmarking, we rephrased the corresponding paragraphs in the introduction and results\n(ll. 89-93 and ll. 274-280):\n“For our in silico benchmarks, we first employ simple synthetic error scenarios (i.e., applying\nfixed error rates) commonly used in the literature. Then, we extend our analysis with\ncomprehensive benchmarks considering the full spectrum of non-idealities present in real-\nworld sequencing data (using DT4DDS12), and verify our results with standardized in vitro\nexperiments”\n“Despite the common use of these simple synthetic benchmarks for codec benchmarking in\nthe literature, they overlook the non-ideal error patterns and biases present in experimental\nsequencing data – from error runs to skewed coverage distributions.12–14 Thus, for all further\nbenchmarks, we implemented realistic experimental workflows using the models\nimplemented in the simulation software DT4DDS12 (see Methods). This simulation software\ntakes into account the aforementioned non-idealities which are missing from the synthetic\nbenchmarks previously performed in the literature.”\n3. Fixed parameter sets without optimization: The study relies on fixed parameter sets for each code\nwithout performing any code-specific parameter optimization. As a result, some codes may appear to\nunderperform not due to intrinsic limitations but because their configurations were suboptimal for the\ntested conditions. This could be addressed by including a sensitivity analysis or an optimization run to\nassess how performance scales with parameter tuning.\nThe reviewer raises an important point about the choice of codec parameters throughout our study.\nWe agree with the reviewer that the lack of parameter optimization may have led to the use of\nsuboptimal parameter sets for some codec-scenario combinations, which we also acknowledged in our\noriginal discussion. However, our goal with this study is to compare existing codecs from an end-user\nperspective. Following this perspective, we always start from each codecs’ default settings and refrain\nfrom extensive parameter optimization, which a common end-user would not have the ability to\nperform due to a lack of a priori knowledge of the error rates encountered in an experiment (i.e.,\nusually codec parameters are estimated and not optimized, as there is no opportunity to test codec\nparameters prior to the experiment).\nMoreover, we assume the original studies’ authors have performed an appropriate selection of their\ncodecs’ parameters for common DNA data storage scenarios when publishing their study. Our changes\nto these parameters to achieve the standardized code rates and sequence lengths thus only change\nthe overall redundancy level, while leaving other parameters, such as the balance between inner and\nouter redundancy, at approximately the values selected by the original authors.\nTo better highlight this intention and explain the rationale of using the codec settings as selected by\nthe original authors as baselines for our parameter choices, we added a paragraph to the beginning\nof the results (see ll. 119-127):\n“Importantly, codec parameters were not optimized specifically for this study. Instead, the\nparameters selected by the codecs’ original authors were only adjusted to achieve\nstandardized code rates and similar sequence lengths. This effectively changed only each\ncodec’s overall redundancy level (via the code rate), whereas other codec features – such as\nthe balance between inner and outer redundancy – were kept at approximately the level\nselected by the original authors. While this approach might lead to suboptimal codec\nconfigurations in our benchmarks, this approach mimics the use case for experimentalists and\nmatches each codec’s authors’ selected implementation most closely. This was deemed\nimportant, as the codecs’ original authors were assumed to have estimated their ideal\nparameters based on their specific use cases in DNA data storage.”\n4. Lack of computational performance analysis: A detailed analysis of runtime and memory usage is\nalso required. A comprehensive benchmark should include not only error-correction capabilities but\nalso computing performance measures. For real-world DNA storage systems, the speed and scalability\nof decoding pipelines are crucial performance metrics, and omitting them leaves a gap in the overall\nassessment.\nWe agree with the reviewer’s assessment regarding the importance of performance measures for a\ncomprehensive benchmark. This is the reason why we already enforced both runtime (60 min),\nmemory (8 GB RAM), and computing (1 CPU core) constraints during all of our experiments.\nSupplementary Figures 4-7 and 9-13 show the dependence of both runtime and memory usage on\nthe investigated experimental parameters (error rate, sequence loss, sequencing depth, physical\nredundancy). We have now also added a comparison of the computational performance to the results\nsection (see ll. 259-263, 380-381) and include an overview in the discussion (see ll. 534-540), both\nreproduced below:\n“The runtime constraint also limited codec performance of DNA-Aeon, DNA-RS, and HEDGES\nin this scenario (see Supplementary Fig. 11). Interestingly, both DNA-Aeon and HEDGES were\nonly limited by time at high error rates while operating unconstrained at high sequence\ndropout, suggesting their implementations of within-sequence redundancy caused excessive\nruntimes. As previously, memory use did not limit any codec in this scenario. […]. This suggests\nthat also in this benchmark, high error-correction capability is in a direct trade-off with\ncomputational power. Expectedly, memory use did not limit codec performance in either\nscenario.”\n“The analysis of the computational performance throughout all benchmarks in this study (see\nSupplementary Figs. 4-7 and 9-13) shows that only DNA-Aeon, DNA-RS, and HEDGES were\nlimited by the runtime constraint at all. However, runtime generally increased sharply as error\nrates approached each codec’s limits, such that, in many cases, codecs failed to successfully\ndecode the data before being limited by our time constraint. Nonetheless, codecs’ runtime\nand parallelizability should become an important metric for consideration as data volumes for\nDNA data storage continue to increase.”\nSupplementary Figure 9: Decoding time as a function of error rate in the basic error scenario with individual errors. The\nruntime of the decoding step is shown for the DNA-Aeon (a), DNA Fountain (b), DNA-RS (c), Goldman (d), HEDGES (e), and\nYin-Yang (f) codecs at all used code rates, when substitutions (red), deletions (blue), or insertions (green) are introduced\nindividually. Points correspond to individual runs of the pipeline at the specified error rate and error type. Open circles denote\nindividual runs which failed the decoding step, either due to violation of the runtime constraint or due to insufficient error-\ncorrection capabilities.\nSupplementary Figure 10: Memory use as a function of error rate in the basic error scenario with individual errors. The\nmemory use of the decoding step is shown for the DNA-Aeon (a), DNA Fountain (b), DNA-RS (c), Goldman (d), HEDGES (e), and\nYin-Yang (f) codecs at all used code rates, when substitutions (red), deletions (blue), or insertions (green) are introduced\nindividually. Points correspond to individual runs of the pipeline at the specified error rate and error type. Open circles denote\nindividual runs which failed the decoding step, either due to violation of the runtime constraint or due to insufficient error-\ncorrection capabilities.\n5. Codec–clustering algorithm pairing: The pairing between codes and clustering algorithms\nsignificantly impacts performance. While the paper tests various clustering strategies and selects the\nbest-performing one for each code, it does not fully explore whether these pairings are theoretically\noptimal or practically justified. In some cases, general-purpose tools like CD-HIT outperformed\nspecialized algorithms, which raises questions about code-specific biases or incompatibilities with\nclustering strategies. A deeper investigation into why certain pairings fail, or how code design interacts\nwith clustering, would have added depth to the analysis.\nWe support the reviewer’s comments regarding the importance of the codec-clustering pairing. While\nwe had already mentioned the possibility of certain code-specific biases as the reason for the low\nperformance of some codec-clustering pairings, we have performed a deeper investigation into these\nbiases as Supplementary Figs. 1-3 and Supplementary Table 4. As suspected by the reviewer, certain\npairings fail due to the interaction between a codec’s sequence design and some clustering algorithm’s\naggressive merging of clusters. This causes a pronounced increase in sequence dropout for these\ncodec-clustering pairings, drastically reducing decoding performance, see Supplementary Table 4.\nTo highlight this detailed investigation into codec-clustering pairings, we added a paragraph outlining\nits findings to the results section (see ll. 217-230):\n“Interestingly, nine of the thirteen tested codecs and code rates performed best with the\nestablished general-purpose clustering algorithm CD-HIT (red bars in Fig. 1d). Unsurprisingly,\nthis general preference for CD-HIT was caused by this clusterer’s superior performance,\nyielding the fewest and least erroneous clusters across all tested clustering algorithms up to\nan overall error rate of around 10% (see Supplementary Figs. 1-3). In addition, several\ncombinations of clustering algorithms and codecs failed completely (see Supplementary Table\n3 for full results). Further investigation suggests incompatibilities exist between these\nclustering algorithms and the sequence features generated by some codecs (e.g., indexing\nregions, overlapping sections in the Goldman codec). Specifically, some codec-clustering\npairings exacerbate sequence loss by clustering reads from different design sequences\ntogether. In some cases, up to 92% of sequences present in the sequencing data no longer\nappeared in the clustered reads, explaining the incompatibility of such codec-clustering\npairings. Thus, the ideal clustering algorithms must reduce the error rate in consensus reads\nefficiently, thereby reducing the workload for the inner decoder, without causing sequence\nloss themselves, which drastically burdens codecs’ outer decoders.”\nSupplementary Figure 3: Relationship between the number of clusters for each reference seqeunce and the applied error\nrate during the basic error scenario for each codec-clustering pairing. For each investigated codec (rows), code rate\n(columns), and clustering algorithm (color), the number of generated clusters for each reference sequence is shown. As the\nbasic error scenario generates exactly 30 erroneous reads per reference sequence, a low number of clusters per sequence is\ndesirable. However, all clustering algorithms generate multiple clusters for each reference sequence as the applied error rate\nincreases.\nSupplementary Table 4: Sequence recovery after clustering 30 perfect reads per reference sequence\nfor each codec-clustering pairing.\nCodec Code rate Naïve CD-HIT Clover LSH Starcode\nbit nt-1 % % % % %\n0.50 100 100 98.2 100 100\nDNA-Aeon 1.00 100 100 98.6 100 100\n1.50 100 100 99.2 100 100\n0.50 100 99.7 97.8 100 100\nDNA Fountain 1.00 100 99.9 98.6 100 100\n1.50 100 100 99.1 100 100\n0.50 100 100 98.4 100 100\nDNA-RS 1.00 100 100 98.2 100 100\n1.50 100 100 98.2 100 100\nGoldman 0.34 100 98.6 79.1 96.3 100\n0.63 100 96.5 8.1 99.4 98.0\nHEDGES\n1.07 100 99.5 92.4 99.8 99.5\nYin-Yang 1.85 100 100 98.7 100 100\n6. Importance of motifs in realistic evaluation: It is also worth mentioning that motifs are highly\nimportant for any realistic evaluation. There are known sequence motifs that interact with enzymes\nduring sequencing and also some affecting DNA synthesis. PCR and other processes are also influenced\nby sequence motifs (or more precisely, by how these motifs interact with enzymes). This important fact\nshould be more prominently highlighted, as only one code in the tested list can handle motifs. This\ndirectly relates to my first point: the selection of codes. There are other published codes that can handle\nsequence motifs. A fair, and more importantly, a realistic comparison would include motif-aware codes.\nA code that is not able to handle motifs will inevitably fail from time to time in practice, especially when\nlarge amounts of data are encoded.\nWe agree with the reviewer that sequence motifs (together with other sequence properties such as\nGC content, homopolymer length, and secondary structure) can bias the enzymatic steps involved in\nDNA data storage. However, based on our review of the literature, quantitative insight into the extent\nand relevance of sequence motifs for the enzymatic steps is still lacking. Without quantitative data on\nthe effects of sequence motifs however, we consider it difficult to accurately include them in our\nanalysis, without unduly over- or understating the benefits of motif-aware codecs. Nonetheless, we\nhave now performed a new sensitivity analysis which shows the impact of motif-driven biases in\nerror rate and sequence loss (see Supplementary Fig. 19), using the motifs implemented in the MESA\nDNA Simulator by Schwarz et al.16\nWe find that motif-driven biases in both error rate and sequence loss can have considerable impact on\ndecoding performance, if they are sufficiently strong (i.e., the likelihood of errors in or sequence loss\nof a motif-containing sequence is increased by >10x). We have highlighted our analysis on the effects\nof sequence motifs to the discussion (see ll. 546-549):\n“For example, only the DNA-Aeon codec can be constrained to avoid biological motifs (see\nTable 1), which can negatively affect codec performance if motif-containing sequences are\nconsiderably more prone to errors or dropout (see Supplementary Fig. 19).”\nSupplementary Figure 19: Benefits of motif avoidance during error-correction coding. The DNA-Aeon codec supports\ncustom codebooks to avoid the introduction of user-defined motifs into the design sequences. To test the potential benefit\nof motif avoidance, a baseline scenario of 3% errors (53% substitutions, 45% deletions, 2% insertions) and 1% sequence\ndropout was modified such that all sequences containing any of the 46 motifs with biological meaning defined in the MESA\nDNA Simulator by Schwarz et al.16 either had drastically increased error rates (a) or were drastically more likely to drop out\nof the sequencing data (b). Using a custom codebook avoiding these motifs with DNA-Aeon (right-most bar) precluded any\nnegative effect of increased error rates or sequence loss, as expected.\nIn conclusion, the paper provides a valuable and ambitious framework for standardized code\nbenchmarking in DNA data storage. However, its conclusions are somewhat constrained by the limited\ncode selection, oversimplified error models, lack of parameter optimization, and absence of\ncomputational performance metrics. Expanding the scope and granularity of the evaluation would\nfurther enhance the robustness and applicability of the findings.\nWe thank the reviewer for highlighting the value of our study and believe our changes have sufficiently\nexpanded the scope and granularity of our benchmarking. We believe our efforts to expand the\nanalysis of codec runtime and introduce an investigation into codec-clustering pairings considerably\nimproved the applicability of our findings.\nReferences\n1. Zan, X., Xie, R., Yao, X., Xu, P. & Liu, W. A Robust and Efficient DNA Storage Architecture Based on\nModulation Encoding and Decoding. J. Chem. Inf. Model. 63, 3967–3976 (2023).\n2. Su, Y. et al. A Robust and Efficient Representation-based DNA Storage Architecture by Deep\nLearning. Small Methods 9, 2400959 (2025).\n3. Zheng, X. et al. A generative adversarial network for multiple reads reconstruction in DNA storage.\nSci. Rep. 14, 32071 (2024).\n4. Chandak, S. et al. Improved read/write cost tradeoff in DNA-based data storage using LDPC codes.\nin 2019 57th Annual Allerton Conference on Communication, Control, and Computing 147–156\n(2019).\n5. Song, L. et al. Robust data storage in DNA by de Bruijn graph-based de novo strand assembly. Nat.\nCommun. 13, 1–9 (2022).\n6. Welzel, M. et al. DNA-Aeon provides flexible arithmetic coding for constraint adherence and error\ncorrection in DNA storage. Nat. Commun. 14, 628 (2023).\n7. Antkowiak, P. L. et al. Low cost DNA data storage using photolithographic synthesis and advanced\ninformation reconstruction and error correction. Nat. Commun. 11, 5345 (2020).\n8. Meiser, L. C. et al. Reading and writing digital data in DNA. Nat. Protoc. 15, 86–101 (2019).\n9. Erlich, Y. & Zielinski, D. DNA Fountain enables a robust and efficient storage architecture. Science\n355, 950–954 (2017).\n10. Zhao, X. et al. Composite Hedges Nanopores codec system for rapid and portable DNA data\nreadout with high INDEL-Correction. Nat. Commun. 15, 9395 (2024).\n11. Ping, Z. et al. Towards practical and robust DNA-based data archiving using the yin–yang codec\nsystem. Nat. Comput. Sci. 2, 234–242 (2022).\n12. Gimpel, A. L., Stark, W. J., Heckel, R. & Grass, R. N. A digital twin for DNA data storage based on\ncomprehensive quantification of errors and biases. Nat. Commun. 14, 6026 (2023).\n13. Heckel, R., Mikutis, G. & Grass, R. N. A Characterization of the DNA Data Storage Channel. Sci. Rep.\n9, 1–12 (2019).\n14. Chen, Y.-J. et al. Quantifying molecular bias in DNA data storage. Nat. Commun. 11, 1–9 (2020).\n15. Ceze, L., Nivala, J. & Strauss, K. Molecular digital data storage using DNA. Nat. Rev. Genet. 2019\n208 20, 456–466 (2019).\n16. Schwarz, M. et al. MESA: automated assessment of synthetic DNA fragments and simulation of\nDNA synthesis, storage, sequencing and PCR errors. Bioinformatics 36, 3322–3326 (2020)."}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have solved most of my concerns, but I still concern the following issues:\n1. I insist that the authors should select real SOTA codec to make comparison in the main text instead of adding some results in supplementary section.\n2. I suspect the result in Supplementary Figure 21 (a) and (b) about the error tolerance rate about modulation based codec as it has a supper error correction capability than any current codecs.\n3. In fig 1(d), concerning the error rate per nt, I want to know what the error types is, it means the sum of the insertion-deletion-substitution?further, what the ratio of the three errors? this is very important as I mention last time that in DNA storage, the unique error pattern is the complex IDS errors which makes the information retrieval is very challenge.\n4. As this paper intend to make a comprehensive comparison of SOTA codes in error correction, it is very valuable to discuss which codec have the potential to deal with future large scale applications considering cost, time and error capability.", "reviewer_2": "In my view, the authors have made a substantial effort to address the concerns raised in the first round of reviews. Given the large number of points raised, I will focus primarily on those I emphasized in my initial review. I find that these have been addressed to my satisfaction, and the quality of the paper has improved considerably compared to the original submission.", "reviewer_3": "The authors addressed my concerns adequately."}, "author_response": "Response to Referees\nReviewer comments in italics, author replies in red with actions bolded.\nLine numbers refer to the revised manuscript, in which changes have been highlighted in yellow.\nComments by Referee #1\nThe authors have solved most of my concerns, but I still concern the following issues:\n1. I insist that the authors should select real SOTA codec to make comparison in the main text instead\nof adding some results in supplementary section.\nWe follow the reviewer’s suggestion to move the comparison of other codecs to the main text. We\nhave therefore moved Supplementary Fig. 21 to the main manuscript, as the new Fig. 5. In addition,\nwe have considerably extended our discussion of this comparison to other codecs in the results\nsection (ll. 479-509):\n“Finally, to highlight the utility of the developed benchmarking pipeline for codec comparisons,\nthe scope of literature codecs was extended to cover additional codecs by Song et al. (DBGPS)1,\nChandak et al. (LDPC)2 and Zan et al. (Modulation)3. Figure 5 shows the results of the in silico\nbenchmarks for these three additional codecs, with the results by the best-performing codecs\nfrom the main study – DNA-Aeon by Welzel et al.4 and DNA-RS by Heckel et al.5,6 – also shown\nfor comparison. In line with the results from Fig. 1d, clustering by CD-HIT drastically improved\nerror tolerance across these additional codecs, see Fig. 5a. Moreover, these codecs’ ability to\ncorrect individual error types did not differ from the results of DNA-Aeon and DNA-RS (see Fig.\n5b).\nImportantly, the Pareto fronts across code rates in Fig. 5c+d show that DBGPS by Song et al.1\nperformed similarly to DNA-Aeon and DNA-RS in most cases. Thus, its performance is on-par\nwith these state-of-the-art codecs, to within the experimental error associated with the lack\nof parameter optimization in this study. In contrast, both LDPC by Chandak et al.2 and the\nmodulation-based codec by Zan et al.3 fall behind the performance of DNA-Aeon, DNA-RS,\nand DBGPS in our in silico benchmarks.\nTaken together, the extension of our in silico pipeline to three additional codecs from the\nliterature demonstrates its simplicity and utility for codec benchmarking, without requiring any\nexperimental effort. In addition, the results obtained for the DBGPS codec by Song et al.1\nfurther support the observation that a well-executed, balanced inner/outer code separation\nstrategy is more important for codec performance than the specific choice of employed error-\ncorrection code. Thus, these results showcase both how error-correction codecs for DNA data\nstorage are optimally constructed and how their performance is realistically assessed.”\n2. I suspect the result in Supplementary Figure 21 (a) and (b) about the error tolerance rate about\nmodulation based codec as it has a supper error correction capability than any current codecs.\nIn our analysis, under the expected error levels given, the modulation-based codec by Zan et al.3\nperforms worse than other codecs from the literature from our selection. We note that the difference\nin performance between our study and the study by Zan et al.3 is likely caused by two reasons:\n1. The modulation-based codec by Zan et al.3 does not implement any redundancy for error-\ncorrection on the data level (such as the Reed-Solomon or Fountain codes by other codecs,\nsee Table 1). Instead, it relies fully on the error-correcting capability of clustering and sequence\nalignment. As a result, it fails to decode the data if any design sequence is missing or contains\nany error after consensus generation.\n2. The codec implementation provided by Zan et al.3 on GitHub does not perform sequence\nindexing. Thus, decoding also fails if the decoder receives sequences out of order.\nAs a result, the codec is missing from the Pareto plots in Fig. 5c+d, which show benchmarks including\nboth sequence dropout and unordered output. We now include a detailed discussion of differences\nin performance assessments between codecs and their limitations in the results section (ll. 494-502):\n“In the case of the modulation-based codec by Zan et al.3, its lack of data-level error correction\nrenders it comparable to the Yin-Yang codec by Ping et al.7 investigated in our initial codec\nbenchmark. Reassuringly, both of these codecs relying solely on clustering and sequence\nalignment for error correction achieved similar tolerated error rates of around 4-5% in our test\n(compare Fig. 1d and 5a). However, contrary to the Yin-Yang codec by Ping et al.7, the\nimplementation of the modulation-based codec provided in the repository by Zan et al.3 does\nnot include sequence indexing. As a result, it cannot recover the stored data if sequences are\nrecovered out of order, unless sequence indexing is implemented separately from the codec.\nThese two limitations of the modulation-based codec cause the discrepancy in observed\nperformance between Zan et al.3 and the benchmark in Fig. 5.“\n3. In fig 1(d), concerning the error rate per nt, I want to know what the error types is, it means the sum\nof the insertion-deletion-substitution? further, what the ratio of the three errors? this is very important\nas I mention last time that in DNA storage, the unique error pattern is the complex IDS errors which\nmakes the information retrieval is very challenge.\nThe reviewer’s interpretation of the error rate in Fig. 1d is correct. The error rate in this benchmark is\nthe sum of the insertion, deletion, and substitution rates. Their ratio is fixed at 53% substitutions, 45%\ndeletions, and 2% insertions. This resembles the error pattern we observed experimentally in Ref. 8,\nand is similar to the error pattern we observed experimentally in this study (see Suppl. Fig. 15a+b). We\nhave added this important information to the figure legend (ll. 211-213):\n“The error rate shown corresponds to the combined rate of substitutions, deletions, and\ninsertions at a fixed composition of 53% substitutions, 45% deletions, and 2% insertions,\nthereby resembling the experimental error pattern in Ref. 8.”\n4. As this paper intend to make a comprehensive comparison of SOTA codes in error correction, it is\nvery valuable to discuss which codec have the potential to deal with future large scale applications\nconsidering cost, time and error capability.\nThe reviewer raises an important point regarding the suitability of the different codecs for large-scale\napplications in the future. DNA data storage is still in its infancy, but one can observe that coding\nprinciples previously developed for other problems perform well for DNA data storage, if appropriately\nadapted. We have now included additional discussion on this observation, together with the\nremaining open problems of insertion/deletion correcting codes and broken sequence\nreconstruction. (ll. 580ff):\n\"Our analysis shows that codec comparison and benchmarking is important for discussing new\ncoding schemes, and our in silico pipeline contributes suitable benchmarking tools to the\ncurrent literature. In addition, our analysis demonstrates that although DNA data storage\nrepresents an error channel with error patterns different to preceding data storage\ntechnologies, error-correcting principles developed for existing channels remain highly\nrelevant. Consequently, the state-of-the-art codes for DNA data storage leveraging existing\nReed-Solomon-, Fountain- and LDPC-codes perform best if in-sequence error correction is\ncombined with approaches to reconstruct lost sequences and compensate for remaining\nerrors within the sequences. An additional advantage of using pre-existing coding principles is\nthe certainty that such codes can be brought to production level coding/decoding speeds with\nappropriate implementation into dedicated hardware, as e.g. in CD-ROM drives.9 Under the\nDNA data storage scenarios investigated in this work, constrained coding approaches do not\nhave competitive edge over non-constrained coding, as also discussed in detail by Weindel et\nal.10 However, better codes for correcting deletions and insertions might be beneficial for\nfuture DNA storage systems and allow data reconstruction in error prone scenarios even\nwithout the current aligning of multiple reads for insertion/deletion correction.\"\nReferences\n1. Song, L. et al. Robust data storage in DNA by de Bruijn graph-based de novo strand assembly. Nat.\nCommun. 13, 1–9 (2022).\n2. Chandak, S. et al. Improved read/write cost tradeoff in DNA-based data storage using LDPC codes.\nin 2019 57th Annual Allerton Conference on Communication, Control, and Computing 147–156\n(2019).\n3. Zan, X., Xie, R., Yao, X., Xu, P. & Liu, W. A Robust and Efficient DNA Storage Architecture Based on\nModulation Encoding and Decoding. J. Chem. Inf. Model. 63, 3967–3976 (2023).\n4. Welzel, M. et al. DNA-Aeon provides flexible arithmetic coding for constraint adherence and error\ncorrection in DNA storage. Nat. Commun. 14, 628 (2023).\n5. Antkowiak, P. L. et al. Low cost DNA data storage using photolithographic synthesis and advanced\ninformation reconstruction and error correction. Nat. Commun. 11, 5345 (2020).\n6. Meiser, L. C. et al. Reading and writing digital data in DNA. Nat. Protoc. 15, 86–101 (2019).\n7. Ping, Z. et al. Towards practical and robust DNA-based data archiving using the yin–yang codec\nsystem. Nat. Comput. Sci. 2, 234–242 (2022).\n8. Gimpel, A. L., Stark, W. J., Heckel, R. & Grass, R. N. A digital twin for DNA data storage based on\ncomprehensive quantification of errors and biases. Nat. Commun. 14, 6026 (2023).\n9. Geisel, W. A. Tutorial on Reed-Solomon Error Correction Coding.\nhttps://ntrs.nasa.gov/citations/19900019023 (1990).\n10. Weindel, F., Gimpel, A. L., Grass, R. N. & Heckel, R. Embracing Errors Can Be More Efficient Than\nAvoiding Them Through Constrained Coding for DNA Data Storage. IEEE Trans. Mol. Biol. Multi-\nScale Commun. 1–1 (2025) doi:10.1109/TMBMC.2025.3610330."}, {"round": 3, "reviewer_comments": {"reviewer_1": "The authors have solved part of my concerns, but I still have some main concerns:\n1. The YY code has no error correction ability, but your result shows it tolerates about 4% errors (Fig 1d). Could you explain the reason? Moreover, why did you select it as a SOTA method to study the error correction capability in this paper?\n2. AI-based techniques have been introduced to solve the error issues in DNA storage, and recent studies showed that they have promising results both in error correction, data compression and encoding etc. But this paper neglects the function of these studies. As I know, compared with the traditional methods used in this paper, deep learning methods are more suitable for future DNA storage.\n3. Concerning the modulation-based method, I agree it could not solve sequence loss, but HEDGES and DBGPS also can not deal with such issues. I still suspect the reported results in this paper.\n4. Last, the loss issue in DNA storage is usually solved by fountain code or inner-outer code strategy. Therefore, base errors in sequences are usually corrected by various methods, which may combine the fountain code or inner-outer code strategy."}, "author_response": "Response to Referees\nReviewer comments in italics, author replies in red with actions bolded.\nLine numbers refer to the revised manuscript, in which changes have been highlighted in yellow.\nComments by Referee #1\nThe authors have solved part of my concerns, but I still have some main concerns:\n1. The Yin-Yang code has no error correction ability, but your result shows it tolerates about 4% errors\n(Fig 1d). Could you explain the reason? Moreover, why did you select it as a SOTA method to study\nthe error correction capability in this paper?\nThe Yin-Yang codec by Ping et al.1 does indeed not have error-correction capabilities itself. As already\ndescribed in ll. 187-190 of the manuscript, this codec nevertheless tolerates around 4% of errors in our\nbasic error scenario because of the error-correcting effect of clustering and consensus generation. Up\nto an error rate of 4%, the consensus reads after clustering do not contain any errors and thus no\ncodec-level error correction is required to recover the information.\nWe have extended the existing explanation to highlight how the Yin-Yang codec is able to recover\nthe data at this error rate (ll. 190-192):\n“Specifically, the clustering step already completely eliminates all errors in the consensus reads\nup to this error rate (see also Supplementary Fig. 1), removing the need for any codec-level\nerror correction for error-free data recovery.”\nWe note that the Yin-Yang codec by Ping et al.1 was not selected as a SOTA method. It was included\nfor benchmarking in this study, but only the best-performing codecs (DNA-Aeon, DNA-RS, and DBGPS)\nwere selected as the state-of-the-art in the manuscript. Perhaps the inclusion of the Yin-Yang codec in\nTable 1 of the manuscript was misleading. However, this table does not list the codecs which we\nidentify as the state-of-the-art. Rather, it lists the codecs which were included in the main\nbenchmarking study.\nTo make the distinction between codecs included in the study and those selected as the state-of-\nthe-art clearer, we have adjusted the caption of Table 1 (ll. 102-105):\n“Table 1: Codecs included in the in silico and in vitro benchmarks, as well as their properties.\nClustering denotes the clustering approach used in the original study, if any. The pairing of\ncodecs with clustering algorithms used in this study is detailed in Supplementary Table 3. Based\non the benchmarking results (see below), only the DNA-Aeon and DNA-RS codecs were\nselected as the current state-of-the-art.”\n2. AI-based techniques have been introduced to solve the error issues in DNA storage, and recent studies\nshowed that they have promising results both in error correction, data compression and encoding etc.\nBut this paper neglects the function of these studies. As I know, compared with the traditional methods\nused in this paper, deep learning methods are more suitable for future DNA storage.\nWe agree with the reviewer that machine learning approaches are actively being investigated for DNA\ndata storage. Unfortunately, we were unable to include such codecs despite our best efforts because\nthe corresponding codes and models are not available.\nSpecifically, the studies on deep learning-based methods by Su et al.2 and Zheng et al.3 did not deposit\nthe corresponding code, rendering it impossible for us to include these codecs in our study. While the\nstudy by Bar-Lev et al.4 did deposit the corresponding code, we were unable to use their code due to\npoor documentation and multiple implementation challenges. Specifically, the code contains many\nhard-coded paths and requires multiple compilations of C++ code during the encoding and decoding\nsteps. This lack of code portability rendered implementation of this codec impossible on our computing\ncluster. Finally, retraining of the deep learning models would be required to obtain representative\nperformance, which would be outside the scope of our study.\nTo facilitate this study, we had to define criteria to determine which codes to include into the study,\nas described in ll. 111-115. These criteria were:\n- The full open-source implementation of the code is available. While we have performed\ntweaking of software implementations (see Supplementary Note 1), it was outside the scope\nof this comparative study to re-implement or adapt codecs, for which not code is available.\n- The corresponding study includes in vitro experiments to demonstrate suitability for realistic\nexperimental workflows.\n- The corresponding study was published prior to the cut-off date of October 2023 (when we\nstarted the study). As our study includes vast testing of software as well as in vitro\nexperiments, which took considerable time.\nApart of not being reproducible by us, the study by Bar-Lev et al.4 also does not fulfil the last criterium.\nThus, we added a paragraph to the discussion to note the ongoing efforts to improve codecs using\ndeep learning and to highlight how our benchmarking approach can help with ensuring code\nreproducibility (ll. 605-608):\n“For example, recent efforts to employ deep learning in DNA data storage2–4 were not covered\nin this study, but could benefit from the benchmarking scheme proposed in this study to\ncomprehensively assess their scalability and transferability.”\n3. Concerning the modulation-based method, I agree it could not solve sequence loss, but HEDGES and\nDBGPS also can not deal with such issues. I still suspect the reported results in this paper.\nWhile not a key focus of these studies, the implementations of both the HEDGES codec by Press et al.5\nand the DBGPS codec by Song et al.6 can tolerate and correct sequence loss, contrary to the reviewer’s\nstatement. This is shown both by the results in the manuscript (see Fig. 1c for HEDGES and Fig. 5c for\nDBGPS) and noted in the original studies of these codecs:\n- In Press et al.5, the authors state: “we used an outer-code concatenated design with packets\nof 255 strands of length 300 protected by RS(255,223)”5 and “erasures (unknown bits or bytes)\n[…] can be corrected in the outer code”5. The authors also state that the deposited code\nimplements an outer RS code: “The computer code used for the generation and testing of the\ninner HEDGES code and outer RS code is available at https://github.com/whpress/hedges.”5\n- In Song et al.6, the authors state: “The strand redundancy was set to 7.8%, which supports\nreliable data recovery when the decoder receives more than 95% of strands”6 and “raw reads\ngenerated were processed by DBGPS for strand reconstruction followed by decoding using the\nouter fountain codes”6. The authors also state that the deposited code implements an outer\nFountain code: “Python implementation of DBGPS using fountain codes as outer codes is\navailable at https://doi.org/10.5281/zenodo.6833784”6 (see SI)\n4. Last, the loss issue in DNA storage is usually solved by fountain code or inner-outer code strategy.\nTherefore, base errors in sequences are usually corrected by various methods, which may combine the\nfountain code or inner-outer code strategy.\nWe agree with the reviewer that sequence loss in DNA data storage is commonly addressed using\nfountain codes and an inner-outer code architecture, as noted in the manuscript (e.g., ll. 55-57).\nHowever, it is important to emphasize that implementing these mechanisms to mitigate sequence loss\nis a fundamental requirement for any practical DNA data storage codec. For this reason, we believe it\nis valuable to point out that some of the evaluated codec implementations lack this crucial capability,\nas doing so underscores the significance of sequence loss and provides important information for\nusers.\nWe have added an explanation about the potential retrofitting of codecs with an outer code, and\nour rationale against considering this in our benchmarks, to the manuscript in ll. 505-509:\n“Naturally, features such as sequence indexing and an outer code for tolerance to sequence\nloss can be retrofitted to existing codecs, using established methods like Fountain or Reed-\nSolomon codes. However, such adaptions to existing codec implementations require\nadditional programming, testing and optimization efforts, which is not realistic for most use\ncases and was therefore not in scope of this study.”\nReferences\n1. Ping, Z. et al. Towards practical and robust DNA-based data archiving using the yin–yang codec\nsystem. Nat. Comput. Sci. 2, 234–242 (2022).\n2. Su, Y. et al. A Robust and Efficient Representation-based DNA Storage Architecture by Deep\nLearning. Small Methods 9, 2400959 (2025).\n3. Zheng, X. et al. A generative adversarial network for multiple reads reconstruction in DNA storage.\nSci. Rep. 14, 32071 (2024).\n4. Bar-Lev, D., Orr, I., Sabary, O., Etzion, T. & Yaakobi, E. Scalable and robust DNA-based storage via\ncoding theory and deep learning. Nat. Mach. Intell. 7, 639–649 (2025).\n5. Press, W. H., Hawkins, J. A., Jones, S. K., Schaub, J. M. & Finkelstein, I. J. HEDGES error-correcting\ncode for DNA storage corrects indels and allows sequence constraints. Proc. Natl. Acad. Sci. 117,\n18489–18496 (2020).\n6. Song, L. et al. Robust data storage in DNA by de Bruijn graph-based de novo strand assembly. Nat.\nCommun. 13, 1–9 (2022)."}]

Decision letter