Peer review of: Effect of sequence depth and length in long-read assembly of the maize inbred NC358 (2019)

Reviewers' comments: 

Reviewer #1 (Remarks to the Author): 

In the manuscript entitled “Effect of Sequence Depth and Length in Long-read Assembly of the Maize 
Inbred NC358,” Ou et al., leverage a high-quality set of Pacific Bioscience reads to develop guidelines 
for sequencing and assembly for a complex plant genome. The authors generate a high coverage, long 
read dataset, and analyze eight different assemblies varying read depth and read length N50 with 
three different assembly approaches. They scaffold the assemblies using BioNano optical maps and 
anchor into chromosomes using GoldenGate and pan-genome genetic markers. Then the authors carry 
out some very nice comparisons of the genome assemblies looking at completeness (BUSCO), 
complex repeats (centromeres, rDNA, etc.) and transposable elements. The authors argue, and not 
surprisingly, that higher coverage and longer reads generate higher quality assemblies. Specifically, 
they show for a plant that has a predominant 10kb repeat structure, reads exceeding that repeat 
length and coverage enable assembly of more of the overall repeat content. Specifically, the use of 
BioNano maps to validate the assembly contiguity and repeat content is interesting and compelling. 
Overall, the concept of this manuscript is on point and useful to the genome assembly and plant 
communities. However, the separation in assemblies could be broader, more assembly methods could 
be tested, and the results need to be placed more clearly in the current state of the art in plant 
genome assembly. 

The introduction is missing some key background to set the stage for this work. First off, the authors 
do not introduce the current state genome assembly using long reads in terms of platforms, tools and 
results. Multiple papers have addressed the different long read assemblers and approaches; alerting 
the reader to this background will greatly improve the introduction. There are many genomes 
published using long reads at this point, some that are extremely high quality such the telomere to 
telomere rice genome, and it would put the current work into context based on these previous results. 
Moreover, background on the Maize genome and its complexity would also place the current work in 
context. The last author published the B73v4 PacBio assembly and some introduction to this work 
would allow the reader to understand the complexity of the Maize genome. The authors compare the 
Arabidopsis and wheat genomes; this comparison is less relevant to the current work since the Maize 
genome is diploid and complex in a different way than wheat. 

In terms of methodology it would greatly increase the power of this analysis if the authors extend 
their analysis to different long read assemblers, and mixes of data that create greater difference in the 
resulting assemblies. The authors focus on Canu and Falcon as well as a hybrid assembly method 
(really just a difference in error correction), but they don’t test several other popular long read 
assemblers like FlyE, WTDBG2, MECAT2 and minimap2/miniasm. Also, the authors only tested 
assemblies post error correction. The other long read assemblers mentioned above also work on 
uncorrected read, which would be a very interesting and important comparison since error correction 
consumes a lot of compute time. Recently the developer of Falcon released a new assembler called 
Peregrine based on minimizers of minimizers that works on high-quality PacBio reads (really meant for 
HiFi circular consensus reads but may work on the reads presented here). It would be extremely 
interesting to see how Peregrine performs on this Maize dataset; if not then a mention in the 
introduction or discussion concerning future developments. Since much has been done to compare 
many of these tools both in the literature and online across several species, the authors really need to 
place their findings squarely in the context of how this helps plant biologists beyond what is already 
known. 

Two things in the text that were not clear and require some mention. First, the authors refer to the 
reads as “raw reads” in the text. Are these raw polymerase reads or subreads? In the methods it 
seems the authors are using “raw reads” in the body to mean subreads. This needs to be clear in the 
body. Second the authors mention the B73v4 and human PacBio datasets, and in there are some 
mention of them in the methods as well as supplement. The relevant data is in Table S2, and this is 
referenced once at line 72. Currently the relevance of adding this dataset is not clear in the body of 
the text; a sentence to highlight the comparison in Table S2 could make this clearer. 

Minor points 
Line 96: should it read dramatic read in contiguity since the reference is to contig N50? 

Figure 1I: It is difficult to determine the number of SVs from the graph; would be nice to get a sense 
of number as well as size. 

Line133: Why only annotate two assemblies? Where the missing genes found in the gaps paralogs? 

At what level are the heuristics of the assemblers impacting the results? For instance, is there a 
coverage threshold that makes the 20x coverage assembly break at some points? Since the authors of 
Canu are on the manuscript some mention of assembler strengths and limitations would be 
informative. 

If other assemblers were/are not tested, a justification of why only these long read assemblers were 
used would be good for the reader. 

Reviewer #2 (Remarks to the Author): 

This paper gives a detailed report of an in-depth evaluation on how to use long-read sequencing data 
to produce a high-quality assembly for a complex plant genome. Focusing on maize, the authors 
report assembly results using differently sized data sets and make a completeness assessment of 
genes, transposable elements, and highly repetitive features. Overall, the applied strategy and 
methods are sound but on some points, for example when discussing genic regions, the reported 
results are incomplete and as such do not offer detailed information on potential consequences for 
downstream biological data analysis (which is of major importance for most plant biologists). Whereas 
most of the results are in line with previous reports on long-read sequencing, this paper does describe 
a useful resource reference to the community. 

Major 
1. The title and abstract are somewhat misleading, as all evaluated assemblies (Table 1) are not the 
result of solely assembling an increasing number of long reads sequences, they also contain 
information from optical maps and GoldenGate genetic markers to perform superscaffolding. As such, 
most of the reported results do not only deal with long read assembly, but include additional assembly 
strategies which also come at a cost. Related to terminology, if in the text the term superscaffolding is 
used, please also use this term in Table 1 (as from your text I conclude you report N50 stats on 
superscaffolds here). 
2. I am missing some key references which should be integrated to put the presented results in 
context. For example, the Belser et al paper (https://www.nature.com/articles/s41477-018-0289-4) 
generated and evaluated genome assembly using nanopore long reads for 3 different plant species. 
Likewise, Van Bel et al (https://www.ncbi.nlm.nih.gov/pubmed/30797187) gave a detailed overview 
of different data types and methods to assess completeness and actually performed a gene space 
comparison on different maize B73 assemblies. None of the presented results are discussed in light of 
these papers or other literature, which is extensive, on genome assembly strategies for complex 
plant/eukaryotic genomes. 
3. The evaluation of the genic regions (line 133-) reports some mapping statistics but lacks strong 

conclusions. Based on Figure 1G and Table S17, I learned that the RNA-Seq mapping rate for the 
NC358 tissue samples is higher for the B73 genome compared to the best NC358 assembly. This 
indicates some of the maize gene space present in B73 and NC358 is missing in the NC358 
assemblies, though this is not discussed. Are these repeat-rich genes like disease resistant genes or 
are these genes located near centro/telomeres? It would be good to also differentiate between 
multimapped and non-mapping RNA-Seq reads on the NC358 assemblies. It would be informative of 
the authors can report how many and which type of genes these are missing in the NC358 assemblies, 
based on the B73 gene models recovered by the NC358 RNA-Seq data, as this has major 
consequences for downstream gene-based anaslyses. Furthermore, taking the B73 gene models as a 
reference (though the Van Bel paper indicates >1000 of full-length genes present in V3 were removed 
in V4, which came as a surprise to me), how many of these genes are found back as full-
length/partial, models in the NC358 assemblies? This would help to put the “obtained a total of 28,275 
and 39,578 genes” (line 134) in perspective and could reveal if the best N50/contiguity assemblies 
also correspond with the assemblies best representing the maize gene space. 

Minor 
4. For the downsampling to lower coverages, every time only one sub-sample was selected, 
assembled and analyzed. Although this reviewer acknowledges that generating assemblies is resource 
intensive, it would be good to see, for one/few of the lower coverages, how big the standard 
deviations are on the reported statistics in Table 1, when considering multiple sub-samples (e.g. n=5). 
5. If the aim of this paper is serving as a guide for future plant sequencing initiatives, adding a brief 
analysis of the costs associated with the different data types could be relevant. 

Reviewer #3 (Remarks to the Author): 

This manuscript focuses on practical, technical concerns for the many biological researchers who are 
using next generation genomic sequencing technologies to advance their research interests. Getting 
accurate and comprehensive sequencing results from plant genomes is especially difficult. In this set 
of studies, the authors have compared different datasets for a strain of maize to help determine what 
characters are most important in providing good, useful assembled sequences according to different 
criteria. PacBio and Illumina sequencing platforms were used to provide hybrid assemblies of excellent 
quality. Further, they use BioNano chromosomal mapping to extend their results. By comparing the 
use of different PacBio read lengths, read-depths, and evaluating each for different applications, the 
authors provide very useful information for others seeking to make best use of their resources for 
accomplishing their individual goals. The presentation is well-written, clear, and effective in using 
graphs. The research is very detailed, even exhaustive and should be a great example for others 
wanting to take on the daunting task of sequencing a genome as difficult as many plant genomes are 
(including maize). The only question I have, and it is only an academic one is whether they had 
thoughts or experience with respect to Oxford Nanopore technology, and whether their would be a 
value of including this platform for sequencing in cases where it seems (according to the current 
study) that read length seems most important to the quality of the results. The authors can address 
this if they wish. I think the manuscript should be accepted for publication. I came across no errors 
that needed to be corrected at this time.