dataesr/scientific-paragraphs-categorization huggingface dataset:
To address this gap, we benchmarked LRS alignment tools with the datasets generated from the Joint Initiative for Metrology in Biology's Genome in a Bottle Initiative (GIAB) sample, NA12878, which was sequenced on Oxford Nanopore's MinION-platform by the Whole Genome Sequencing Consortium  (9)  and on the Pacific Biosciences SMRT-sequence data by the National Institute of Standards and Technology  (39). We compared computational performance (peak memory utilization, central processing unit (CPU) time, file size/storage requirements), genome coverage depth (x-times), and quantified the reads left unaligned in any given experiment. We have limited this manuscript to tools that are 1) widely adopted so that analysts may have something for comparison, 2) platform-agnostic so that LRS data of any sort may be used in pipeline, and 3) easy to install so as to be equitable in their utility to researchers and future clinical laboratories.
RESULTS All experiments were performed on the NA12878 sample, for which publicly available datasets exist for both nanopore and SMRT sequencing, and rigorously annotated variants are available in the Database of Genomic Variants (DGV)  (41). We present our results in four sections: 1) the tools that passed inclusion/exclusion criteria, 2) computational performance and benchmarking, 3) an analysis of aligned and unaligned reads, 4) an analysis of structural variation present in each alignment compared to a baseline.
Tools that passed the inlcusion/exclusion criteria Following a literature review of available alignment tools and search of long-read-tools. org (38) , we established a set of inclusion criteria (see Material and Methods), which accounted for both types of LRS data (a tool must be able to handle both nanopore and SMRT reads), as well as the state of the field in terms of software updates (must not have been superseded by another tool) and adoption of the tool, as measured by citations (must be cited by manuscripts written by outside parties). All surveyed tools and criteria are outlined in Table  1. Three alignment tools, GraphMap, minimap2, and NGMLR were included in this study  (40, 42, 43). Kart, Mashmap2, LAMSA, MECAT2, lordFAST, smsMAP, and Winnowmap were excluded for lack of adoption based on citations  (43) (44) (45) (46) (47) (48) (49).
Minimap2 was the least resource-demanding tool. Minimap2 successfully aligned nanopore data every time with unrestricted and restricted resources. Unrestricted runs used an average of 14. 2 gigabytes (Gb) and the jobs took ~540 CPU hours (~13 wall clock hours) to complete. Minimap2 also successfully aligned SMRT data every time with unrestricted or restricted resources. The runs used an average of 16. 9 Gb and the jobs took ~641 CPU hours (~16 wall clock hours) to complete, a little longer than for the nanopore dataset. Memory usage (see peak_rss columns in Supplemental Table  S1 ) and runtime were consistent across triplicate runs with unrestricted resources and did not change with restriction of resources when the tool was used with either dataset. The consistency of results, as well as the speed and relatively low computational demands of minimap2 make it a strong candidate for inclusion in clinical analysis pipelines.
All unrestricted resource jobs used ~50 Gb of memory. One restricted run (30 threads) used ~49 Gb of memory while one (20) used ~46 Gb. The restricted run limited to 10 threads failed due to time out at 14 wall clock days. Runtime across all runs did not directly relate to memory usage or resource restriction (Table  2 , Supplemental Table  S1 ). The resource-intensive requirements of GraphMap for whole human genome alignment combined with failures to run in both datasets likely precludes including it in the design of most analysis pipelines, however alignment files from successful runs were included in further analyses.
NGMLR completed all tasks, but performance was very resource-dependent. NGMLR successfully aligned nanopore data every time with unrestricted and restricted resources. The unrestricted runs used an average of 33. 8 Gb and the jobs took ~2,539 CPU hours (~63. 5 wall clock hours) to complete, intermediary between Minimap2 and GraphMap. NGMLR was much faster on SMRT data. It successfully aligned every time even with restricted resources. The runs used an average of 29. 7 Gb the jobs took ~658 CPU hours (~16. 5 wall clock hours, almost 4 times faster than on the nanopore data) to complete, on par with the time taken by Minimap2. Memory usage and runtime were consistent with the average across triplicate runs, but, unlike the other tools, NGMLR's performance was resource-dependent: runtime increased and memory usage decreased with resource restriction.
Whole genome alignment Aligned reads. In every experiment that completed successfully, alignments of the same genome with the same tool had the same genome coverage, whether resource-restricted or not (Supplemental Table  S1 , column O). For this reason, one file generated from the first unrestricted run was used for subsequent analysis (shown in Table  2 ). Genome coverages of ~30x and ~44x were reported for nanopore  (9)  and SMRT  (39, 55)  analyses respectively. We calculated the actual coverage after read alignment to the hg38 reference by each tool. GraphMap produced higher coverage genomes than expected on both nanopore (32. 8 x) and SMRT (45. 4x) datasets. Minimap2 produced higher coverage nanopore genomes (33. 4x), and expected (within 1x) coverage SMRT genomes (43. 6x). NGMLR produced higher coverage nanopore genomes (31.
1x) and lower coverage (39. 7x) SMRT genomes. For nanopore data, coverage was higher than expected, by 3. 7%, 9. 3% and 11. 3% for NGMLR, GraphMap, and Minimap2 respectively. For SMRT data, GraphMap also produced slightly higher than expected coverage (by 3. 2%), while NGMLR came in at 9. 8% lower than expected. Minimap2 showed a small difference (-0. 9%). These results are not sufficient evidence to pass judgment on the utility of any tool, but prompted us to examine where those discrepancies come from. We analyzed the unmapped reads to find the overlap in consensus-excluded reads. Unaligned reads. The readname assigned to each read in a fastq retained in the BAM allowed us to write an R script to directly compare the lists of reads that were not included in the alignment by each alignment tool (Fig.
a. Differences in variants among alignment files Deletions (Fig. 2, A, B ). The total number of identified deletions varied widely, from ~6,500 to ~14,300, across alignment files. For all three alignment tools, the nanopore data yielded the most sniffles-identified deletions. Minimap2 preserved the most deletions on both datasets. GraphMap-aligned genomes did not allow detection of any deletions larger than 20 kbp in nanopore data or 6 kbp in SMRT data. Insertions (Fig. 2, C, D ). The total number of identified insertions also varied (from ~6,000 to ~13,700). In contrast to deletions, the SMRT data yielded the most insertions for all three alignment tools. Minimap2 again allowed identification of the most insertions on both datasets. The three tools agreed for insertions in the 1,000-4,000 bp range.
Duplications and inversions: GraphMap preserved no breakpoints that sniffles recognized as duplications or inversions. Both minimap2 and NGMLR datasets produced similar numbers of inversions (within 30-50 variants; Fig. 2, E, F ) and inversions (within 4-7 variants; Fig. 2, G, H ). NGMLR allowed calling of the highest number of duplications and inversions on both datasets, outperforming minimap2 in all but three SV length bins for duplications. This was particularly evident in the 6-10 kbp size range where no duplications were called from the minimap2 datasets (5-10 kbp in nanopore data).
(DGV variants are shown in red in Fig. 2 ; details of counts are shown in Supplemental Table  S3. ) While these discrepancies may largely reflect the characteristics of LRS output since DGV variants are curated from several data sources, our analysis showed that the different alignment tools yielded different callsets from each other as well. Deletions (Fig. 2A, B ). The overestimation of deletions of all sizes called in the LRS datasets compared to DGV ranged from 4. 8 times more in the SMRT/GraphMap data to 10. 6 times more in the nanopore/Minimap2 data. For variants smaller than 10 kbp, the alignment files contained more SVs than are present in DGV. Indeed, for both datasets and all three alignment tools, 10 times more SVs between 51-1000 bp were present in VCFs than are present in DGV.
Compared to the variants in DGV, minimap2 and NGMLR resolve tens of variants less in each bin greater than 20,000 bp but less than 100,000 bp. Insertions (Fig. 2C, D ). Sniffles identified more insertions ≤6,000 bp in nanopore alignments and ≤4,000 bp in SMRT alignments than are present in DGV. For both datasets and all three alignment tools, more than 100 times more insertions sized 51-1000 bp are present in VCFs compared to DGV. In contrast, none of the tools were able to identify large insertions beyond 8 kbp or 4 kbp on the nanopore or SMRT data, respectively. DGV contains 934 insertions between 9 and 80 kbp, yet none are resolved by Sniffles on any alignment file from the nanopore or SMRT set. Duplications (Fig. 2E, F ). None of the tools performed well for identification of duplications.
DISCUSSION In this study we have highlighted the key differences between alignment files generated by three tools that have seen wide adoption for experiments involving the reference-guided alignment of large genomes. By using a well-characterized genome standard, NA12878, we were able to directly compare the performance of each tool on the sequencing datasets obtained on different platforms. We further analyzed the reads that were included or excluded from the alignment, and how these reads alignments revealed breakpoints that could be resolved as structural variants in genome architecture. We then compared those variants to a set of previously published variants discovered through multiple platforms. Reassuringly, each alignment tool was internally consistent: when an alignment tool was given the same fastq and the same reference genome, it produced the same result as judged by bamstats and sniffles variant callsets.
On data from both platforms, it allowed calling of the most insertions and deletions, but fell short on inversions and duplications. NGMLR was the most discerning aligner, in that it left the highest number of reads unmapped. It used more compute power than Minimap2, but handled the SMRT-based data in nearly the same amount of time, while taking considerably more time when handling nanopore-based data. While it was designed specifically to resolve structural variation  (40) , it calls a high number of very large variants (>100,000 bp) that have not been validated with other methodologies and curated in DGV. There is a great divergence in sniffles-called variants from alignment files generated by all tools from the variants present in DGV. This is a concerning expansion of seminal findings in a previous study  (16)  as none of the sniffles VCFs mirror the SVs present in the high-quality curated DGV database.
Unlike Sniffles, which provides a call of type of SV (deletion, insertion etc. ), NanoSV only identifies breakpoints in the alignment, without assigning those breakpoints an SV type. A robust comparison of SV callers on nanopore datasets highlighted the relative strengths of variant calling pipelines and may help users determine the best caller for their experiments  (64). NanoSV may be suitable for identifying breakpoints missed by Sniffles, but comes with the further caveat that it is resource-intensive and may not scale in a clinical setting without vast computational resources  (64). Answers to biological questions can also leverage tools that work with data upstream and downstream of whole genome alignments; indeed, alternate solutions to discover differences in genome structure have been proposed that leverage new tools.
Sequence data Nanopore sequence data were accessed on April 8, 2019, version rel5-guppy-0. o https://github. com/nanopore-wgs-consortium/NA12878/blob/master/Genome. md (66)  o ftp://ftp. ncbi. nlm. nih. gov/giab/ftp/data/NA12878/NA12878_PacBio_MtSinai/ (67).
Database of genome variants DGV data were accessed on January 30, 2020 from http://dgv. tcag. ca/dgv/docs/GRCh38_hg38_variants_2016-05-15. txt (68). The full set of variants was reduced to 11,042 variants confirmed to be present in NA12878. 14 of these variants were excluded as they were called on contigs present out of the main reference assembly contigs.
Whole genome alignment tool benchmarking and analysis There versions of alignment tools used in this project included: GraphMap/0. 5. 2, minimap2/2. 16, and ngmlr/0. 2. 7. o https://github. com/isovic/GraphMapLink (42)  o https://github. com/lh3/minimap2 (43)  o https://github. com/philres/ngmlr (40)  Computational metrics were printed from SLURM job records. Samtools 1. 9 was used for all alignment manipulations, alignment read depth coverage calculations, and to extract unmapped reads  (69). Samtools 1. 10 was used to generate bamstats files. R 3. 5. 2 version Eggshell Igloo with the tidyverse packages were used to compare the readnames across unmapped read files to assess the degree of overlap of unmapped reads across alignment files. Genome coverage was calculated with samtools 1. 9 assuming a 3 billion basepair genome.
to leave ~1. 6 million of the same nanopore reads unaligned, but differed in their overall totals of unaligned reads. NGMLR left the highest number of reads unaligned, ~3. 3 million, which explains why it produced the lowest coverage genome. It agreed with the other tools on approximately half of its discarded reads. Minimap2 and NGMLR agreed to leave a further ~0. 5 million reads unaligned, while NGMLR and GraphMap agreed on a separate ~0. 48 million unaligned reads and GraphMap and Minimap2 on ~0. 4 million further unaligned reads. All tools agreed to leave ~1. 2 million of the same SMRT reads unaligned, but differed in their overall totals of unaligned reads. Similarly, for the ~44x SMRT dataset, all three tools agreed to exclude the same ~1. 2 million reads. NGMLR left the highest number of reads unaligned, ~6 million, which explains why it produced the lowest coverage genome.
AVAILABILITY OF DATA AND MATERIALS All data and tools used for this study are publicly available at the URLs indicated above in Methods. The scripts we used to process those data are available in supplementary materials. Raw and post-processed VCFs, DGV source material, and other dataframes used to create the figures are available at: https://github. com/jlotempiojr/longread_alignment_benchmarking (73)  Interested parties can further reach out to the authors for intermediate BAMs if needed.
A late snowmelt in 2019 and an exceptionally early snowmelt in 2020 across the European Alps provided a natural experiment that allowed us to explore the extent to which Alpine breeding wheatears experience a trophic mismatch in years when the season is advanced. We used light-level geolocators to track wheatears, and collected detailed information on individual breeding activity and success as well as environmental conditions during the reproductive season. In addition, we used citizen science data and remote sensed images to quantify longer term trends in the timing of wheatear migration and of environmental conditions, thus putting the data from our study area in a wider context. We predict that the relatively short migration distance of Alpine breeding wheatears leads to early arrival at the breeding grounds, allowing reproduction in synchrony with environmental conditions, even in the exceptionally early year of 2020.
Migration schedule and arrival dates We retrieved 11 geolocators, of which 9 contained su cient data for further analyses (6 Migrate Technology, 3 SOI, of 5 males and 4 females). 14 tagged individuals were resighted (n = 35, return rate: 40%) of which 13 were recaptured. Two individuals lost the device. 9 adults of the control group were resighted (n = 16, return rate: 56%). The frequency of return rate between tagged and control groups was not signi cant (Fisher exact test, P = 0. 126). Six tags recorded light levels for the full annual cycle. One tag stopped recording after arrival at the main wintering site (24EA). Two tags stopped recording during spring migration (24DK, 24EP). In total, we derived arrival dates at the breeding site from six tracked individuals (Fig. 2b , see for details supplementary gures S1a-c, tables S1a-b).
2b , supplementary table  S3 ). Interannual variation in the timing of spring arrival/transition in the larger region of the study area (observations of wheatears in ornitho. it database) was high over the last ten years (R² < 0. 001), with an average arrival in 2019 (median doy = 119) and a rather late arrival in 2020 (median doy = 130) (Fig. 2b , supplementary gure S2, table  S1b ). Observation dates differed from the geolocator arrival dates in 2020, with earlier arrivals of tagged birds (median doy = 118, supplementary table  S1b ).
However, individuals migrating through the area can come from various wintering sites in Africa and can vary even more strongly in their destination that can range from central Canada to eastern Russia  15, 18. Population speci c migration schedules and potential trends may thus be hidden in the sighting dataset. In fact, wheatears from Greenland (O. o. leucorhoa) have advanced their migration within the last three decades  33  and previous studies on a northern lowland population of the wheatear have shown a continuous trend towards earlier arrival at breeding sites  34. Constant effort trapping data of wheatears stopping over on islands in the Mediterranean Sea has revealed a weak, but not signi cant, trend towards earlier passage during spring migration within the last 18 years  35. More importantly, the tracked individuals provided exact arrival dates for 2020, showing that the studied population arrived with the major migration ow of wheatears in the region (Fig.
Environmental conditions To investigate the longer-term trends in the annual date of snowmelt ("the snow free date"), we used the remotely sensed MODIS (MOD10A1. 006) Terra Snow Cover Daily Global 500 m dataset with recordings starting in 2000  49. Daily values for all pixels (n = 68) within the study area were extracted from Google Earth Engine 50 via the R package rgee  51. For each pixel and each year, we identi ed the snow free date as the rst date without snow that was followed by at least 60 days of snow free conditions, ignoring potential snow free periods within the winter. To derive higher-resolution snow free dates for the actual nest sites in the study area during 2019 and 2020, we used the remotely sensed Sentinel-2 MSI MultiSpectral Instrument Level-2A dataset with a spatial resolution of 10 m 52.
On the Google Earth Engine Server, we rst selected all images taken in 2019 and 2020 intersecting at least one of the nest sites. Next, we masked cloud pixels using the Sentinel-2 Cloud Probability layer and a probability threshold of 60. We then calculated the normalised differenced snow index (ndsi, Band 8 & 4) and the normalised vegetation index (ndvi, Band 3 & 11). For each nest site and each image, we extracted the mean ndsi and ndvi values of all pixels within a 10 m radius (n = 2-4) around the nest. For each nest site and for both years, we tted a smooth line (loess t with span = 0. 2 from the R package stats) to the ndvi and ndsi values over the day of the year, and extracted the day when the line fell below 0. 1 ndsi and exceeded 80% of the ndvi annual amplitude. In the eld, snow cover and grass height, as proxies for the spring green-up (i.
e. the seasonal development of new vegetation growth) were monitored throughout the two breeding seasons (see for details and sample sizes supplementary table  S6 ). Data were collected using a standardised protocol for territory mapping. In summary, we de ned four sectors above the treeline (total area: 161 ha), characterized by open grassland habitat, a variation in elevation (~ 700 m) and a high density of wheatear territories. We de ned a route (~ 6 km in each sector) along which we de ned habitat survey points separated by a distance of 200 m (6-8 points each sector, 30 in total). Territory mapping and habitat point surveys were conducted over ve visits between 27th May and 21st July 2019. Following the protocol introduced by  53  , positions of males were recorded with a GPS device during the period between sunrise and 12am and during suitable weather conditions.
All animal handling and protocols were carried out in accordance with relevant guidelines and regulations under licenses issued by Istituto Superiore per la Protezione e la Ricerca Ambientale (ISPRA, protocol no. 27303, no. 2463) and Città di Torino (license no. 56. 1433/2019, no. 197. 4765/2019). Light-intensity data were recorded at 5 minute intervals and analysed using a threshold method  55. Sunrise and sunset events (twilight events) were identi ed on log-transformed light data and a threshold of 1 log lux, using the R package TwGeos 56. Twilight events recorded at the known deployment site during periods after deployment and before recapture where used for calibration, i. e. to estimate the error distribution of twilight events and the individual reference sun elevation angle (position of the sun when twilight events where detected).
For recordings from Migrate Technology Ltd. tags, migratory movements were identi ed as sudden and directed changes of consecutive sunrise or sunset events (for details, see "invMovement" method description in  57  ). Stationary periods with a minimum of two consecutive twilight events were then identi ed as periods surrounded by migratory movements. For data recordings from Swiss Ornithological Institute tags, we used the additional air pressure measurements to identify movement periods with a change in pressure threshold of 4, using the method implemented and described in the R package PAMLr 58. For the nal track estimation, we used a Bayesian approach from the R Package SGAT  59  allowing the incorporation of twilight events, their error distribution (gamma density distribution), the information on periods of movement and residency, a spatial probability mask and expected ight speed distribution (gamma distribution with shape = 2.
it) and lay date. To estimate and compare the breeding success of the population between the two consecutive years 2019 and 2020, we considered three different measures: a) the hatching success rate (eggs that hatched), b) the edging success rate (hatchlings that edged), and c) the nest success rate (successful nests/all nest attempts) (binomial response). Measures a and b allowed us to separate the breeding success in two states of a nest, i. e. nests during incubation and nests during chick rearing, by considering the success rates of individual eggs and hatchlings estimated as probabilities of reaching the next state ("period" hatching and edging). We applied a general linear mixed model to quantify the success rate of hatching and edging (a, b) with the xed effects "year" (2019, 2020) and "period" (hatching, edging), adding "nest id" as a random effect (using the function glmer from package lme4 61 ).
To quantify the nest success rate (c), we applied a generalized linear model with the xed effect "year" (using the function glm from package stats). Model assumptions were tested by drawing residual plots and checking for overdispersion (function dispersion_glmer from package blmeco 62 ). Conclusions from test statistics were based on the credibility intervals from the model predictions calculated using the function sim from package arm (Gelman, 2007), and by drawing 2000 random samples from the posterior distribution of the model parameters to obtain their 95% credible intervals from these simulations. An effect was considered signi cant if the credible interval of the model estimate did not include zero 63. b) The day of the year, from 2000 to 2020, after which the study area was generally snow free in spring for at least 60 days.