dataesr/scientific-paragraphs-categorization huggingface dataset:
The new addition is the use of autofill that is called via a GitHub Actions 'bot'. This generates and uploads to the pull request in a comment a partially completed library metadata table that can be filled in, reviewed for accuracy and appended to the corresponding AncientMetagenomeDir library table as a part of the original sample pull request. $ AMDirT convert --curl <filtered_table>. tsv ancientmetagenome-hostassociated -o. / In the command above, options, input files and output files are defined with standard command line flags and positional arguments. The resulting file AncientMetagenomeDir_curl_download_script. sh file from the command above will be present in the directory specified in the command. The user can then simply run the bash script to download all libraries of the samples present in the input table.
$ AMDirT viewer AMDirT [ INFO]: [AMDirT] To close app, press on your keyboard: ctrl+c You can now view your Streamlit app in your browser. Local URL: http://localhost:8501 Network URL: http://172. 16. 9. 75:8501 Once completed, the user can close the tab and cancel the command in their terminal (e. g. , with ctrl + c). Alternatively, for individuals who wish to use the viewer but do not wish to deal with software installation and/or are not comfortable with command line interfaces, a hosted online version of the AMDirT viewer is available at https:// spaam-community. org/AMDirT accessible with a web browser.
In order to distinguish modern and aDNA, the user will likely want to examine the DNA for evidence of chemical degradation, which can be used to authenticate aDNA. To do this, the user will already have selected their preferred dedicated aDNA analysis workflow, such as nf-core/eager, that integrates DNA damage analysis into its pipeline. nf-core/eager requires an input 'sample sheet' that describes whether a particular sample has been sequenced over multiple lanes or libraries, and whether aDNA damage has been already removed during laboratory processing. We will show how AMDirT can assist with the creation of this sample sheet with the desired dataset. This example assumes that the user has already installed AMDirT and nf-core/eager, and has downloaded a Homo sapiens sapiens reference genome for host DNA removal.
To load the GUI based viewer and downloading tool, a user enters the following command into their terminal: As shown in Figure  3a , the viewer is loaded into the user's web browser. Using the left sidebar, the user can navigate dropdown menus to select the release of AncientMetagenomeDir to use (for reproducibility purposes), and the desired AncientMetagenomeDir table to explore (i. e. , ancient environmental metagenomes, ancient host-associated metagenomes, or ancient microbial single-genomes). The user can also specify the number of rows for the table to display and which tool to use for the download scripts generated later in the example.
Here, we recommend that users always download the corresponding AncientMetagenomeDir libraries table, as well as the BibTeX citation file. In this example, the user would also download the 'curl download script' and the 'nf-core/eager' input TSV using the corresponding buttons. Hovering over the download script button also provides an estimate for the user of how much hard-drive space the download of all selected data will use. To close the GUI viewer, the user can close the tab in their browser, and then in their terminal press ctrl + c on their keyboard to stop the Streamlit server. Alternatively, a user could also follow the same process but via the command line interface. The user first can run the new download (download) command to retrieve a particular samples or libraries table. $ AMDirT download --table ancientmetagenome-hostassociated --table_type samples --release v23.
04 The user could then use typical shell commands, or load the table into languages such as R or Python and Pandas for filtering the table to the data they would be interested in. To replicate the same conditions as in Figure  3b  above, a user could run the following command with the common CLI tool awk $ awk '{if ($2 >= 2020 && match($7, 'Spain')) print $0}' ancientmetagenome-hostassociated_samples. tsv > ancientmetagenome-hostassociated_samples_gt2020_ spain. tsv Finally, the user can pass the resulting filtered table to the AMDirT convert command, to perform the same download script generation, pipeline-input file conversion, and citation file download as with the GUI interface $ AMDirT convert --librarymetadata --curl --eager --bibliography ancientmetagenome-hostassociated_samples_gt2020_spain.
Solutions: We suggest two solutions: first, ancient metagenomic researchers should ensure that library and sample names are descriptive (i. e. , in a structured system in which a certain level of information can be inferred just by the name) and that sequencing metadata uploads match those reported in the publication; second, where this is not possible (e. g. , if an upload is carried out by a third party), then researchers should at a minimum include a key in their supplementary files. This could be in the form of a table that includes all ID codes for each sample, library, and sequencing batch, including internal laboratory codes, other-analysis codes, and external sequencing archive accession codes.
Neither the primary nor secondary publication indicated which of the experimental protocols from the original methods study was actually used. Solutions: Ancient metagenomic researchers should make an effort to more clearly describe their protocols and explicitly indicate which library protocol is linked to each sequencing file. At a minimum, the information provided should include critical metadata for downstream analysis, such as library treatment protocols that affect DNA damage. This can be accomplished by providing expanded, plain-language descriptions of laboratory methods in article supplementary information files (rather than simply citing and re-citing) and providing a supplementary table that lists each library name and their corresponding treatments. For improved compliance with FAIR principles, researchers are encouraged to further provide or cite a protocol written up in a citable protocol format and/or on open platforms.
For example, platforms such as protocols. io 20 allow critical protocol information to be communicated consistently and unambiguously, by providing a persistent identifier (DOI) that points to a specific version of a given protocol. Uploading of mapped BAM files or merged FASTQ files rather than raw metagenomic data. Problem: Occasionally, we found that in some cases ancient metagenomic researchers uploaded mapped BAM files or merged FASTQ files rather than 'raw' FASTQ files (i. e. , against ENA/SRA specifications). Both formats present obstacles for downstream analysis. For example, mapped BAM files include only reads mapped to a particular reference genome and thus do not represent a full metagenomic dataset. For BAM files containing reads mapped to the human genome, microbial DNA will be absent, including ancient pathogen DNA that could be highly relevant for an archaeological study.
Base quality scores are often altered during the read merging process, which can interfere with tools reliant on such scores, and most de novo sequence assemblers either require or perform better on unmerged reads. Furthermore, many metagenomic tools, including taxonomic classifiers, do not accept merged paired-end FASTQ files or BAM files as an input format. Example: An ancient metagenomic researcher generates both damage and damage-removed libraries, but merges them together in BAM format and uploads to a sequencing archive. However another researcher wishes to analyse only the data deriving from the damage-removed libraries. Solution: Ideally researchers should upload FASTQ files that match the 'raw' output from sequencing, i. e. , demultiplexed datasets separated per library, applying only the preprocessing steps recommended by the sequence repository (e.
g. , for the ENA/SRA, adapter removal but not read merging). If this is not possible, authors should, at a minimum, describe exactly how the merging steps were performed so that other researchers can manually separate merged sequences (e. g. , using sequencing read headers) when required for downstream analysis. Unique sample accessions applied to multiple libraries of the same sample. Problem: Another common error was found to occur when researchers mistakenly uploaded each library or sequencing dataset with a unique sample accession code. While often not a critical error, because in these cases the correct sample could usually be inferred from the file name, this nevertheless makes automated data processing more difficult and requires manual intervention. To reuse such data, a researcher must manually reassociate the library datasets with the correct sample based on the file names, rather than relying on the sequence archive sample accession ID, as expected by metadata schemas of the ENA/SRA data repositories.
Note on AMDirT generated pipeline scripts It is important to note that the aim of the pipeline TSV sheets generated by AMDirT is to provide a template for data input to the pipelines. Due to the high heterogeneity in the way that sequence (meta) data are uploaded, some information in AncientMetagenomeDir may be missing or erroneous, despite the best efforts of the SPAAM community experts to standardise the information, resolve ambiguities, and correct errors. However, we hope that by providing this functionality, it reduces the time it takes to create such input sheets from scratch.
This is despite the fact that there is already standard reporting guidance. 21  However, we also call on ancient metagenomics researchers to report both uncalibrated and calibrated dates and associated metadata (radiocarbon lab code, calibration curve, software, etc. ), and not to rely solely on secondary citations to facilitate adding such data to repositories such as AncientMetagenomeDir, as well as refinement of chronological modelling in the future. In the same vein, we also aim to synchronise AncientMetagenomeDir with upstream standardised sequencing data metadata schemas and repositories such as MIxS checklists  22  via another SPAAM-established project, MInAS (https:// mixs-minas. org/), to further ensure common standards across both modern and ancient sequencing data. Given that the functionalities of AMDirT provide simple data exploration without requiring advanced computational knowledge, but also offers semi-prepared templates for aDNA and metagenomics, the dataset and tooling are ideal for further generation of community resources.
Conclusions By extending AncientMetagenomeDir to include library-level metadata, not only do we make ancient metagenomics data more findable, but also we make them more accessible by providing improved transparency of the diverse library and sequencing treatments performed in the field of palaeogenomics. Furthermore, AMDirT has been designed to improve the experience of researchers in the downloading and processing of previously published ancient metagenomics data. By providing both a graphical user interface and a command-line interface to filter and generate relevant download scripts and input sample sheets for aDNA analyses, we provide more flexibility and choice for the wide range of computational backgrounds that ancient metagenomic researchers can have. Finally, we hope that by informing researchers about inconsistencies in past data and metadata uploads and providing templates of standardised metadata for future publications, we will contribute to improving aDNA data reporting and FAIR data sharing.
9. Eaton K NCBImeta: efficient and comprehensive metadata retrieval from NCBI databases. J. Open Source Softw. I appreciate the extensive AMDirT documentation, including tutorials and videos. However, the manuscript lacks in the description of the tool's workflow and usability. The "Use cases" section of the manuscript makes the tool much clearer. At the same time, it gives the impression that AMDirT is limited to the GUI exploration and template generation guidance. Specifically, in my opinion, the manuscript poorly describes and explores the "Validate" and "Autofill" commands. For example, the Validate command does "a variety of checks. " It would help if you made it more transparent. Do you have a document detailing the checks and standards? If so, it should be placed here; if not, I would like to see it in the manual.
For the Autofill command, I missed a more detailed description of the "improvements" done using R scripts after pulling the metadata from ENA. Specifically, since those commands could be used outside the realm of ancient metagenomic samples. I believe a figure containing the complete schematic workflow from AMDirT would help. Regarding the software operation, I missed a storage requirement specification. The paper mentions that the installation requires Python 3. 9, but nothing is specified in the GitHub. This paper describes the 'AMDirT' open source software for interaction with ancient metagenome sequence sample metadata tables. The tables have been produced through a massive communal curation effort by the international SPAAM (The Standards, Precautions, and Advances in Ancient Metagenomics community) group to systematically improve on what is available from the public repositories of archived short read sequence data.
AMDirT allows viewing and searching of the tables using the Python streamlit library and facilitates download of data and integration with open source nextflow analysis pipelines. AMDirT also helps with initial creation and validation of new metadata tables. I successfully installed AMDirT version 1. 4. 6 using conda on my Intel MacBook following instructions in the manuscript. I was able to replicate the commands described. There is also a public server https://www. spaam-community. org/AMDirT/ for the AMDirT viewer. The documentation for the software is very good and includes video tutorials. The manuscript is well-written and clearly outlines the functions of the software. The development of AMDirT represents a significant effort, not least because of the community-wide consultation. The need for community-based metadata addresses a well-known problem with the SRA/ENA/DDBJ databases and the discussion provides some nice examples of the type of issues faced when downloading.
○ There should be a list of computational environments that the software has been tested on (e. g do M1-3 macs work?) ○ I did not see mention in the text that there is actually a public facing server https://www. spaam-community. org/AMDirT/ ○ "Newly added library information columns include the library name (how data are typically reported in original publications), the aDNA library generation method (e. g. , doublestranded or single-stranded libraries), the library indexing polymerase (e. g. , proof-reading or non-proofreading), and the library pretreatment method (e. g. , non-Uracil-DNA Glycosylase (UDG), full-UDG, or half-UDG treatments). The latter three fields represent information about the sequencing library construction that influence the presence of aDNA damage, a factor that is critical for the processing of aDNA NGS data.
Further questions Finally, I have some three questions about the project that it would be great to get comments on. How sustainable is the community effort to maintain these databases moving forward into the future with the probability of the number of samples increasing each year? 1. Have you tried sending the improved metadata back to SRA/ENA? 2. How difficult would it be to adapt the software for a different community of researchers that wanted to improve annotation but use fields that would be different from the ancient metagenomes community? 3.
Acknowledgements The authors thank the  SPAAM community  for the ongoing maintenance, testing, and general support of both the  AMDirT and AncientMetagenomeDir  projects. We also thank all the supervisors and managers of all the authors for allowing us to contribute to the AncientMetagenomeDir and AMDirT projects. We are also grateful for all authors of publications who we contacted with queries about their metadata, particularly those who subsequently took the time to update their original sequencing archive uploads to correct mistakes or improve metadata of their given study.
Nicolás Rascovan ). I. J. was supported by the  Science Foundation Ireland Centre for Research   Training in Genomics Data Science  (Grant  18/CRT/6214 ). J. G. S. was supported by  European Research Council  under the  European Union 's  Horizon 2020 research and innovation program Starting  Grant agreement number  851733 IsoCAN  (to  Dr. Rosa Irene Fregel Lorenzo ). K. T. Ö. was supported by 'Constructing the Limes: Employing citizen science to understand borders and border systems from the Roman period until today' (C-Limes), funded by the  Dutch Research Council (NWO)  as part of the  Dutch Research Agenda  (NWA,  2021-2026 , project number:  NWA. 1292. 19. 364 ). K. E. B. was supported by  Leverhulme Trust   Research Grant  'What's in a house?  Exploring the kinship structure of the world's first houses ' (Project Ref.
84009 ). K. G. D. was supported by a  Science Foundation Ireland -Irish Research Council  ( SFI-IRC) Pathways grant  (Grant  21/PATH-S/9515 ). M. L. was supported the  Institut Pasteur 's  INCEPTION-program (Investissement d'Avenir  grant  ANR-16-CONV-0005 ). N. O. was supported by the  Knut and Alice Wallenberg Foundation. O. S. S. was supported by the  NSF   Graduate Research Fellowship   Program  (Grant no. DGE 2137420 ). O. L. was supported by the  European Union 's  Horizon 2020 research and innovation programme  under the  Marie Sklodowska-Curie  grant agreement no. 895107. R. E. was supported by the  Carlsberg Fellowship for Associate Professors   CF20-0460  (awarded to Assoc. Prof. Antton Alberdi ). S. L. R. was supported by the  NIH   Genetic Mechanisms of Evolution Training Grant  (Grant No.
Data availability Source data The source data for the sample-level metadata used by AMDirT is from the AncientMetagenomeDir project originally published in Ref. 3 under a CC-BY 4. 0 license. The existing sample-level and new library-level data is stored on GitHub: https://github. com/SPAAM-community/AncientMetagenomeDir Each release is archived on Zenodo: https://doi. org/10. 5281/zenodo. 3980833.
Underlying data New sequencing library-level metadata is also stored in the AncientMetagenomeDir project from version v22. 09 (Pyu Ancient Cities) onwards. 12  The version of the dataset used for the demonstration of AMDirT, statistics, and figures in this updated manuscript is v24. 03 (Monticello). 24  Zenodo: SPAAM-community/AncientMetagenomeDir: Monticello (v24. 03). https://doi. org/10. 5281/zenodo. 10942606. Data are available under the terms of the Creative Commons Attribution 4. 0 International license (CC-BY 4. 0).
Software availability • Software available from: -PyPi: https://pypi. org/project/AMDirT/ -Bioconda: https://bioconda. github. io/recipes/amdirt/README. html -Hosted web version of AMDirT viewer: https://spaam-community. org/AMDirT • Source code available from: https://github. com/SPAAM-community/AMDirT • Archived source code at time of publication revision (AMDirT v1. 6): https://doi. org/10. 5281/zenodo. 10941007 • License: GNU General Public License v3. 0.
The general help of AMDirT (v1. 6) is available from the CLI: $ AMDirT --help Usage: AMDirT [OPTIONS] COMMAND [ARGS]… AMDirT: Performs validity check of AncientMetagenomeDir datasets Authors: AMDirT development team and the SPAAM community Homepage & Documentation: https://github. com/SPAAM-community/AMDirT Options: --version Show the version and exit. --verbose Verbose mode --help Show this message and exit. Commands: autofill Autofills library and/or sample table(s) using ENA API and… convert Converts filtered samples and libraries tables to eager,… download Download a table from the AMDirT repository merge Merges new dataset with existing table validate Run validity check of AncientMetagenomeDir datasets… viewer Launch interactive filtering tool Most tools follow a standard CLI based interface.
Morphologically, interacting bedforms in the experiments and simulations are similar. A distinctive feature of upstreamdominant coalescence seen in both datasets is the formation of a plateau towards the end of the interaction. Figures  4a  and b  compare the morphology of an experimental and a numerical example, respectively. The plateau is seen in both cases and was ubiquitous in all upstream-dominant coalescence events. The same intermediate morphology was observed experimentally by  Groh et al. (2009)  and in active dune fields  (Gao et al. , 2015). This plateau could have implications for boundaries in the sedimentary record of previous dune-forming environments. Figure  4c  shows the location of the selected events in the regime diagram (as determined from simulations), which has been transformed to be in terms of dune heights as opposed to areas, since height is easier to measure experimentally.
Acknowledgements. This research was funded by  Royal Society Challenge  Grant  CH160065  and  Isaac Newton Trust Early Career  Grant  RG 74916. NMV was supported by a  Dorothy Hodgkin Fellowship   DH120121  and a  Royal Society University Research  Fellowship No. URF/R1/191332. The authors thank  Karol Bacik  for useful discussions. We also thank  Suleyman Naqshband  and three anonymous reviewers 280 for their helpful and insightful reviews and comments on an earlier manuscript. CN and OR acknowledge support from the  UnivEarthS LabEx program of Sorbonne Paris Cité  ( ANR-10-LABX-0023  and  ANR-11-IDEX-0005-002 ) and the  French National Research Agency  ( ANR-17-CE01-0014/SONO ).
In this paper, the LES is implemented in the commercial CFD software ANSYS Fluent V. 15. 0  [16]. With regard to the SGS model, the Werner-Wengle wall function is utilised for near-wall shear stress approximations together with a dynamic "Smagorinsky Lilly model". The details of the numerical techniques will be discussed in Section 4. The spectral synthesizer is used for the inflow generation and a reduced turbulence integral length scale at the inlet, which will be also utilised for the generation of the inflow velocity fluctuations inside the domain. The implementation of these reduced length scales is significantly important, in particular the reduced integral length scales, can substantially decrease the size of the computational domain. Furthermore, User-Defined Functions (UDFs) and MATLAB (R2023a) codes are developed for the pre-processing and post-processing of the results.
Location Specification and Measurements Figure  1  shows the platform of the region and the location of the case study investigated. The building and its neighbour are distinguished using red and grey shades. Meteorological data of the location have been gathered from the National Climatic Data Canter (NCDC, from National Oceanic and Atmospheric Administration in the United States), which maintains the Climate Data Online portal (http://www. ncdc. noaa. gov/cdo-web/, accessed on 12 May 2014). the key factors that influence the performance of wind turbines, and this is one of the sources of noise generation, as we have shown in another research  [15]. To the authors' knowledge, despite its importance, such effects have not been investigated before. Furthermore, we discuss the factors that determine whether the spectral synthesizer inflow generation method using the reduced integral length scale can provide reliable wind flow simulation around the buildings and justify the results, comparing them with local wind measurements.