TITLE: SRA and Genbank BioSample-Linked Submission with Mercury_Prep and Mercury_Batch
AUTHORS: Francis J Ambrosio

[DESCRIPTION]
Submitting sequencing data to public data repositories is a meaningful yet tedious procedure. Linking submissions between SRA and Genbank will enhance the value of both submissions the the public health community. The Mercury protocols offered by Theiagen Genomics allows users to efficiently and accurately produce all required inputs for SRA and Genbank submissions (the Mercury workflows also allow for GISAID submission, but that will not be covered in this protocol). This protocol provides a detailed procedure for submitting BioSample-linked sequencing data to SRA and Genbank.

[STEPS]
SECTION: Data Preparation
1. The Titan Genomic Characterization workflow must be run prior to submitting sequences to SRA and Genbank in order to prepare the data for submission. Please use the Titan workflow that is compatible with your sequencing data.
         
 





SECTION: Data Preparation
1.1. Please check that all samples have been analyzed using the appropriate Titan workflow prior to running the Mercury workflows by navigating to the 'Data' tab, selecting the data table of choice, and select the 'assembly_fasta' and 'assembly_method' columns.
  

SECTION: Metadata Formatting
2. The Terra Metadata Formatter is an excel spreadsheet too that will help you by collecting all required metadata for each of the sequencing data repositories and formatting this data into a Terra-uploadable data table.
  

SECTION: Metadata Formatting
2.1. Download and open the Terra Metadata Formatter:
https://storage.googleapis.com/theiagen-public-files/terra/mercury-files/Terra_Metadata_Formatter_2022_01_06.xlsx
SECTION: Metadata Formatting
2.2. Enter the sample metadata into the 'User Input' tab of the Terra Metadata Formatter. The required fields are highlighted in blue. The optional fields are highlighted in grey. We recommend that you attempt to include as much data about your samples as is available at the time of submission, with particular emphasis on the fields of 'Purpose of Sampling' and 'Purpose of Sequencing', which will be used to correct for statistical biases in the data due to diversity of the sampling methodologies.

Note that some of the fields have dropdown menus. These have been implemented for fields that have a controlled vocabulary in order to reduce typo-based rejections from the various databases.
 


The General Metadata section consists of two required fields:
Root Entity: This input will define the name of the Terra Data Table when this metadata is uploaded in subsequent steps.
Submission ID Prefix: This input will be the prefix to the submission ID in the final NCBI submission files. Typical inputs are formatted as the state abbreviation and laboratory abbreviation separated by a hyphen.


The Laboratory Data section consists of eight required fields and one optional field:
GISAID Submitter ID (required): the GISAID Submission ID in the final GISAID submission files (if you have already submitted these samples to GISAID then list the GISAID Submission ID that was used)
Authors (required): the list of authors included in the final SRA, Genbank and GISAID submission files
BioProject (required): the BioProject accession number used in the SRA and Genbank submissions
State: the state of the Originating Laboratory
Country (required): the country of the Originating Laboratory
Continent (required): the continent of the Originating Laboratory
Submitting Laboratory (required): the name of the Submitting Laboratory
Submitting Laboratory Address (required): the address of the Submitting Laboratory
Submitter Email (optional): The email associated with the NCBI account that will be used to submit to SRA and Genbank

The Sequencing Run section consists of five required fields and two optional fields:
Platform (required): the sequencing Platform used to generate this sequencing data
Instrument Model (required): the sequencing Instrument Model used to generate this sequencing data
Library Strategy (required): the Library Strategy used to generate the sequencing libraries (if using Artic V3 or similar amplicon-based protocol then "AMPLICON" is the most accurate entry for this field.)
Library Source (required): the material used as the Library Source in the generation of the sequencing libraries (if extracting viral RNA as starting material then "VIRAL RNA" is the most accurate entry for this field.)
Library Selection (required): the tool used to select libraries to be sequenced
Primer Scheme (optional): the Primer Scheme in the amplicon generation step of the library preparation
Amplicon Size (optional): the average Amplicon Size of the Primer Scheme

The Sample Metadata section consists of nine required fields and nine optional fields:
Samples (required): the unique ID of the Samples
Submission ID Suffix (required): the second component of the Submission ID (this field can be the same as Samples)
Library ID Suffix (required): this input is used to keep track of samples that have been sequenced more than once, or on multiple platforms (for the first or only sequencing submission for these samples it is recommended to use "01" for this field)
Collection Date (required): the date the samples were originally collected
Originating Lab(required): the laboratory where the samples were originally collected
Originating Lab Address (required): the address of the laboratory where the samples were originally collected
Organism (required): the target organism of the sequencing run (if sequencing SARS-CoV-2 the "SARS-CoV-2" is the most accurate entry for this field
Isolation Source (required): source of the sample (if sequencing samples that were collected as part of a diagnostic assay or or surveillance program from humans then "Clinical" would be the most accurate entry for this field
Host Disease (required): disease caused by the target Organism (if sequencing SARS-CoV-2 the "COVID-19" would be the most accurate entry for this field) 
Run ID (optional): the Run ID of the samples
Patient Gender (optional): the gender of the individual from whom the sample was collected
Patient Age (optional): the age of the individual from whom the sample was collected
County (optional): the county from which the sample was collected
BioSample Accession (optional): if the sample has already been registered with NCBI then include the BioSample here
Specimen Processing (optional): sample processing steps such as transport media and extraction method can be included here
Purpose of Sampling (optional): this input can be clinical diagnostics if the sample was taken as a human specimen for SARS-CoV-2 testing
Purpose of Sequencing (optional): this input can be used to tag samples as Baseline Surveillance or Targeted Sampling (for detailed guidance on what entry is most accurate for your samples please see the APHL guidance document here:https://www.aphl.org/programs/preparedness/Crisis-Management/Documents/Technical-Assistance-for-Categorizing-Baseline-Surveillance-Update-Oct2021.pdf)
        For Baseline Surveillance:
         1. Sampled randomly for genomic surveillance
         2. Those not identified in a targeted sampling effort (targeted efforts defined below)
         3. Sampled across targeted sequencing efforts to be representative of the community
        For Targeted Sequencing:
         1. Sampled based on cluster/outbreak investigations
         2. Longitudinally or repeatedly sampled from the same individual
         3. Sampled based on pre-screening for a particular variant (e.g., S-gene target failure)
         4. Sampled for the purpose of vaccine escape studies
         5. Sampled based on travel history
         6. Sampled based on disease severity (i.e., targeted sequencing of cases resulting in hospitalization
             or death)
Sequencing Protocol Name: if using a named sequencing protocol enter the name in this field
SECTION: Upload Metadata
3. Upload the Terra Data Table
SECTION: Upload Metadata
3.1. Once the sample metadata has been entered into the User Input tab of the Metadata Formatter click the 'Terra Data Table' tab:
  

SECTION: Upload Metadata
3.2. Select the whole sheet by hitting control+'a' on your keyboard.
Copy the whole sheet by hitting control+'c' on your keyboard.

  
SECTION: Upload Metadata
3.3. Log in and navigate to the Data tab in your workspace on Terra.bio:

  
SECTION: Upload Metadata
3.4. Select the plus button in the blue circle to add a Terra Data Table:

  
SECTION: Upload Metadata
3.5. Select the Text Import tab:
  
SECTION: Upload Metadata
3.6. Paste your metadata into the text input field:
  
SECTION: Upload Metadata
3.7. Click UPLOAD
  
SECTION: Mercury
4. Mercury Prep
SECTION: Mercury
4.1. Select Mercury_Prep_SE or Mercury_Prep_PE from the Workflows tab in your Terra workspace:

  


SECTION: Mercury
4.2. Choose the appropriate Version and Root Entity, then click Select Data:
  

SECTION: Mercury
4.3. Select the samples that you would like to prepare for submission:
  

SECTION: Mercury
4.4. Enter the input attributes:  
  
  
  
  
  
  
  

SECTION: Mercury
4.5. Select the default outputs:
  

SECTION: Mercury
4.6. Once the inputs and outputs have been defined, Save the workflow parameters:
  

SECTION: Mercury
4.7. Click RUN ANALYSIS
  
Confirm and launch the analysis by clicking LAUNCH:
  

SECTION: Mercury
5. Mercury Batch
SECTION: Mercury
5.1. Once Mercury Prep has successfully completed navigate to the Mercury Batch workflow:
  

SECTION: Mercury
5.2. Select the appropriate version of the workflow:
  

SECTION: Mercury
5.3. Select the SET LEVEL root entity type:
  

SECTION: Mercury
5.4. Click SELECT DATA:
  

SECTION: Mercury
5.5. Select the dataset of sample that you would like to batch for submission (Note: the dataset root entity is the plural form of the original root entity):

  

SECTION: Mercury
5.6. Enter the INPUTS. The inputs for Mercury Batch will be entered at the Array Level. This means the notation will be formatted as this.data_sets.{attribute}:

  
And note the set level attribute (middle of the two decimal points) is the pleural form of the original root entity.
  
  
  
  

SECTION: Mercury
5.7. Enter the public GCP bucket to stage your data for the final submission to NCBI SRA:

  

SECTION: Mercury
5.8. Select the default OUTPUTS:
  

SECTION: Mercury
5.9. Once the inputs and outputs have been defined, Save the workflow parameters:
  

SECTION: Mercury
5.10. Click RUN ANALYSIS
  
Confirm and launch the analysis by clicking LAUNCH:
 
 
SECTION: Mercury
5.11. Retrieve your submission files by navigating to the Terra Data Table containing the Mercury Batch outputs:
  
Click on the file names in blue:
  

Download the files:
  
These are the four files that will be required for SRA and Genbank submission:
  

SECTION: SRA Submission
6. Submit your data to SRA (and simultaneously generate BioSample accession numbers for your samples)
SECTION: SRA Submission
6.1. Navigate and Log in to the SRA Submission Portal:

  


SECTION: SRA Submission
6.2. Select New Submission:
  

SECTION: SRA Submission
6.3. Enter your submitter information, select your submission group, and enter the information for your organization:

  

SECTION: SRA Submission
6.4. Enter your BioProject number:
  

SECTION: SRA Submission
6.5. Select 'No' if you do not already have BioSample accession numbers for your samples in order to generate them upon SRA submission:

  

SECTION: SRA Submission
6.6. Select your Release Date (we recommend releasing your data immediately following processing:
  

SECTION: SRA Submission
6.7. Select the appropriate submission package (if you are submitting SARS-CoV-2 sequences extracted from a human specimen please select the SARS-CoV-2 clinical or host-associated package):
  

SECTION: SRA Submission
6.8. Choose the 'Upload a file...' option and upload the BioSample attributes file downloaded in previous steps:

 
 
SECTION: SRA Submission
6.9. Choose the 'Upload a file...' option and upload the SRA Metadata file downloaded in previous steps:

  
There may be a warning after the sra_metadata file is uploaded regarding the taxonomical identifier. If you are uploading SARS-CoV-2 data these warnings can be ignored:
  

SECTION: SRA Submission
6.10. Select the 'AWS or GCP bucket' option and enter the name of the public data bucket where your reads have been placed in the staging phase of the data submission procedure:
  

SECTION: SRA Submission
6.11. Review and Submit to complete your SRA submission! You will be able to download your BioSample accession numbers from the SRA submission portal as soon as they become available.
SECTION: SRA Submission
7. Retrieve the BioSample accession numbers '.tsv' file from the SRA portal
SECTION: SRA Submission
7.1. Navigate to the SRA Submission Portal (you should already be logged in)
Locate the Status column of the submissions table:
  

SECTION: SRA Submission
7.2. Click 'Download attributes file with BioSample accessions' for the SRA submission executed earlier in this protocol:
  

SECTION: Genbank Submission
8. Add BioSample accession numbers to Genbank_meta_upload file
SECTION: Genbank Submission
8.1. Open the attributes file downloaded from SRA containing the BioSample accession numbers
  

SECTION: Genbank Submission
8.2. Open the Genbank_meta_sra file downloaded from Terra (the output from Mercury Batch)
  

SECTION: Genbank Submission
8.3. Use XLOOKUP to algorithmically add the BioSample accession numbers to the Genbank_meta_sra file.
  
  
  

SECTION: Genbank Submission
8.4. Save the Genbank_meta_upload file (now including the BioSample accession numbers)
SECTION: Genbank Submission
9. Genbank submission
SECTION: Genbank Submission
9.1. Navigate to the Genbank submission portal
  

SECTION: Genbank Submission
9.2. Select SARS-CoV-2

  

SECTION: Genbank Submission
9.3. Click the submit button under the Genbank heading:
  

SECTION: Genbank Submission
9.4. Select 'New submission':
  

SECTION: Genbank Submission
9.5. Select 'SARS-CoV-2, Influenza, Norovirus, or Dengue virus' and 'SARS-CoV-2' t the questions 'What do your sequences contain?' and 'Which virus?', respectively.


  

  

SECTION: Genbank Submission
9.6. Enter the required submitter information:
  

SECTION: Genbank Submission
9.7. Select the Sequencing Technology used to generate the sequencing data of which the Genbank assembly submissions are composed. Select 'Assembled sequences (...)' as the assembly state:

Illumina:
  
Oxford Nanopore Technologies (and Clear Labs):
  
Note: the assembly method is a default output for the Titan Genomic Characterization workflow. The assembly software and version can be found in your Terra data table:
  

SECTION: Genbank Submission
9.8. 

Select 'Release immediately following processing' and upload the Genbank_assembly.fasta file:

  
  

SECTION: Genbank Submission
9.9. You will be asked to explain the strings of N's in your assemblies. The software used by the Titan Genomic Characterization Workflows estimates the length between sequenced regions using the Wuhan-1 reference genome for alignments:

  

SECTION: Genbank Submission
9.10. We recommend selecting yes for the question 'During processing, should NCBI remove sequences with errors and process the rest?':
  

SECTION: Genbank Submission
9.11. Indicate whether the source of your genomic material was an individual isolate:

  

SECTION: Genbank Submission
9.12. Upload the Genbank_meta_upload file downloaded from Terra in previous steps (Note: This should be the version with the BioSample accession numbers added in Step 8)
 
After submitting the Genbank metadata file a warning may be issued regarding formatting. If the entries in the warning look correct then this warning can be ignored:
   

SECTION: Genbank Submission
9.13. Enter authors to be publicly credited for the submission of this sequencing data. If there is a publication associated with thissequence data please enter the name of the publication as well as the authors listed on the publication:

  

SECTION: Genbank Submission
9.14. Review your submission information and click 'Submit' to complete the Genbank submission process!
  

SECTION: Genbank Submission
9.15. Congratulations! You have submitted both read and assembly data to NCBI, linked by the BioSample accession number. This type of submission greatly enhances the statistical power of the data in public genomic repositories. Thank you for your contribution to public health!