TITLE: SRA and Genbank BioSample-Linked Submission with Mercury_Prep and Mercury_Batch
AUTHORS: Francis Ambrosio

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">Submitting sequencing data to public data repositories is a meaningful yet tedious procedure. Linking submissions between SRA and Genbank will enhance the value of both submissions the the public health community. The Mercury protocols offered by Theiagen Genomics allows users to efficiently and accurately produce all required inputs for SRA and Genbank submissions (the Mercury workflows also allow for GISAID submission, but that will not be covered in this protocol). This protocol provides a detailed procedure for submitting BioSample-linked sequencing data to SRA and Genbank.</div></div>

[STEPS]
?. [Data Preparation]
The Titan Genomic Characterization workflow must be run prior to submitting sequences to SRA and Genbank in order to prepare the data for submission. Please use the Titan workflow that is compatible with your sequencing data.
?. [Metadata Formatting]
The Terra Metadata Formatter is an excel spreadsheet too that will help you by collecting all required metadata for each of the sequencing data repositories and formatting this data into a Terra-uploadable data table.
?. [Metadata Formatting]
Download and open the Terra Metadata Formatter:https://storage.googleapis.com/theiagen-public-files/terra/mercury-files/Terra_Metadata_Formatter_2021_09_09.xlsx
?. [Metadata Formatting]
Enter the sample metadata into the 'User Input' tab of the Terra Metadata Formatter. The required fields are highlighted in blue. The optional fields are highlighted in grey. We recommend that you attempt to include as much data about your samples as is available at the time of submission, with particular emphasis on the fields of 'Purpose of Sampling' and 'Purpose of Sequencing', which will be used to correct for statistical biases in the data due to diversity of the sampling methodologies.Note that some of the fields have dropdown menus. These have been implemented for fields that have a controlled vocabulary in order to reduce typo-based rejections from the various databases.The General Metadata section consists of two required fields:Root Entity: This input will define the name of the Terra Data Table when this metadata is uploaded in subsequent steps.Submission ID Prefix: This input will be the prefix to the submission ID in the final NCBI submission files. Typical inputs are formatted as the state abbreviation and laboratory abbreviation separated by a hyphen.The Laboratory Data section consists of eight required fields and one optional field:GISAID Submitter ID (required): the GISAID Submission ID in the final GISAID submission files (if you have already submitted these samples to GISAID then list the GISAID Submission ID that was used)Authors (required): the list of authors included in the final SRA, Genbank and GISAID submission filesBioProject (required): the BioProject accession number used in the SRA and Genbank submissionsState: the state of the Originating LaboratoryCountry (required): the country of the Originating LaboratoryContinent (required): the continent of the Originating LaboratorySubmitting Laboratory (required): the name of the Submitting LaboratorySubmitting Laboratory Address (required): the address of the Submitting LaboratorySubmitter Email (optional): The email associated with the NCBI account that will be used to submit to SRA and GenbankThe Sequencing Run section consists of five required fields and two optional fields:Platform (required): the sequencing Platform used to generate this sequencing dataInstrument Model (required): the sequencing Instrument Model used to generate this sequencing dataLibrary Strategy (required): the Library Strategy used to generate the sequencing libraries (if using Artic V3 or similar amplicon-based protocol then "AMPLICON" is the most accurate entry for this field.)Library Source (required): the material used as the Library Source in the generation of the sequencing libraries (if extracting viral RNA as starting material then "VIRAL RNA" is the most accurate entry for this field.)Library Selection (required): the tool used to select libraries to be sequencedPrimer Scheme (optional): the Primer Scheme in the amplicon generation step of the library preparationAmplicon Size (optional): the average Amplicon Size of the Primer SchemeThe Sample Metadata section consists of nine required fields and nine optional fields:Samples (required): the unique ID of the SamplesSubmission ID Suffix (required): the second component of the Submission ID (this field can be the same as Samples)Library ID Suffix (required): this input is used to keep track of samples that have been sequenced more than once, or on multiple platforms (for the first or only sequencing submission for these samples it is recommended to use "01" for this field)Collection Date (required): the date the samples were originally collectedOriginating Lab(required): the laboratory where the samples were originally collectedOriginating Lab Address (required): the address of the laboratory where the samples were originally collectedOrganism (required): the target organism of the sequencing run (if sequencing SARS-CoV-2 the "SARS-CoV-2" is the most accurate entry for this fieldIsolation Source (required): source of the sample (if sequencing samples that were collected as part of a diagnostic assay or or surveillance program from humans then "Clinical" would be the most accurate entry for this fieldHost Disease (required): disease caused by the target Organism (if sequencing SARS-CoV-2 the "COVID-19" would be the most accurate entry for this field) Run ID (optional): the Run ID of the samplesPatient Gender (optional): the gender of the individual from whom the sample was collectedPatient Age (optional): the age of the individual from whom the sample was collectedCounty (optional): the county from which the sample was collectedBioSample Accession (optional): if the sample has already been registered with NCBI then include the BioSample hereSpecimen Processing (optional): sample processing steps such as transport media and extraction method can be included herePurpose of Sampling (optional): this input can be clinical diagnostics if the sample was taken as a human specimen for SARS-CoV-2 testingPurpose of Sequencing (optional): this input can be used to tag samples as Baseline Surveillance or Targeted Sampling (for detailed guidance on what entry is most accurate for your samples please see the APHL guidance document here: https://www.aphl.org/programs/preparedness/Crisis-Management/Documents/Technical-Assistance-for-Categorizing-Baseline-Surveillance.pdf)        For Baseline Surveillance:         1. Sampled randomly for genomic surveillance
         2. Those not identified in a targeted sampling effort (targeted efforts defined below)
         3. Sampled across targeted sequencing efforts to be representative of the community        For Targeted Sequencing:         1. Sampled based on cluster/outbreak investigations
         2. Longitudinally or repeatedly sampled from the same individual
         3. Sampled based on pre-screening for a particular variant (e.g., S-gene target failure)
         4. Sampled for the purpose of vaccine escape studies
         5. Sampled based on travel history
         6. Sampled based on disease severity (i.e., targeted sequencing of cases resulting in hospitalization
             or death)Sequencing Protocol Name: if using a named sequencing protocol enter the name in this fieldThe Laboratory Data section consists of eight required fields and one optional field:GISAID Submitter ID (required): the GISAID Submission ID in the final GISAID submission files (if you have already submitted these samples to GISAID then list the GISAID Submission ID that was used)Authors (required): the list of authors included in the final SRA, Genbank and GISAID submission filesBioProject (required): the BioProject accession number used in the SRA and Genbank submissionsState: the state of the Originating LaboratoryCountry (required): the country of the Originating LaboratoryContinent (required): the continent of the Originating LaboratorySubmitting Laboratory (required): the name of the Submitting LaboratorySubmitting Laboratory Address (required): the address of the Submitting LaboratorySubmitter Email (optional): The email associated with the NCBI account that will be used to submit to SRA and GenbankThe Sequencing Run section consists of five required fields and two optional fields:Platform (required): the sequencing Platform used to generate this sequencing dataInstrument Model (required): the sequencing Instrument Model used to generate this sequencing dataLibrary Strategy (required): the Library Strategy used to generate the sequencing libraries (if using Artic V3 or similar amplicon-based protocol then "AMPLICON" is the most accurate entry for this field.)Library Source (required): the material used as the Library Source in the generation of the sequencing libraries (if extracting viral RNA as starting material then "VIRAL RNA" is the most accurate entry for this field.)Library Selection (required): the tool used to select libraries to be sequencedPrimer Scheme (optional): the Primer Scheme in the amplicon generation step of the library preparationAmplicon Size (optional): the average Amplicon Size of the Primer SchemeThe Sample Metadata section consists of nine required fields and nine optional fields:Samples (required): the unique ID of the SamplesSubmission ID Suffix (required): the second component of the Submission ID (this field can be the same as Samples)Library ID Suffix (required): this input is used to keep track of samples that have been sequenced more than once, or on multiple platforms (for the first or only sequencing submission for these samples it is recommended to use "01" for this field)Collection Date (required): the date the samples were originally collectedOriginating Lab(required): the laboratory where the samples were originally collectedOriginating Lab Address (required): the address of the laboratory where the samples were originally collectedOrganism (required): the target organism of the sequencing run (if sequencing SARS-CoV-2 the "SARS-CoV-2" is the most accurate entry for this fieldIsolation Source (required): source of the sample (if sequencing samples that were collected as part of a diagnostic assay or or surveillance program from humans then "Clinical" would be the most accurate entry for this fieldHost Disease (required): disease caused by the target Organism (if sequencing SARS-CoV-2 the "COVID-19" would be the most accurate entry for this field) Run ID (optional): the Run ID of the samplesPatient Gender (optional): the gender of the individual from whom the sample was collectedPatient Age (optional): the age of the individual from whom the sample was collectedCounty (optional): the county from which the sample was collectedBioSample Accession (optional): if the sample has already been registered with NCBI then include the BioSample hereSpecimen Processing (optional): sample processing steps such as transport media and extraction method can be included herePurpose of Sampling (optional): this input can be clinical diagnostics if the sample was taken as a human specimen for SARS-CoV-2 testingPurpose of Sequencing (optional): this input can be used to tag samples as Baseline Surveillance or Targeted Sampling (for detailed guidance on what entry is most accurate for your samples please see the APHL guidance document here: https://www.aphl.org/programs/preparedness/Crisis-Management/Documents/Technical-Assistance-for-Categorizing-Baseline-Surveillance.pdf)        For Baseline Surveillance:         1. Sampled randomly for genomic surveillance
         2. Those not identified in a targeted sampling effort (targeted efforts defined below)
         3. Sampled across targeted sequencing efforts to be representative of the community        For Targeted Sequencing:         1. Sampled based on cluster/outbreak investigations
         2. Longitudinally or repeatedly sampled from the same individual
         3. Sampled based on pre-screening for a particular variant (e.g., S-gene target failure)
         4. Sampled for the purpose of vaccine escape studies
         5. Sampled based on travel history
         6. Sampled based on disease severity (i.e., targeted sequencing of cases resulting in hospitalization
             or death)Sequencing Protocol Name: if using a named sequencing protocol enter the name in this fieldThe Sequencing Run section consists of five required fields and two optional fields:Platform (required): the sequencing Platform used to generate this sequencing dataInstrument Model (required): the sequencing Instrument Model used to generate this sequencing dataLibrary Strategy (required): the Library Strategy used to generate the sequencing libraries (if using Artic V3 or similar amplicon-based protocol then "AMPLICON" is the most accurate entry for this field.)Library Source (required): the material used as the Library Source in the generation of the sequencing libraries (if extracting viral RNA as starting material then "VIRAL RNA" is the most accurate entry for this field.)Library Selection (required): the tool used to select libraries to be sequencedPrimer Scheme (optional): the Primer Scheme in the amplicon generation step of the library preparationAmplicon Size (optional): the average Amplicon Size of the Primer SchemeThe Sample Metadata section consists of nine required fields and nine optional fields:Samples (required): the unique ID of the SamplesSubmission ID Suffix (required): the second component of the Submission ID (this field can be the same as Samples)Library ID Suffix (required): this input is used to keep track of samples that have been sequenced more than once, or on multiple platforms (for the first or only sequencing submission for these samples it is recommended to use "01" for this field)Collection Date (required): the date the samples were originally collectedOriginating Lab(required): the laboratory where the samples were originally collectedOriginating Lab Address (required): the address of the laboratory where the samples were originally collectedOrganism (required): the target organism of the sequencing run (if sequencing SARS-CoV-2 the "SARS-CoV-2" is the most accurate entry for this fieldIsolation Source (required): source of the sample (if sequencing samples that were collected as part of a diagnostic assay or or surveillance program from humans then "Clinical" would be the most accurate entry for this fieldHost Disease (required): disease caused by the target Organism (if sequencing SARS-CoV-2 the "COVID-19" would be the most accurate entry for this field) Run ID (optional): the Run ID of the samplesPatient Gender (optional): the gender of the individual from whom the sample was collectedPatient Age (optional): the age of the individual from whom the sample was collectedCounty (optional): the county from which the sample was collectedBioSample Accession (optional): if the sample has already been registered with NCBI then include the BioSample hereSpecimen Processing (optional): sample processing steps such as transport media and extraction method can be included herePurpose of Sampling (optional): this input can be clinical diagnostics if the sample was taken as a human specimen for SARS-CoV-2 testingPurpose of Sequencing (optional): this input can be used to tag samples as Baseline Surveillance or Targeted Sampling (for detailed guidance on what entry is most accurate for your samples please see the APHL guidance document here: https://www.aphl.org/programs/preparedness/Crisis-Management/Documents/Technical-Assistance-for-Categorizing-Baseline-Surveillance.pdf)        For Baseline Surveillance:         1. Sampled randomly for genomic surveillance
         2. Those not identified in a targeted sampling effort (targeted efforts defined below)
         3. Sampled across targeted sequencing efforts to be representative of the community        For Targeted Sequencing:         1. Sampled based on cluster/outbreak investigations
         2. Longitudinally or repeatedly sampled from the same individual
         3. Sampled based on pre-screening for a particular variant (e.g., S-gene target failure)
         4. Sampled for the purpose of vaccine escape studies
         5. Sampled based on travel history
         6. Sampled based on disease severity (i.e., targeted sequencing of cases resulting in hospitalization
             or death)Sequencing Protocol Name: if using a named sequencing protocol enter the name in this fieldThe Sample Metadata section consists of nine required fields and nine optional fields:Samples (required): the unique ID of the SamplesSubmission ID Suffix (required): the second component of the Submission ID (this field can be the same as Samples)Library ID Suffix (required): this input is used to keep track of samples that have been sequenced more than once, or on multiple platforms (for the first or only sequencing submission for these samples it is recommended to use "01" for this field)Collection Date (required): the date the samples were originally collectedOriginating Lab(required): the laboratory where the samples were originally collectedOriginating Lab Address (required): the address of the laboratory where the samples were originally collectedOrganism (required): the target organism of the sequencing run (if sequencing SARS-CoV-2 the "SARS-CoV-2" is the most accurate entry for this fieldIsolation Source (required): source of the sample (if sequencing samples that were collected as part of a diagnostic assay or or surveillance program from humans then "Clinical" would be the most accurate entry for this fieldHost Disease (required): disease caused by the target Organism (if sequencing SARS-CoV-2 the "COVID-19" would be the most accurate entry for this field) Run ID (optional): the Run ID of the samplesPatient Gender (optional): the gender of the individual from whom the sample was collectedPatient Age (optional): the age of the individual from whom the sample was collectedCounty (optional): the county from which the sample was collectedBioSample Accession (optional): if the sample has already been registered with NCBI then include the BioSample hereSpecimen Processing (optional): sample processing steps such as transport media and extraction method can be included herePurpose of Sampling (optional): this input can be clinical diagnostics if the sample was taken as a human specimen for SARS-CoV-2 testingPurpose of Sequencing (optional): this input can be used to tag samples as Baseline Surveillance or Targeted Sampling (for detailed guidance on what entry is most accurate for your samples please see the APHL guidance document here: https://www.aphl.org/programs/preparedness/Crisis-Management/Documents/Technical-Assistance-for-Categorizing-Baseline-Surveillance.pdf)        For Baseline Surveillance:         1. Sampled randomly for genomic surveillance
         2. Those not identified in a targeted sampling effort (targeted efforts defined below)
         3. Sampled across targeted sequencing efforts to be representative of the community        For Targeted Sequencing:         1. Sampled based on cluster/outbreak investigations
         2. Longitudinally or repeatedly sampled from the same individual
         3. Sampled based on pre-screening for a particular variant (e.g., S-gene target failure)
         4. Sampled for the purpose of vaccine escape studies
         5. Sampled based on travel history
         6. Sampled based on disease severity (i.e., targeted sequencing of cases resulting in hospitalization
             or death)Sequencing Protocol Name: if using a named sequencing protocol enter the name in this fieldPurpose of Sequencing (optional): this input can be used to tag samples as Baseline Surveillance or Targeted Sampling (for detailed guidance on what entry is most accurate for your samples please see the APHL guidance document here: https://www.aphl.org/programs/preparedness/Crisis-Management/Documents/Technical-Assistance-for-Categorizing-Baseline-Surveillance.pdf)        For Baseline Surveillance:         1. Sampled randomly for genomic surveillance
         2. Those not identified in a targeted sampling effort (targeted efforts defined below)
         3. Sampled across targeted sequencing efforts to be representative of the community        For Targeted Sequencing:         1. Sampled based on cluster/outbreak investigations
         2. Longitudinally or repeatedly sampled from the same individual
         3. Sampled based on pre-screening for a particular variant (e.g., S-gene target failure)
         4. Sampled for the purpose of vaccine escape studies
         5. Sampled based on travel history
         6. Sampled based on disease severity (i.e., targeted sequencing of cases resulting in hospitalization
             or death)
?. [Upload Metadata]
Upload the Terra Data Table
?. [Upload Metadata]
Once the sample metadata has been entered into the User Input tab of the Metadata Formatter click the 'Terra Data Table' tab:
?. [Upload Metadata]
Select the whole sheet by hitting control+'a' on your keyboard.Copy the whole sheet by hitting control+'c' on your keyboard.
?. [Upload Metadata]
Log in and navigate to the Data tab in your workspace on Terra.bio:
?. [Upload Metadata]
Select the plus button in the blue circle to add a Terra Data Table:
?. [Upload Metadata]
Select the Text Import tab:
?. [Upload Metadata]
Paste your metadata into the text input field:
?. [Upload Metadata]
Click UPLOAD
?. [Mercury]
Mercury Prep
?. [Mercury]
Select Mercury_Prep_SE or Mercury_Prep_PE from the Workflows tab in your Terra workspace:
?. [Mercury]
Choose the appropriate Version and Root Entity, then click Select Data:
?. [Mercury]
Select the samples that you would like to prepare for submission:
?. [Mercury]
Enter the input attributes:
?. [Mercury]
Select the default outputs:
?. [Mercury]
Once the inputs and outputs have been defined, Save the workflow parameters:
?. [Mercury]
Click RUN ANALYSISConfirm and launch the analysis by clicking LAUNCH:
?. [Mercury]
Mercury Batch
?. [Mercury]
Once Mercury Prep has successfully completed navigate to the Mercury Batch workflow:
?. [Mercury]
Select the appropriate version of the workflow:
?. [Mercury]
Select the SET LEVEL root entity type:
?. [Mercury]
Click SELECT DATA:
?. [Mercury]
Select the dataset of sample that you would like to batch for submission (Note: the dataset root entity is the plural form of the original root entity):
?. [Mercury]
Enter the INPUTS. The inputs for Mercury Batch will be entered at the Array Level. This means the notation will be formatted as this.data_sets.{attribute}:And note the set level attribute (middle of the two decimal points) is the pleural form of the original root entity.
?. [Mercury]
Enter the public GCP bucket to stage your data for the final submission to NCBI SRA:
?. [Mercury]
Select the default OUTPUTS:
?. [Mercury]
Once the inputs and outputs have been defined, Save the workflow parameters:
?. [Mercury]
Click RUN ANALYSISConfirm and launch the analysis by clicking LAUNCH:
?. [Mercury]
Retrieve your submission files by navigating to the Terra Data Table containing the Mercury Batch outputs:Click on the file names in blue:Download the files:These are the four files that will be required for SRA and Genbank submission:
?. [SRA Submission]
Submit your data to SRA (and simultaneously generate BioSample accession numbers for your samples)
?. [SRA Submission]
Navigate and Log in to the SRA Submission Portal:
?. [SRA Submission]
Select New Submission:
?. [SRA Submission]
Enter your submitter information, select your submission group, and enter the information for your organization:
?. [SRA Submission]
Enter your BioProject number:
?. [SRA Submission]
Select 'No' if you do not already have BioSample accession numbers for your samples in order to generate them upon SRA submission:
?. [SRA Submission]
Select your Release Date (we recommend releasing your data immediately following processing:
?. [SRA Submission]
Select the appropriate submission package (if you are submitting SARS-CoV-2 sequences extracted from a human specimen please select the SARS-CoV-2 clinical or host-associated package):
?. [SRA Submission]
Choose the 'Upload a file...' option and upload the BioSample attributes file downloaded in previous steps:
?. [SRA Submission]
Choose the 'Upload a file...' option and upload the SRA Metadata file downloaded in previous steps:
?. [SRA Submission]
Select the 'AWS or GCP bucket' option and enter the name of the public data bucket where your reads have been placed in the staging phase of the data submission procedure:
?. [SRA Submission]
Review and Submit to complete your SRA submission! You will be able to download your BioSample accession numbers from the SRA submission portal as soon as they become available.
?. [SRA Submission]
Retrieve the BioSample accession numbers '.tsv' file from the SRA portal
?. [SRA Submission]
Navigate to the SRA Submission Portal (you should already be logged in)Locate the Status column of the submissions table:
?. [SRA Submission]
Click 'Download attributes file with BioSample accessions' for the SRA submission executed earlier in this protocol:
?. [Genbank Submission]
Add BioSample accession numbers to Genbank_meta_upload file
?. [Genbank Submission]
Open the attributes file downloaded from SRA containing the BioSample accession numbers
?. [Genbank Submission]
Open the Genbank_meta_sra file downloaded from Terra (the output from Mercury Batch)
?. [Genbank Submission]
Use XLOOKUP to algorithmically add the BioSample accession numbers to the Genbank_meta_sra file.
?. [Genbank Submission]
Save the Genbank_meta_upload file (now including the BioSample accession numbers)
?. [Genbank Submission]
Genbank submission
?. [Genbank Submission]
Navigate to the Genbank submission portal
?. [Genbank Submission]
Select SARS-CoV-2
?. [Genbank Submission]
Click the submit button under the Genbank heading:
?. [Genbank Submission]
Select 'New submission':
?. [Genbank Submission]
Select 'SARS-CoV-2, Influenza, Norovirus, or Dengue virus' and 'SARS-CoV-2' t the questions 'What do your sequences contain?' and 'Which virus?', respectively.
?. [Genbank Submission]
Enter the required submitter information:
?. [Genbank Submission]
Select the Sequencing Technology used to generate the sequencing data of which the Genbank assembly submissions are composed. Select 'Assembled sequences (...)' as the assembly state:Illumina:Oxford Nanopore Technologies (and Clear Labs):Note: the assembly method is a default output for the Titan Genomic Characterization workflow. The assembly software and version can be found in your Terra data table:
?. [Genbank Submission]
Select 'Release immediately following processing' and upload the Genbank_assembly.fasta file:
?. [Genbank Submission]
You will be asked to explain the strings of N's in your assemblies. The software used by the Titan Genomic Characterization Workflows estimates the length between sequenced regions using the Wuhan-1 reference genome for alignments:
?. [Genbank Submission]
We recommend selecting yes for the question 'During processing, should NCBI remove sequences with errors and process the rest?':
?. [Genbank Submission]
Indicate whether the source of your genomic material was an individual isolate:
?. [Genbank Submission]
Upload the Genbank_meta_upload file downloaded from Terra in previous steps (Note: This should be the version with the BioSample accession numbers added in Step 8)
?. [Genbank Submission]
Enter authors to be publicly credited for the submission of this sequencing data. If there is a publication associated with thissequence data please enter the name of the publication as well as the authors listed on the publication:
?. [Genbank Submission]
Review your submission information and click 'Submit' to complete the Genbank submission process!
?. [Genbank Submission]
Congratulations! You have submitted both read and assembly data to NCBI, linked by the BioSample accession number. This type of submission greatly enhances the statistical power of the data in public genomic repositories. Thank you for your contribution to public health!
?. [Data Preparation]
Please check that all samples have been analyzed using the appropriate Titan workflow prior to running the Mercury workflows by navigating to the 'Data' tab, selecting the data table of choice, and select the 'assembly_fasta' and 'assembly_method' columns.