TITLE: Complete WGS data processing in HPC environment
AUTHORS: Ireneusz Stolarek, Michał Zeńczak, Małgorzata Marcinkowska-Swojak, Magdalena Rakoczy, Luiza Handschuh, Natalia Koralewska, Marek Figlerowicz

[DESCRIPTION]
In the rapidly evolving landscape of genomics research, the advent of whole genome sequencing (WGS) has ushered in an era of unprecedented data volumes, necessitating robust and efficient computational strategies for data processing. This protocol delineates a comprehensive and optimized workflow specifically tailored for the intricate demands of WGS data analysis on High-Performance Computing (HPC) infrastructure, leveraging the Slurm scheduler for efficient resource management.
The protocol begins with meticulous data preprocessing, encompassing quality control, trimming, and adapter removal, while concurrently harnessing the parallel processing capabilities of HPC clusters. The subsequent alignment of preprocessed reads to a reference genome is performed utilizing parallelized tools, ensuring computational efficiency and minimizing analysis time. Variant calling, a pivotal step in WGS analysis, is seamlessly integrated, and the protocol underscores the exploitation of HPC resources for parallelized execution, essential for managing the computational intensity of this step.
Quality control metrics, variant annotation, and prioritization are addressed with a focus on HPC-specific considerations, providing a robust foundation for downstream analysis.
Emphasizing troubleshooting strategies, best practices, and recommendations, this protocol equips researchers with the tools necessary to navigate the complexities of WGS data processing on HPC infrastructure. By providing a structured and adaptable framework, this protocol serves as a valuable resource for genomics researchers aiming to harness the power of high-performance computing in the analysis of whole genome sequencing data.

[STEPS]
SECTION: Data Demultiplexing
9. In processing NGS data, data demultiplexing is often a first computational step. 
The demultiplexing of illumina based data can be conducted using bcl2fastq software.


 


SECTION: Pre-Alignment QC
10. QC of the NGS data can be performed with a variety of tools. Here we present a typical output of an interactive QC tool multiqc


 




 


SECTION: Alignment
12. Mapping paired truncated
 


SECTION: Post-Alignment QC
14. 
 


SECTION: Variant Calling
15. Base recalibration according to known polymorphic sites
 


SECTION: Functional Annotation
17. Annotate variants with VEP

 

SECTION: Software stack installation for HPC data processing
4. To process the NGS data from WGS experiment in an HPC environment we recommend installing the necessary software through conda or by other means.

SECTION: Software stack installation for HPC data processing
5. Install conda
 

Download and install the conda linux installer from:
https://docs.conda.io/projects/miniconda/en/latest/index.html

 

SECTION: Software stack installation for HPC data processing
5.2. Create conda virtual environment through mamba

 

SECTION: Software stack installation for HPC data processing
5.3. Activate conda virtual environment
* When working in scheduler mode (sbatch), please remember to activate the virtual environment when on local access node before submitting the job to the slurm scheduler
* When workin in interactie mode (srun), activate the virtual environment after being allocated a compuational node (srun --pty /bin/bash)
 

SECTION: Software stack installation for HPC data processing
5.1. Install mamba
* mamba is an optional add-on, which makes installation of the downstream software much more efficient. We highly recommend installation of mamba.
* We recommend to install mamba in the base environment
SECTION: Create base workflow script
8. Here we introduce a scheme of a basic slurm workflow script written in bash. This script automates all of the steps required to conduct the desired analyses

The first part is the header. It contains the desired resources that will be allocated on a computing node for the scheduled job.


 

The second part consists of all important placeholders and paths to the input and output data


 

The third part consists of computational workflow specific commands


 

*In the following sections we will modify only the third section of the workflow scriptss
SECTION: Data Demultiplexing
9.1. Sample sheet preparation

 

SECTION: Optimizing HPC resources
13. To optimize resource allocation for human WGS data processing first we need to establish the minimal hardware requirements. The basic components consist of number of CPUs used, RAM usage and computation time. All three parameters are interrelated. The faster we want to finish the alignment step, the more CPUs we need to allocate, which requires more RAM allocation.
As a rule of thumb we are targeting to finish the alignment within 24-36 hours (this mostly depends on the hardware quality at our disposal). Given that, the required resources most optial to paralellize the computation are per 1 computational node:
- 24 CPUs
- 32 GB RAM


 

 

SECTION: Pre-Demultiplexing sequencing run QC
1. The very first data obtained after the NGS run are not the well-known FASTQ files. The first data consists of raw pixel-based data and intensity measures taken by the instrument in the form of .bcl files. The data are generated by the Real Time Analysis software by the Illumina sequencing systems during the run.


 
* The desired specifications of the run to be reached are available on the illumina website per each sequencing system
SECTION: Pre-Demultiplexing sequencing run QC
2. Install Sequencing Analysis Viewer
SECTION: Pre-Demultiplexing sequencing run QC
3. Pre-Demultiplexing sequencing run QC


 

We recommend sanity checks for signal intensity separation for each channel, error rate calculated as a percentage from phiX alignment, Q30 fraction, and cluster passing filter to identify potential problems with the suboptimal run.
SECTION: Pre-Alignment data filtering
11. 
 

SECTION: Post-Alignment QC
14.1. Additionally as a sanity check we advise to calculate average genome coverage per each sample

 

SECTION: Create scheduling scripts
7. To fully automate HPC data processing prepare scheduling scripts.

SECTION: Variant Calling
15.1. BQSR recalibration
 

SECTION: Variant Calling
15.2. Indexing

 

SECTION: Variant Calling
15.3. Variant calling
 

SECTION: Software stack installation for HPC data processing
6. Install basic software into virtual environment

* remember to conduct software installation while being on the local access mode, and after activation of the selected virtual environment (conda activate virtual_env name)
* installation of software while being on a computational node might revert any changes to the virtual environment once exiting
SECTION: Software stack installation for HPC data processing
6.1. Illumina data demultiplexing software

 

 

SECTION: Software stack installation for HPC data processing
6.2. NGS read alignment software

 

 

SECTION: Software stack installation for HPC data processing
6.3. Pre and Post-Alignment QC software

 

 

SECTION: Software stack installation for HPC data processing
6.4. Pre-alignment data filtering software


 


 

SECTION: Software stack installation for HPC data processing
6.5. Post-Alignment data processing software


 

 

 

 

SECTION: Software stack installation for HPC data processing
6.6. Variant Calling software

 

 

SECTION: Create scheduling scripts
7.1. Demultiplexing scheduling script

 

SECTION: Create scheduling scripts
7.2. Alignment scheduling script

 

SECTION: Create scheduling scripts
7.3. Genotyping scheduling script

 

SECTION: Create scheduling scripts
7.4. Variant filtering script

 

SECTION: Create scheduling scripts
7.5. Variant annotation scheduling scripts

 

SECTION: Create scheduling scripts
7.6. Variant prioritization scheduling script

 

SECTION: Alignment
12.1. Indexing

 

SECTION: Alignment
12.2. Removing duplicates

 

SECTION: Alignment
12.3. Adding read group information

 

SECTION: Variant Calling
16.5. Prioritize Variants

 

SECTION: Variant Calling
16.4. Indel Variant VQSR

 

SECTION: Variant Calling
16.3. Indel Variant recalibration

 

SECTION: Variant Calling
16.2. SNP Variant VQSR

 


SECTION: Variant Calling
16.1. SNP Variant recalibration

 

SECTION: Variant Calling
16. Variant filtering

SECTION: Functional Annotation
17.1. Filter variants


 

SECTION: Pre-Demultiplexing sequencing run QC
3.1. The analysis tab

 

SECTION: Pre-Demultiplexing sequencing run QC
3.2. The summary tab