# Exscalate4Cov

Exscalate4Cov was a public-private consortium supported by the Horizon Europe program from the European Union, aimed at leveraging high-performance computing (HPC) as a response to the coronavirus pandemic. The project utilized high-throughput, extreme-scale, computer-aided drug design software to conduct experiments.
The Exsclate4Cov project, which stands for EXaSCale smArt pLatform Against paThogEns for Corona Virus, was coordinated by Dompé Farmaceutici and involved 17 participants. It was part of the Horizon 2020 SOCIETAL CHALLENGES - Health, demographic change and well-being founding funding.
The project conducted one of the largest virtual screening and drug repositioning experiments, identifying a potentially effective molecule against SARS-CoV-2.

## Context
### Background

Drug discovery can be a long and costly process, often taking years and requiring substantial financial investment. Pharmaceutical companies have large datasets of chemical compounds, which they test against a drug target, often a protein receptor. The goal is to find compounds that interact with the targets, leading to potential therapeutic effects.
Therefore, the process of finding new drugs usually involves high-throughput screening (HTS). HTS enables the rapid identification of active compounds. For example, virtual screening can be used as an early stage of the drug discovery pipeline to evaluate the interactions between large datasets of small molecules and a drug target, identifying potential hit candidates. This approach helps in identifying potential hit candidates by predicting how different compounds will bind to the target protein, which will go further in the experimental validation.
In an urgent computing scenario, such as a pandemic, where time to solution is critical, virtual screening is used to identify hit molecules for the latter stages of the drug discovery pipeline, such as lead optimization and clinical trial. The Exscalate4Cov project was initiated after the COVID-19 pandemic outbreak. This project aimed to leverage the computational power of EU supercomputers to accelerate the discovery of effective treatments for the coronavirus. By utilizing high-throughput virtual screening, Exscalate4Cov aimed to find faster solutions to the crisis.

### Scope

Exscalate4Cov's approach involved screening billions of compounds against various protein targets of the SARS-CoV-2 virus, identifying those with a higher binding affinity with the target. The project's objectives were:
* Identify potential drug candidates against the coronavirus to combat the COVID-19 pandemic;
* Conduct a large-scale experiment as an example for future pandemic scenarios;
* Develop a computer-aided drug design platform that leverages supercomputer capabilities;
* Fast sharing of data and scientific discoveries with the community to work in an urgent computing scenario.

### Previous projects

The Exscalate4Cov project followed the ANTAREX4ZIKA project, both of which aimed to leverage HPC for drug discovery, albeit targeting different viruses. While Exscalate4Cov focused on the SARS-CoV-2 virus responsible for COVID-19, ANTAREX4ZIKA was dedicated to addressing the Zika virus. The ANTAREX4ZIKA project concluded at the end of 2018 and involved a virtual screening campaign on the CINECA Marconi machine, with a total of 10 PetaFLOPS. The ANTAREX project, which stands for AutoTuning and Adaptivity appRoach for Energy efficient eXascale HPC systems, emphasized auto-tuning and energy efficiency of HPC applications, making them more effective in various research scenarios, including drug discovery.

### Consortium

The Exscalate4Cov consortium of public-private entities has been coordinated by Dompè, and it involved 17 other institutions, from research centers to universities.
| Organization | Type | Industry | Country |
| Dompé Farmaceutici | Private | Pharmaceutical industry | Italy |
| CINECA | Public research center | Supercomputing | Italy |
| Politecnico di milano | Public university | Scientific and technological research, education | Italy |
| University of Milan | Public university | Scientific and technological research, education | Italy |
| Katholieke Universiteit, Leuven | Public university | Scientific and technological research, education | Belgium |
| International Institute of Molecular and Cell Biology | Public research center | Research center | Poland |
| Elettra Sincrotrone Trieste | Research Organisations | Research center | Italy |
| Fraunhofer-Gesellschaft | Research Organisations | Research center | Germany |
| Barcelona Supercomputing Center | Public research center | Supercomputing | Spain |
| Forschungszentrum Jülich | Public research center | Supercomputing | Germany |
| University of Naples Federico II | Public university | Scientific and technological research, education | Italy |
| University of Cagliari | Public university | Scientific and technological research, education | Italy |
| SIB Swiss Institute of Bioinformatics | Public research center | Research center | Switzerland |
| KTH Royal Institute of Technology | Public university | Scientific and technological research, education | Sweden |
| Lazzaro Spallanzani National Institute for Infectious Diseases | Research Organisations | Hospital | Italy |
| Associtazione Big Data | Company | Other | Italy |
| Istituto Nazionale di Fisica Nucleare | Public research center | Research center | Italy |
| Chelonia SA | Company | Other | Switzerland |

## Pipeline

Inputs at the application level consist of ligands from the chemical space and the protein target of the virtual screening campaign, specifically the spike protein in the case of Exscalate4Cov. Following a molecular docking stage that generates potential ligand conformations, a scoring stage assesses the interaction strength between each ligand's pose and the protein. The pipeline ultimately produces a ranking of hit compounds as its output, indicating the most promising candidates for further investigation.
At the software level, the project utilizes the EXSCALATE docking platform. LiGen (Ligand Generator) is one of the main components of the platform, and it is used to perform molecular docking and scoring simulations. LiGen is responsible for generating and evaluating the conformations of ligands. Another relevant component at the same level is the libdpipe library, which facilitates scaling across multi-node and cores.
To hinge the computational power offered by HPC centers, the docking platform uses MPI to scale multi-node and CUDA acceleration to take advantage of supercomputer's GPUs. The CUDA version has undergone various optimizations, including OpenACC, OpenMP, and other techniques, to enhance performance and efficiency.

## Virtual screening campaign

The project's main experiment evaluated the interactions between 12 viral proteins of SARS-CoV-2 against 70 billion molecules from the EXSCALATE chemical library. In November 2020, consortium members coordinated one of the largest virtual screening campaigns, harnessing the combined computational power of two supercomputers totaling 81 PFLOPS.
The supercomputers used are:
* Marconi100: Operated by CINECA, each node consists of 1 IBM POWER9 AC922 CPU (32 cores, 128 threads) and 4 NVIDIA V100 GPUs with 16 GB of VRAM. The machine consists of 970 nodes, providing a total of 29.3 PFLOPS.
* HPC5: Operated by Eni, each node consists of 1 Intel Xeon Gold 6252 24C CPU (24 cores, 48 threads) and 4 NVIDIA V100 GPUs with 16 GB of VRAM. The machine consists of 1820 nodes, providing a total of 51.7 PFLOPS.

### Throughput

The large-scale campaign used a reservation of 800 Marconi100 nodes and 1500 HP5 nodes for 60 hours. Achieving an average throughput was 2400 ligands per second (lig/s) on Marconi100 and 2000 lig/s on HPC5.

### Data storage

Another critical aspect of the experiment was data storage management. The platform leveraged efficient MPI I/O operations to handle multi-node computations. The input data required 3.3 TB of space in SMILES format. However, SMILES data needed to be expanded in a pre-processing step involving 100 nodes over five days. Similarly, the post-processing step involved 19 nodes over five days.

### Output data

The final output consisted of CSV files containing scores for each input ligand, occupying 69 TB. The resulting dataset, containing 570 million hit compounds, is freely available.

### Drug repositioning

The Exscalate4Cov project also conducted drug repositioning experiments. Drug repurposing offers an interesting approach to address unmet clinical needs in case of urgent computing, due to pandemics. Hence, repurposing existing drugs with established safety and toxicology profiles provides a significant advantage by saving time in identifying potential new treatments. During the European Exscalate4Cov project activities, raloxifene was selected through a combined approach of drug repurposing and in-silico screening on SARS-CoV-2 target’s proteins, followed by subsequent in-vitro screening.

## Results
### Mediate

The project's large-scale campaign results are available through the MEDIATE (MolEcular DockIng AT homE) platform. The objective of MEDIATE is to collect a chemical library of Sars-COV-2 inhibitors. The MEDIATE portal provides access to a set of small molecules that research can use to start de-novo drug design from a reduced set of molecules.

### Raloxifene

Raloxifene is a known chemical compound used to treat osteoporosis. As a result of drug repositioning experiments, the E4C project identified raloxifene as a possible candidate to treat early-stage COVID-19 patients, aiming to prevent clinical progression. In October 2020, AIFA authorized clinical trials to treat COVID-19 patients, and it is currently undergoing testing for approval.

## Public interest

The experiments, including the discovery of raloxifene as a possible drug candidate against COVID-19, gained significant interest from the scientific community, as documented in several scientific articles.
The project's results also captured national interest in Italy, highlighted by various newspaper articles, due to the use of Italian supercomputers during the pandemic. Additionally, the large-scale campaign results gained attention from international journals.