# ParSMURF-NG: A Machine Learning High Performance Computing System for the Analysis of Imbalanced Big Omics Data

Alessandro Petrini, Marco Notaro, Jessica Gliozzo, Tiziana Castrignanò, Peter

N. Robinson, Elena Casiraghi, Giorgio Valentini

► To cite this version:

Alessandro Petrini, Marco Notaro, Jessica Gliozzo, Tiziana Castrignanò, Peter N. Robinson, et al.: ParSMURF-NG: A Machine Learning High Performance Computing System for the Analysis of Imbalanced Big Omics Data. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.424-435, 10.1007/978-3-031-08341-9_34. [https://doi.org/10.1007/978-3-031-08341-9_34](https://doi.org/10.1007/978-3-031-08341-9_34)

HAL Id: hal-05335635

https://inria.hal.science/hal-05335635v1

Submitted on 28 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# ParSMURF-NG: A Machine Learning
High Performance Computing System
for the Analysis of Imbalanced Big
Omics Data

Alessandro Petrini<fnref n="1" />,<fnref n="3" />(✉) ID, Marco Notaro<fnref n="1" />,<fnref n="3" /> ID, Jessica Gliozzo<fnref n="1" />,<fnref n="3" /> ID,
Tiziana Castrignanò<fnref n="5" /> ID, Peter N. Robinson<fnref n="6" /> ID, Elena Casiraghi<fnref n="1" />,<fnref n="2" />,<fnref n="3" /> ID,
and Giorgio Valentini<fnref n="1" />,<fnref n="2" />,<fnref n="3" />,<fnref n="4" /> ID

**Abstract.** In the context of Genomic and Precision Medicine, prediction problems are often characterized by a high imbalance between classes and Big Data. This requires specialized tools, as traditional Machine Learning approaches may struggle with big datasets and often fail to predict the minority class with unbalanced classification problems.

In this work we present ParSMURF-NG, a High Performance Computing-oriented Machine Learning approach designed to scale well on big omics data. We measured its performance capabilities on three current-generation HPC systems and we showed its usefulness in the context of Genomic Medicine, providing a powerful model for the detection of pathogenic single nucleotide variants in the non-coding regions of the human genome.

**Keywords:** Parallel machine learning tool for big data · Machine learning for genomic medicine · Prediction of deleterious variants · Machine learning tool for imbalanced data

# 1 Introduction

The latest developments in high-throughput technologies and Artificial Intelligence have provided unprecedented tools for the development of Precision and Personalized Medicine. In particular, in recent years whole genome sequencing (WGS) has become cheap, accessible and reliable, hence making large population genome sequencing projects feasible [1, 19]. Moreover, a pivotal role in the

<footnote n="1">¹ AnacletoLab - Dip. Informatica, Università degli Studi di Milano, Milan, Italy
alessandro.petrini@unimi.it</footnote>

<footnote n="2">² Data Science Research Center, Università degli Studi di Milano, Milan, Italy</footnote>

<footnote n="3">³ CINI - Laboratorio Nazionale Infolife, Roma, Italy</footnote>

<footnote n="4">⁴ CINI - Laboratorio Nazionale Big Data, Roma, Italy</footnote>

<footnote n="5">⁵ Department of Ecological and Biological Sciences, Università della Tuscia,
Viterbo, Italy</footnote>

<footnote n="6">⁶ The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA</footnote>
analysis of the massive amount of data generated by Next Generation Sequencing (NGS) techniques [10] is played by Machine Learning (ML), as automatic learning algorithms are capable of detecting underlying patterns in data which traditional statistical methods may overlook. Advances in NGS and ML have made variant identification through the analysis of NGS data central in Genomic Medicine; however, most previous studies focus on the investigation of disease-associated protein-coding variants [2,9], and our understanding of the impact of variants located in the non-coding regions of the genome is mostly incomplete. However, several studies reported that most of the potential pathogenic variants are not in the coding areas of the genome [5].

In particular, a pivotal problem in this research field is the identification of the causal variants for the ~8000 known genetic Mendelian diseases, as one of the major problems in Genomic Medicine is the lack of a molecular diagnosis for about 50% of them, and in particular the interpretation of non-coding variants that affect coding genes [3].

Several computational approaches for the interpretation of non-coding variants have been proposed [14,15]. In particular, HyperSMURF [16] a hyper-ensemble method that constitutes the machine learning core of Genomiser (a state of the art tool for the diagnosis of Mendelian diseases [18]), achieved competitive results in the detection of pathogenic single nucleotide variants (SNV) in the non-coding portions of the human genome.

Following HyperSMURF, ParSMURF [11], introduced several improvements to the original approach: it addressed most of its computational inefficiencies, while simultaneously providing a more reliable learning model thanks to its hyper-parameter auto-tuning capabilities.

In this paper we present the following novel contributions to the above-mentioned research line:

- The release of an improved version of ParSMURF, that is the highly scalable ParSMURF-NG application<fnref n="1" />, able to fully exploit the computational features of current supercomputers for solving relevant prediction problems in the context of Big Data and Genomic Medicine.

- The application of ParSMURF-NG to big omics data, with the aim of predicting pathogenic variants by employing an impressive number of features. We envision to achieve a significant advance in the prediction of Mendelian pathogenic variants by proposing novel breakthrough models.

## 2 Related Work

As briefly stated in the introduction, variant identification through the analysis of NGS data plays a pivotal role in genomic and precision medicine. Also, only in recent years the research community shifted its attention towards the understanding of the impact of variants occurring in the non-coding regions of

<footnote n="1">¹ https://github.com/AnacletoLAB/parSMURF-NG.</footnote>
the genome, as several studies showed that most of the potential pathogenic and deleterious variants may lie there [5].

In recent years, researchers proposed several tools for the identification and prioritization of relevant non-coding variants. CADD [8] was one of the first methods that relied on machine learning strategies - logistic regression in the latest version, but originally an ensemble of Support Vector Machines (SVM) - for the prediction of pathogenic and deleterious variants in the non-coding region of the genome [14]. Part of its success was due to the precise construction of the dataset used for training the classifier, as each variant is characterized by merging in a single measure a combination of different annotations. Thanks to its dataset and the employed learning strategy, CADD is one of the most commonly used tools in this research field.

Subsequently, several other tools were developed, each one employing different learning methods, from multiple kernel learning [17] to deep neural networks [13]. Moreover, some tools based on gkm-SVM [6] or deep learning [20] trained their classifier with the actual genomic sequence. All these methods, however, are challenged by the scarcity and sparsity of known pathogenic variants, being overwhelmed in number by neutral ones. From a machine learning standpoint, the classification problem emerging from this task is very challenging: imbalance-unaware algorithms correctly classify samples belonging to the majority class only - in this case, neutral variants - due to the unbalance between classes of examples to be learned.

Recently, it has been shown that methods designed to explicitly consider the imbalance between classes provide more reliable predictions. Among these methods, GWAVA [15] exploits a modified random forest classifier, and NCBoost [4] uses gradient tree boosting with a partial rebalancing strategy.

HyperSMURF [16] addresses the imbalancing problem through the cooperation of several techniques, such as:

- improving data coverage through partitioning of the training set;

- over/under-sampling of the minority/majority classes for training set rebalancing;

- improvement of the accuracy of the learners by employing an ensemble of ensemble of random forests.

As shown in [16], HyperSMURF showed excellent performance on extremely imbalanced datasets, such as those used in this field of genomic research: as a matter of fact, HyperSMURF was trained on the Genomiser dataset [18], composed of 406 manually curated pathogenic Mendelian SNVs and more than 14 million neutral SNVs, all located in non-coding regions of the human genome. Each SNV is annotated with 26 heterogeneous genomic features, including DNAse hypersensitivity, conservation scores, regulation annotations and more. In the Genomiser framework, HyperSMURF trained with this dataset is called Regulatory Mendelian Mutation (ReMM): HyperSMURF assigns a score to each SNV (ReMM score), predicting the pathogenicity of each variant. Genomiser and the ReMM scores have been widely used by the genomic diagnostics community thus motivating novel research to provide more accurate and sensitive
models and software implementations able to deal with the complexity of the
underlying prediction problem. Currently, however, advancements in this field
of research are only feasible with the use of supercomputing systems, as compu-
tational requirements are often very high. Originally, ReMM scores were eval-
uated with HyperSMURF in its first software implementation, a straightfor-
ward sequential version of the algorithm. Nevertheless, next studies showed that
HyperSMURF performance strongly depends on the careful tuning of its learning
hyper-parameters [12], thus requiring a more efficient and parallel implementa-
tion of the original algorithm to address the computational complexity of the
hyper-parameter tuning procedure.

ParSMURF [11] was developed with the goal of overcoming the limitations of HyperSMURF: its efficient parallel implementation reduces the execution times noticeably, and its hyper-parameters auto-tuning features are able to improve the quality of the learner, hence improving the predictions. Indeed, [11] shows that ParSMURF improves HyperSMURF performance when tested over the Mendelian SNV dataset by using the Cineca Marconi SKL supercomputer. Note that, by using such HPC system, ParSMURF solved in less than two weeks an hyper-parameter optimization task that, using a single core machine, would have taken 18 years to complete.

3 Methods

In this section we present ParSMURF-NG (ParSMURF - New Generation), an
improved version of ParSMURF, even more suitable for tackling Big Data prob-
lems in Genomic Medicine. For this purpose, we also present its application in
a project in the context of precision medicine.

3.1 ParSMURF-NG

ParSMURF-NG is a complete re-development of ParSMURF and, at the best of
our knowledge, is the only High Performance Computing-oriented ML software
able to process extremely imbalanced genomic dataset. The effectiveness of the
learning strategy at the core of all the implementations has been shown in [16]:
here the authors report that HyperSMURF learning approach outperformed
all the considered alternative tools when applied to problems in the context
of genomic and personalized medicine. Also, in ParSMURF-NG the underlying
algorithm and approach are the same of ParSMURF and HyperSMURF; hence,
for the sake of brevity, we will focus on the novelties introduced in ParSMURF-
NG, reminding the reader that a complete algorithm dissertation and analysis
is available on [11] and [16].

ParSMURF-NG is written in C++ for performance and optimal memory
usage reasons. By using C++, we managed to finely tune the implementation
depending on the target hardware: the code is optimized for standard x86-64
processors, but it also exploits the additional features of Intel XeonPhi CPUs, if
available. Moreover, fine tuning of the intercommunication and synchronization between processes was possible by directly managing each MPI operation.

The master-slave programming paradigm of ParSMURF represents the major reason that limited the scalability of the approach to only a few dozens of computing nodes: under this model, pool of worker processes is orchestrated by a single master. As a consequence, the workload on the master process increases with the number of workers, thus limiting the scalability of the algorithm as, no matter how many workers are available, it is limited by the processing power of the single master.

In contrast, ParSMURF-NG new design circumvents this limitation: in its master-less parallel programming model a set of processes independently work on different subsets of the dataset. Thanks to that, synchronization happens only when the final results need to be collected, so efficiency and scalability are maximized. In this approach no orchestration by a master is required: scalability is not affected (if not marginally), and hundreds of processes - even distributed over hundreds of computing nodes - can work for quickly solving a single massive task.

In ParSMURF-NG we used nested levels of parallelization to reach a high degree of efficiency: computation is distributed across nodes of an HPC facility and the MPI library leverages process intercommunication. Inside each node, parallelization is distributed by means of threading (low-level C++ 11/14 STL threads library). Lowest level of parallelization occurs at instruction level, exploiting x86-64 vector instructions - up to AVX-512 instruction set.

Access to storage and networking happens only at the beginning and the end of each run, when each process reads the data from the shared file system, and when results need to be gathered and saved to disk, respectively. Data import can substantially stress the I/O infrastructure, as it occurs concurrently to the entire the pool of MPI processes. For this reason, I/O operations (disk read and write, network access) have been updated to rely on the MPI library, forcing the usage of underlying hardware infrastructure. Tests executed on Marconi KNL with non-MPI disk operation functions (standard C++ STL), turned into a 75× increase of import time (128 MPI processes import a dataset of 12 GB in 20 s using the MPI optimized version, and 25 min using C++ STL functions).

## 3.2 Improving the Detection of Pathogenic SNV with ParSMURF-NG

As a follow-up of the studies presented in [16] and [11], we leveraged ParSMURF-NG to create a learning model for the prediction of pathogenic Mendelian variants able to outperform the current state of the art. The starting point of this investigation is the very same dataset used in those works, that is a dataset of more than 14 million SNVs, out of which only 406 are pathogenic (deleterious), with each SNV annotated using 26 genomic attributes.

We attempted to boost the generalization capabilities of the previously proposed models by leveraging the more efficient architecture of ParSMURF-NG
and by increasing the informativeness of the dataset. To this end, newly discovered pathogenic variants, with strong evidence from literature, were added and each SNV was better characterized gathering new features. For the former, we added 80 new pathogenic SNVs, which had the side effect of slightly decrease the imbalance between classes; the latter is done by extracting new epigenomic features from public available repositories, and adding this newly generated annotation to the 26 features of the original dataset [18]. Since ParSMURF-NG had been designed to cope with Big Data problems, we were able to extract and evaluate more than 600 new features.

**Feature Extraction and Generation.** Features were extracted from the International Human Epigenomic Consortium (IHEC) data portal. From there we collected up to 6 histone modification assays (H3K27ac, H3K27me3, H3K36me3, H3K4me1, H3K4me3, H3K9me3) for each of the 111 cell line considered (Assembly: hg19, Build: 2020-10). We collected assays from the following consortia: Encode, DEEP, AMED-CREST and Blueprint.

Specifically, a new annotation for each SNV is generated from each combination of these epigenomic assays and cell lines considered for this study - represented by a single cell in the IHEC data grid. If more than one experiment was present for each combination, data was aggregated considering the maximum value for each SNV. This produced more than 600 new annotations, each one representing a specific epigenomic feature for a distinct tissue/cell line.

**Feature Importance and Model Selection.** We applied different feature selection techniques to assess the informativeness of each newly generated feature. In particular, we used:

- the feature importance score returned by the random forests of ParSMURF-NG, based on the impurity index;

- Maximum Relevance Minimum Redundancy (MRMR), a multivariate method based on mutual information that returns the minimum set of features that, considered together, provides the highest informativeness;

- Spearman and Pearson correlation indexes, measured as correlation of a single feature with the label vector;

- Kruskal-Wallis rank sum and Mann-Whitney U tests between the two classes.

Spearman, Pearson, Kruskal-Wallis and Mann-Whitney were also evaluated by rebalancing the classes, i.e. by dividing the majority classes in subsets whose size was equal to the number of samples of the minority classes, evaluating each index for each subset and averaging the results. Those techniques were also used for evaluating the informativeness of the original 26 features for comparison and reference.

**Model Selection and Generation.** ParSMURF-NG was trained with the set of the most informative features selected by the above techniques. However, for
ParSMURF-NG one of the most critical task in model selection is the tuning of its hyper-parameters, as it noticeably affects the prediction capability of its trained model. Incidentally, it is also the most computing demanding task, thus specifically requiring Tier-0 HPC systems to be completed in a feasible amount of time. We chose to perform a grid search to exhaustively explore a finite space of hyper-parameters and ensure the best learning model given the selected dataset.

Finally, we train the final model using the best hyper-parameter returned by the grid selection procedure, and compared the predicting performance of the model with the one trained with the original dataset.

# 4 Experiments and Results

Before applying ParSMURF-NG to the prediction of pathogenic Mendelian SNVs, we performed a comprehensive evaluation of its scaling properties, by evaluating the speed-up and efficiency of the proposed parallel hyper-ensemble model on three HPC cluster systems using synthetic data.

<table><caption>**Table 1.** Summary of the specifications of the three HPC systems used for testing the scalability of ParSMURF-NG.</caption><thead><tr><th></th><th>Cineca Marconi<br>A2 KNL partition</th><th>HLRS Apollo<br>HAWK partition</th><th>LRZ<br>SuperMUC-NG</th></tr></thead><tbody><tr><td>CPU Family</td><td>Intel Xeon-Phi<br>Knights Landing</td><td>AMD Rome</td><td>Intel Skylake Xeon<br>Platinum 8174</td></tr><tr><td>CPU Specs</td><td>68 cores<br>4x hyper-threading<br>1.4 GHz</td><td>64 cores<br>2x hyper-threading<br>2.24 GHz</td><td>24 cores<br>2x hyper-threading<br>2.66 GHz</td></tr><tr><td>CPUs per node</td><td>1</td><td>2</td><td>2</td></tr><tr><td>Memory per node</td><td>96 GBytes</td><td>256 GBytes</td><td>96 GBytes</td></tr><tr><td>Number of nodes</td><td>3600</td><td>5632</td><td>6336</td></tr><tr><td>Interconnection</td><td>Intel Omnipath 2:1<br>Fat-tree 100GBit/s</td><td>Intel InfiniBand<br>HDR200 Enhanced<br>9D-Hypercube 200 GBit/s</td><td>Intel Omnipath 2:1<br>Fat-tree 100GBit/s</td></tr></tbody></table>

## 4.1 Scalability of parSMURF-NG

The learning performance of the proposed approach has already been assessed in [16] and [11], hence to test the correctness of the new implementation, we simply replicated the tests outlined in those works, comparing the results. Additionally, we tested the improved scalability of ParSMURF-NG across three Tier-0 High Performance Computing systems: Cineca Marconi A2 Knight Landing, HLRS Apollo HAWK and Leibniz Supercomputing Centre SuperMUC-NG. A summary of the most relevant features of each system is available in Table 1.

Assessment of ParSMURF-NG computing performance was accomplished by measuring the speed-up and efficiency in a strong scalability test setup, following the gold standard rules for the evaluation of parallel performance, as in [7]. To
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Speed-up (left), efficiency (right) of ParSMURF-NG using a fixed size problem (strong scaling) on Cineca Marconi KNL partition (blue), Hawk system (green) and Leibnitz SuperMUC-NG (purple). In both graphs, x-axis shows the number of computing nodes. Efficiency is shown in percentage. ParSMURF-NG hyper-parameters used on this test: *nParts* = 256, *fp* = 1, *ratio* = 1, *k* = 5, *nTrees* = 50, *mtry* = default. ParSMURF-NG was configured to execute a 10-fold cross-validation on the dataset. (Color figure online)</figcaption></figure>

insure comparability of the results, we executed the tests using the very same set-up - in terms of dataset, task specification and launch parameters - on the three systems. The only major difference between runs was in the thread count per node, as it was adjusted on each system so to match the number of actual physical cores on each node. Lastly, the software was compiled with Intel ICC on Marconi KNL and SuperMUC-NG, while on HLRS Hawk it was compiled with AMD AOCC, using the same set of optimizations (whenever possible).

Scalability tests were executed by launching the same prediction task several times, each time increasing the number of computing nodes from 1 to 128, following the power-of-2 law. For each run, we collected the overall execution time, including data loading. We used a synthetic dataset composed by a matrix of 100 columns and 30M rows. Out of the 30M samples, 2000 were marked as belonging to the minority class (1), and the rest as negatives (0), to mimic imbalance ratios of actual genomic datasets. Out of the 100 features, only 20 were informative of each sample class, and the rest were randomly generated.

Figure 1 shows a summary of the results, including speed-up and efficiency curves (See also Table 2). Test results show a remarkable speed-up on every system - up to 91× on Marconi KNL, 100× on HAWK and 84× on SuperMUC-NG - at a very high efficiency (always ≥60%). However, although the speed-up between the system being comparable, execution times on HAWK is far higher than the Intel-based system. This may be due to several factors, including the lack of 512 bit-wide vector instructions on the AMD, to different optimizations performed by the compilers. This required further investigations, but tests were performed in the context of "PRACE Preparatory Access" scalability assessment projects,
<table><caption>**Table 2.** Overall execution times (in seconds), speed-up and efficiency of ParSMURF-NG measured under the strong scaling experimental set-up as for Fig. 1 on the three HPC systems.</caption><thead><tr><th rowspan="2">N of MPI processes</th><th rowspan="2">Execut. time</th><th colspan="2">Speed-up</th><th rowspan="2">Execut. time</th><th colspan="2">Speed-up</th><th rowspan="2">Execut. time</th><th colspan="2">Speed-up</th></tr><tr><th>Effic.</th><th>Effic.</th><th>Effic.</th><th>Effic.</th></tr></thead><tbody><tr><td colspan="5">Marconi KNL</td><td colspan="3">Apollo HAWK</td><td colspan="2">LRZ SuperMUC-NG</td></tr><tr><td>1</td><td>37439.7</td><td>-</td><td>100%</td><td>175526</td><td>-</td><td>100%</td><td>14080.7</td><td>-</td><td>100%</td></tr><tr><td>2</td><td>18726.2</td><td>1.99</td><td>100%</td><td>92015</td><td>1.90</td><td>95%</td><td>7322.7</td><td>1.92</td><td>96%</td></tr><tr><td>4</td><td>11300</td><td>3.31</td><td>83%</td><td>55509.9</td><td>3.16</td><td>79%</td><td>4390.92</td><td>3.21</td><td>80%</td></tr><tr><td>8</td><td>7525.62</td><td>4.97</td><td>62%</td><td>36569.9</td><td>4.80</td><td>60%</td><td>2625.88</td><td>5.36</td><td>67%</td></tr><tr><td>16</td><td>3843.57</td><td>9.74</td><td>61%</td><td>18440.9</td><td>9.51</td><td>59%</td><td>1460.85</td><td>9.64</td><td>60%</td></tr><tr><td>32</td><td>1395.35</td><td>26.83</td><td>84%</td><td>6246.51</td><td>28.10</td><td>88%</td><td>531.60</td><td>26.49</td><td>83%</td></tr><tr><td>64</td><td>769.27</td><td>48.66</td><td>76%</td><td>3194.83</td><td>54.94</td><td>86%</td><td>303.93</td><td>46.33</td><td>72%</td></tr><tr><td>128</td><td>408.99</td><td>91.54</td><td>72%</td><td>1735.98</td><td>101.11</td><td>79%</td><td>167.53</td><td>84.05</td><td>66%</td></tr></tbody></table>

where limited time, core/h and resources were given. Further optimization will be done as follow-up of this work.

This experiment show that the new parallel model introduced in ParSMURF-NG provides a major improvement in efficiency and scalability. Results are also impressive, considering they are obtained by simply recompiling the code.

## 4.2 ParSMURF-NG prediction of pathogenic SNVs in Mendelian diseases

We applied all the feature selection methods listed in the Methods section to each feature extracted from the IHEC data portal - and also on the original 26. This produced 10 different feature rankings; however, for the subsequent experiments, we only considered the rankings provided by following feature selection methods:

- ParSMURF-NG random forest impurity index

- rebalanced Spearman correlation index

- rebalanced Kruskall-Wallis rank sum test

as we assessed that all the other rankings were very similar to one of these three (data not shown). The assessment had been done by clustering the distributions of the relative differences and statistically evaluating their correlation.

For each one of these rankings, we assembled five datasets containing respectively the best 5, 10, 25, 50 and 100 features selected by each method, hence generating a total of 15 datasets.

Then we performed model selection and generation: hyper-parameter optimization was done by exploring via grid selection a space of 1440 different configurations in the context of an internal 10-fold cross validation; the best combination was used for training the final model and test it on a separate test set to evaluate its generalization performance. This procedure was performed for each one of the 15 datasets, plus for the original 26 features dataset for comparison purpose. We remark that model selection by hyper-parameter tuning was the most computing intensive task and the only that was performed on the
SuperMUC-NG HPC system. Each tuning procedure was launched over 2560 computing nodes of the infrastructure and the overall execution time of each test was, on average, 14 h. Without the aid of a system like the one we were awarded to use, each model selection test would have taken approximately 5 years, if executed on a single workstation.

<table><caption>**Table 3.** Summary of the results of the 15 selected model compared to the original dataset reference model having 26 features. The result of the best model is highlighted in bold.</caption><thead><tr><th rowspan="2">Number of features</th><th colspan="2">ParSMURF-NG Random Forest</th><th colspan="2">Spearman rebalanced</th><th colspan="2">Kruskal-Wallis rebalanced</th></tr><tr><th>AUROC</th><th>AUPRC</th><th>AUROC</th><th>AUPRC</th><th>AUROC</th><th>AUPRC</th></tr></thead><tbody><tr><td>5</td><td>0.99609</td><td>0.13247</td><td>0.98103</td><td>0.24700</td><td>0.98103</td><td>0.24700</td></tr><tr><td>10</td><td>0.99384</td><td>0.03945</td><td>0.99501</td><td>0.24150</td><td><strong>0.99393</strong></td><td><strong>0.42410</strong></td></tr><tr><td>25</td><td>0.99249</td><td>0.03862</td><td>0.99135</td><td>0.14963</td><td>0.99087</td><td>0.19334</td></tr><tr><td>50</td><td>0.99118</td><td>0.04169</td><td>0.99369</td><td>0.16398</td><td>0.99273</td><td>0.23373</td></tr><tr><td>100</td><td>0.99103</td><td>0.03921</td><td>0.99345</td><td>0.14590</td><td>0.99468</td><td>0.16286</td></tr></tbody><tfoot><tr><td colspan="2">Old Dataset reference</td><td></td><td>AUROC</td><td>AUPRC</td><td></td><td></td></tr><tr><td colspan="2"></td><td></td><td>0.99363</td><td>0.35041</td><td></td><td></td></tr></tfoot></table>

We achieved significantly better results with respect to the models trained with the original reference data set that includes 26 features (Table 3). In particular the best results are obtained with 10 omics features selected through the rebalanced Kruskal-Wallis method. Note that in this highly imbalanced setting the AUPRC is far more informative than the AUROC. Overall, the results show that the combination of massive omics data, feature selection and fine tuning of the ParSMURF-NG hyper-parameters lead to significantly better results. Moreover the results show that feature selection plays a key role to improve the overall results and most of the considered epigenetic features are not significant in improving the prediction performance of the system.

# 5 Conclusions

In this work we presented ParSMURF-NG a Machine Learning approach derived from HyperSMURF, able to efficiently deal with highly imbalanced datasets of big dimension. Along with its excellent scalability performance, we showed its application in the context of Personalized and Genomic Medicine aimed to provide a more powerful model for predicting the pathogenicity or deleteriousness of non-coding SNVs.

We provide a model which improves the current state of the art in terms of generalization, and our experiments showed that the best results were obtained
by using just an handful of new features, out of the ≥ 600 considered. For this reason, as a follow-up, we plan to investigate the use of a more heterogeneous set of features, adding not only histone modifications but also a wider range of genomic and epigenomic assays.

**Acknowledgements.** This study has been developed and performed in the context of the project "ParBigMen: ParSMURF application to Big genomic and epigenomic data for the detection of pathogenic variants in Mendelian diseases". This project had been awarded by the Partnership for Advanced Computing in Europe in its 21st Call for Proposal. We acknowledge PRACE for awarding us access to SuperMUC-NG at LRZ, Germany.

**Code and Data Availability**

ParSMURF-NG is distributed as source code, and it is available at https://github.com/AnacletoLAB/parSMURF-NG

**References**

1. Abecasis, G.R., et al.: An integrated map of genetic variation from 1,092 human genomes. Nature **491**(7422), 56–65 (2012)

2. Adzhubei, I., Jordan, D.M., Sunyaev, S.R.: Predicting functional effect of human missense mutations using polyphen-2. Curr. Protoc. Hum. Genet. **76**(1), 7–20 (2013)

3. Amberger, J.S., Bocchini, C.A., Scott, A.F., Hamosh, A.: Omim.org: leveraging knowledge across phenotype-gene relationships. Nucleic Acids Res. **47**(D1), D1038–D1043 (2019)

4. Caron, B., Luo, Y., Rausell, A.: NCBoost classifies pathogenic non-coding variants in mendelian diseases through supervised learning on purifying selection signals in humans. Genome Biol. **20**(1), 32 (2019)

5. Edwards, S.L., Beesley, J., French, J.D., Dunning, A.M.: Beyond GWASs: illuminating the dark road from association to function. Am. J. Hum. Genet. **93**(5), 779–97 (2013)

6. Ghandi, M., Lee, D., Mohammad-Noori, M., Beer, M.A.: Enhanced regulatory sequence prediction using gapped k-mer features. PLoS Comput. Biol. **10**(7), e1003711 (2014)

7. Grama, A., Karypis, G., Kumar, V., Gupta, A.: Introduction to Parallel Computing, 2nd edn. Addison Wesley, Boston (2003)

8. Kircher, M., Witten, D.M., Jain, P., O’Roak, B.J., Cooper, G.M., Shendure, J.: A general framework for estimating the relative pathogenicity of human genetic variants. Nat. Genet. **46**(3), 310–315 (2014)

9. Kumar, P., Henikoff, S., Ng, P.: Predicting the effects of coding non-synonymous variants on protein function using the sift algorithm. Nat. Protoc. **4**(7), 1073–1081 (2009)

10. Leung, M.K.K., Delong, A., Alipanahi, B., Frey, B.J.: Machine learning in genomic medicine: a review of computational problems and data sets. Proc. IEEE **104**, 176–197 (2016)

11. Petrini, A., et al.: parSMURF, a high-performance computing tool for the genome-wide detection of pathogenic variants. GigaScience **9**(5), giaa052 (2020)
12. Petrini, A., et al.: Parameters tuning boosts hyperSMURF predictions of rare deleterious non-coding genetic variants. In: NETTAB 2017, Methods, Tools and Platforms for Personalized Medicine in the Big Data Era, Palermo, Italy, October 2017

13. Quang, D., Xie, X., Chen, Y.: DANN: a deep learning approach for annotating the pathogenicity of genetic variants. Bioinformatics 31(5), 761–763 (2014)

14. Rentzsch, P., Witten, D., Cooper, G., Shendure, J., Kircher, M.: CADD: predicting the deleteriousness of variants throughout the human genome. Nucleic Acids Res. 47(D1), D886–D894 (2019)

15. Ritchie, G.R.S., Dunham, I., Zeggini, E., Flicek, P.: Functional annotation of non-coding sequence variants. Nat. Methods 11(3), 294–296 (2014)

16. Schubach, M., Re, M., Robinson, P.N., Valentini, G.: Imbalance-aware machine learning for predicting rare and common disease-associated non-coding variants. Sci. Rep. 7(1), 2959 (2017)

17. Shihab, H.A., et al.: An integrative approach to predicting the functional effects of non-coding and coding sequence variation. Bioinformatics 31(10), 1536–1543 (2015)

18. Smedley, D., et al.: A whole-genome analysis framework for effective identification of pathogenic regulatory variants in mendelian disease. Am. J. Hum. Genet. 99(3), 595–606 (2016)

19. Turnbull, C., et al.: The 100 000 genomes project: bringing whole genome sequencing to the NHS. BMJ 361 (2018)

20. Zhou, J., Troyanskaya, O.G.: Predicting effects of noncoding variants with deep learning-based sequence model. Nat. Methods 12(10), 931–934 (2015)