SelectBoost, A General Algorithm to Enhance the Performance of Variable Selection Methods in Biological Network Reverse Engineering
Creators: name: Maumy, Myriam
name: Bertrand, Frederic
Publisher: name: F1000 Research Limited
Year: 2022

description: The SelectBoost CRAN R package implements SelectBoost: a general algorithm to enhance the performance of variable selection methods https://doi.org/10.1093/bioinformatics/btaa855, F. Bertrand, I. Aouadi, N. Jung, R. Carapito, L. Vallat, S. Bahram, M. Maumy-Bertrand, Bioinformatics, 37(5), 2021.<br> <br> Technological innovations make it possible to measure large amounts of data in a single observation. As a consequence, problems in which the number P of variables is larger than the number N of observations have become common. As reviewed by Fan and Li (Fan and Li, 2006), such situations arise in many fields from fundamental sciences to social science, and variable selection is required to tackle these issues. For example, in biology/medicine, thousands of messenger RNA (mRNA) expressions (Lipshutz et al., 1999) may be potential predictors of some disease. Moreover, in such studies, the correlation between variables is often very strong (Segal et al., 2003), and variable selection methods often fail to make the distinction between the informative variables and those which are not. Similarly, inference of gene regulatory networks (GRNs) from perturbation data can enhance the insights of a biological system, (Morgan et al., 2019). In this package, we propose a general algorithm that enhances model selection in correlated variables.<br> <br> Variable selection has become one of the major challenges in statistics. This is due to both the growth of big data and technological innovations, that make it possible to measure large amounts of data in a single observation. As a consequence, problems in which the number P of variables is larger than the number N of observations have become common. Although many methods have been proposed in the literature their performance in terms of recall and precision are limited in a context where the number of variables by far exceeds the number of observations or in a highly correlated setting.<br> <br> The main idea of our algorithm is to consider that any observed value of a group of linearly correlated variables of the X matrix is the independent realization of a given random function. This common random function is then used to perturb the observed values of the relevant correlated variables.<br> <br> Strictly speaking, the use of noise to determine the informative variables is not a new idea. For example, it has been shown that adding random pseudo-variables decreases over-fitting (Wu et al., 2007). In the case where P &gt; N the pseudo-variables are generated either with a standard normal distribution N(0; 1) or by using permutations on the matrix X (Wu et al., 2007). Another approach consists of adding noise to the response variable and leads to similar results (Luo et al., 2006). The rationale of this method is based on the work of Cook and Stefanski (Cook and Stefanski, 1994), which introduces the simulation-based algorithm SIMEX (Cook and Stefanski, 1994). Adding noise to the matrix X has already been used in the context of microarrays (Chen et al., 2007). Simsel (Eklund and Zwanzig, 2012) is an algorithm that both adds noise to variables and uses random pseudo-variables. One new and inspiring approach is stability selection (Meinshausen and Buhlmann, 2010) in which the variable selection method is applied to sub-samples, and informative variables are defined as variables that have a high probability of being selected. Bootstrapping has been applied to the lasso on both the response variable and the matrix X with better results in the former case (Bach et al., 2008). A random lasso, in which variables are weighted with random weights, has also been introduced (Wang et al., 2011).<br> <br> In this package, following the idea of using simulation to enhance the variable selection methods, we propose the SelectBoost algorithm. Unlike other algorithms reviewed above, it takes into account the correlation structure of the data. Furthermore, our algorithm is motivated by the fact that in the case of non-regularized variable selection methods, if a group contains variables that are highly correlated together, one of them will be chosen with precision.<br> <br> This package implements a new general algorithm that improves the precision of any existing variable selection method. This algorithm is based on highly intensive simulations and takes into account the correlation structure of the data. Our algorithm can either produce a confidence index for variable selection or it can be used in an experimental design planning perspective.<br> <br> In order to benchmark the performance of the SelectBoost algorithm in a wide range of settings, we generated data following several patterns with continuous or binary outcomes as well as continuous or binary predictors. We decided to use dataset featuring binary predictors to test the performance of the SelectBoost algorithm in a non-Gaussian predictors setting. The SelectBoost algorithm was applied to three real datasets as well.