dataesr/scientific-paragraphs-categorization huggingface dataset:
Tissue was homogenized into a single-cell suspension using a gentleMACS dissociator, red blood cells were lysed with ACK, and cells were re-suspended in 1mL of FACS buffer. CD3 + cells were depleted from samples using an EasySep Mouse T cell Isolation kit. Flow through cells were then stained with the following mAbs: ghost dye 510 (live/dead), CD3, Ly6G, CD11b, and CD115. Cells were run and collected on a FACSDiva (BD) (see Figure  S5A  for sorting gating strategy). The sorted cells were re-suspended in Trizol, RNA was purified, and libraries were prepared and sequenced using Illumina NovSeq platforms (Genewiz, Azenta Life Sciences). Raw data was trimmed and mapped, and differential gene expression was determined (Institute for Health Informatics, University of Minnesota), where additional biostatistical data analysis was also performed.
The sequencing reads were mapped to the mouse genome (GRCm38) using Bowtie aligner (bowtie2 version 2. 3. 4. 1) with local mode, -L 22 and -N 1 parameters. 71  Reads were assigned to Ensembl gene models (Mus_musculus. GRCm38. 87. gtf) with featureCounts of the Subread software package (version 1. 5. 1). 72  The reads count matrices were organized corresponding to experimental design and used for subsequent statistical analysis using the bioconductor package edgeR (version 3. 24. 3). 73, 74  The raw reads count table were normalized by using default method in the package prior to generating statistics. Multidimensional scaling was performed with edgeR using the top 500 differentially expressed genes across samples. The regression model in the edgeR package was used for statistical analysis to select the corresponding significant genes.
ACKNOWLEDGMENTS This study was supported by  National Institutes of Health  grants  AI154527  (to T. S. G. ),  GM140881  (to T. S. G. ),  GM134880  (to V. P. B. ), and  AI165553  (to J. W. W. ) and a  Veterans Administration Merit Review Award  ( BX001324  to T. S. G. ). V. P. B. is a  University of Iowa Distinguished Scholar. T. S. G. is the recipient of a  Research Career Scientist award  ( IK6BX006192 ) from the  Department of Veterans Affairs. This work was also supported in part by  NIH   P30 CA77598 , utilizing the  Masonic Cancer Center, University of Minnesota, University of Minnesota  Flow Cytometry shared resource, the  Urology Care Foundation Research Scholar Award Program , and the  AUA North Central Section  (to M. D. M. ). The graphical abstract was created with BioRender. com.
This work investigates strategies for optimizing the performance (i. e. , delay, throughput) and energy consumption of bagging ensembles to classify data streams. The experimental evaluation involved six state-of-art ensemble algorithms (OzaBag, OzaBag Adaptive Size Hoeffding Tree, Online Bagging ADWIN, Leveraging Bagging, Adaptive RandomForest, and Streaming Random Patches) applying five widely used machine learning benchmark datasets with varied characteristics on three computer platforms. Such strategies can significantly reduce energy consumption in 96% of the experimental scenarios evaluated. Despite the trade-offs, it is possible to balance them to avoid significant loss in predictive performance.
To address this gap, we have elaborated on previous research  [10, 11]  to investigate how to optimize the classification of data streams with bagging ensembles with respect to energy efficiency, time performance (i. e. , delay, and throughput), and predictive performance. The main contributions of this paper can be summarized as follows: 1. Present the premier study that applies mini-batching (proposed in  [10, 11] ) for improving the energy efficiency of bagging ensembles; 2. Identify the trade-offs between timer performance, energy efficiency, and predictive performance of bagging ensembles for the classification of online data streams; 3. Demonstrate how to achieve a good balance of these trade-offs to obtain significant gains on performance and energy efficiency, at cost of negligible loss in predictive performance on a realistic stream processing testbed with (i) six state of the art ensemble algorithms, (ii) five widely used dataset benchmarks, (iii) three computer architectures ranging from micro-to middle-end servers, and (iv) three levels of stream workloads intensities (i.
Load generation The data stream generator reads the benchmark dataset and transmits the data samples over the network to the data stream processor node at controlled transmission rates (e. g. to generate low, moderate, and high workloads for each CPU type). We used five open access 2  datasets (whose characteristics are summarized in Table  2 ) in the experiments: • The regression dataset from Ikonomovska inspired the Airlines dataset. The task is to predict whether a given flight will be delayed, given information on the scheduled departure. Thus, it has two possible classes: delayed or not delayed. • The Electricity dataset was collected from the Australian New South Wales Electricity Market, where prices are not fixed. These prices are affected by the demand and supply of the market itself and set every 5 min.
The Electricity dataset tries to identify the price changes (two possible classes: up or down) relative to a moving average of the last 24h. An essential aspect of this dataset is that it exhibits temporal dependencies. • The give me some credit (GMSC) dataset is a credit scoring dataset where the objective is to decide whether a loan should be allowed. This decision is crucial for banks since erroneous loans lead to the risk of default and unnecessary expenses on future lawsuits. The dataset contains historical data on borrowers. • The forest covertype dataset represents forest cover type for 30 x 30 m cells obtained from the US Forest Service Region 2 resource information system (RIS) data. Each class corresponds to a different cover type. The numeric attributes are all binary. Moreover, there are seven imbalanced class labels.
Performance and power consumption measurements The data logger (Fig. 2 ) collects all experimental data regarding the performance (e. g. , throughput, processing delay) and the power consumed by the data stream processor for further analysis. The sensor is implemented by a high precision power meter (Yokogawa MW-100) which periodically collects information directly from the Power Distribution Unit (PDU) and sends to the data logger. Our interest is to measure the amount of Energy (E) consumed to perform classification tasks. However, most electricity consumption monitors operate by collecting an instantaneous rate of Power (P) being supplied. Energy is the product of the average power and Time (t): E = P × t (4) Since power can vary in time, the total amount of energy consumed to perform a task is given by E = t 0 P (t) dt where t is the time to perform one task.
Experimental results and analysis As memory constraints of small devices can hinder the execution of large ensembles, our preliminary experiment aimed to find the influence of the ensemble size (i. e. , the number of classifiers in the ensemble) on the accuracy, energy consumption, and throughput. For this experiment, we used the baseline version of the algorithms while measuring energy consumption. Results shown in Fig. 3  demonstrate that accuracy (in red) remains almost constant (less than 1% loss of accuracy), the throughput (in blue) decreases, and energy consumption (in green) increases as we increase the ensemble size (x-axis). Because accuracy loss is negligible for all datasets and all algorithms, used small ensembles in the Raspberry Pi experiments. The next experiment aims at profiling the power consumption of each machine type as we vary the number of CPU cores running at full load.
Figures  5, 6 , and 7 present the results from Raspberry Pi, i5, and Xeon 4208, respectively. The chart shows results for one dataset per row, whereas the columns show results for each algorithm. All charts in the same row have the same scale. The energy consumption (in Joules per instance -JPI) on the left Y-axis, whereas the average Delay (in milliseconds) per instance appears on the right Y-axis. As a general remark, energy efficiency of each algorithm varies according to its model complexity. For instance, all three versions of OzaBag remarkably show a better energy efficiency (i. e. , smaller JPI) than the other algorithms, because OzaBag produce smaller decision trees that require fewer operations and allow a faster traversal. On the other hand, more complex algorithms presented (proportionally) higher reduction in energy consumption (JPI) when compared to its best counterpart without mini-batching.
e. , sleep periods may tend to zero), mini-batching yields power reduction because it reduces cache misses (as demonstrated in  [20] ), thus accelerating the stream processing and reducing the number of cycles and memory accesses per instance. Although mini-batching can improve energy efficiency, the delay resulting from extended and repeated sleep periods may hinder the idea of real-time processing, primarily when the rate of incoming instances is several times smaller than the mini-batch size. Such a phenomenon appears only in the B250 and B500 experiments. On the other hand, the B50 experiments present delays similar to the instance-by-instance implementations (Seq and B1) while having a better energy efficiency. Regardless of the platform and dataset used, SRP has the worst energy efficiency across all the experiments.
Conclusion Ensemble learning is a fruitful approach to improve the performance of ML models by combining several single models. Ensembles are also popular in a data stream processing context, where they achieve remarkable predictive performance. Examples of this class include algorithms such as Adaptive Random Forest, Leveraging Bag, and OzaBag. Despite their relevance, many aspects of their efficient implementation remained to be studied after their original proposals. For example, the original Adaptive Random Forest implementation included a simple multi-thread version, but it did not consider energy efficiency or mini-batches to improve the overall run time. 6 : Throughput for the algorithm LeveragingBag using the Airlines dataset. In this paper, we proposed an experimental framework to evaluate the performance and energy efficiency of the mini-batching strategy proposed in  [11]  under a realistic scenario where data streams are sent through the network at different load intensities, with six state-of-art ensemble algorithms (OzaBag, OzaBag Adaptive Size Hoeffding Tree, Online Bagging ADWIN, Leveraging Bagging, Adaptive RandomForest, and Streaming Random Patches) processing five widely used machine learning benchmark datasets with varied characteristics on three computer platforms.
8 Acknowledgements This study was financed in part by the  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior -Brasil (CAPES)  -Finance Code  001 , and  Programa Institucional de Internacionalização -CAPES-PrInt UFSCar  (Contract  88887. 373234/2019-00 ). Authors also thank  Stic   AMSUD  (project  20-STIC-09 ), and  FAPESP  (contract numbers  2018/22979-2 , and  2015/24461-2 ) for their support. Partially supported by the  TAIAO  project  CONT-64517-SSIFDS-UOW  ( Time-Evolving Data Science / Artificial Intelligence for Advanced Open Environmental Science ) funded by the  New Zealand Ministry of Business, Innovation, and Employment (MBIE). URL https://taiao. ai/.
The identified methods can be grouped into four main categories: regularized multivariate regression techniques, multi-task learning approaches, dimensionality reduction approaches, and bayesian extensions of the multivariate regression framework. Here, we compare each technique presenting its main rationale, strengths, and limitations, and provide codes and guidelines for their application to exposome data. Additionally, we apply all selected methods to a real exposome dataset from the Human Early-Life Exposome (HELIX) project, demonstrating their suitability for exposome research. Although the choice of the best method will always depend on the challenges to be faced in each application, for an exposome-like analysis we find dimensionality reduction and bayesian methods such as reduced rank regression (RRR) or multivariate bayesian shrinkage priors (MBSP) particularly useful, given their ability to deal with critical issues such as collinearity, high-dimensionality, missing data or quantification of uncertainty.
Specifically, we compare each of the techniques, presenting the main rationale behind them, their strengths, and limitations, and provide guidelines for their application to exposome data. In addition, we apply all selected methods to a real exposome dataset from the HELIX project, in which multiple health outcomes and a rich exposome characterization are available for 6 longitudinal European birth cohorts  (Maitre et al. 2018). By synthesizing the most promising methods in this field, illustrating their application to a real dataset, and providing analysis codes, this work aims to provide a valuable resource for researchers in the field of exposome analysis who seek to identify and investigate in a more comprehensive way the complex relationships between environmental exposures and multiple health outcomes.
The layout of this manuscript is as follows. In Section 2, we describe the main analytical challenges behind outcome-wide analysis with exposome data and present each of selected methods, comparing their suitability, strengths and limitations. In Section 3, we introduce additional steps that should be considered after the main outcome-wide analysis to increase the robustness of findings. Section 4 includes details on the codes and software resources generated in this paper. In Section 5, we present the application of all selected methods to a real exposome dataset and detail the analysis findings. Finally, we conclude with a general discussion in Section 6.
• GFLasso: The innovation in GFLasso is that, in addition to using L1 penalty, which induces overall sparsity, it employs another penalty called 'fusion penalty' (γ) that fuses regression coefficients across correlated phenotypes. This penalty is added to the objective function penalizing the difference among the estimated coefficients for the effect of the ith exposure on two correlated outcomes. A larger value for the γ□penalty will lead to a greater fusion effect, and therefore, a greater similarity among the coefficients of the effect of the ith exposure on correlated outcomes. GFLasso, therefore, lays on the drawing of the graph dependency structure underlying the outcome variables in the dataset, which can be quantified by an outcome correlation matrix. Starting from this correlation matrix, two types of fusion penalties are proposed in GFLasso, depending on whether they use an unweighted or weighted connectivity phenotype graph as a guide.
When using this method, the user will need to define the penalty values λ and γ, which are usually selected by cross-validation. As a result, GFLasso favors the selection of predictors with effects on multiple outcomes, flattening the coefficients of each exposure on correlated outcomes. An open-source package called "gflasso" is available in R through GitHub 1  for the implementation of this approach. We implemented a patch modification to the main functions of the package in order to allow the method to correct for the effect of confounders. The corrected codes are available at the supplementary GitHub repository. 2  Although the method was developed to work with continuous outcomes only, it could be extended to the case of binary phenotypes and logistic regression in the future. • The GroupRemMap Method: The main idea behind GroupRemMap is quite similar to the solution proposed in GFLasso.
The combination of the L1 penalty and the bridge penalty on L2 norm of grouped predictors is presented as the GroupRemMap penalty. As a result, GroupRemmap favors selection of predictors with effect in majority of outcomes, flattening the coefficients of exposures belonging to groups. For that, GroupRemMap relies on the existence of expert knowledge for grouping exposures (e. g. , according to exposome domains, families, biological pathways involved in their health effects, etc. ). The selection of tuning parameters is again done through K-fold cross-validation. An R package called "group-RemMap" is available through CRAN. As it happens with GFLasso, GroupRemMap is designed to work only on continuous outcomes.
Finally, MBSP is able to account for the effect of confounders, which can be forced into the model. MBSP is implemented in a comprehensive R package called MBSP. For applying the model, the user needs to set which exact type of GL shrinkage priors to use, and the additional parameter τ (which controls the amount of global shrinkage). The selection of which GL shrinkage priors to use will shape the prior beliefs about the importance and distribution of the different coefficients for exposure-outcome associations, thereby influencing the variable selection and the estimation procedure. Nevertheless, the choice among the three priors is not a trivial question, especially if the researcher has no strong preconceived beliefs on the underlying associations. In general, according to previous simulation studies  (Kundu et al.
2021) , the horseshoe prior has a reasonably good performance across a variety of empirical experiments and situations and is more computationally efficient than other approaches. The NEG prior could be of preference if the researcher is specifically interested in capturing both common effects shared across multiple outcomes and individual effects specific to each outcome. The MBSP R package incorporates a strategy to help researchers identifying the most appropriate prior for each situation. For that, the user can try different priors and finally determine the optimal model by the inspection of goodness of fit criteria such as the Deviance Information Criterion (DIC) or the Watanabe-Akaike information criterion (WAIC). For that task, WAIC may be preferred since it averages over the posterior probabilities and it does not require the them to be approximately normal.
Overview of the properties of the different methods A general overview of all selected methods and their solutions for outcome-wide analysis in exposome research can be found in Table  1. These techniques differ a bit conceptually, but they also differ in the situations they are able to accommodate. Most of the techniques are specially designed to deal with high-dimensional datasets. Among all methods, MBSP was specially designed to work in high or ultra-high dimensional settings, being able to retrieve a small number of exposures affecting the multiple outcomes even when input data contain hundreds of thousands of variables, favoring interpretability and summarization of the problem. Although most of the techniques are designed to work only on continuous outcomes, an option for mixedtype outcomes has been proposed (e.
g. , mRRR). Regarding missing data in the outcomes, again only the mRRR is able to deal with incomplete records, which suggests this functionality is a key aspect to be considered by future methodological developments. Since the methods GFLasso, MTL_L21, sRRR and MBSP did not incorporate a solution for accounting for the effect of confounders, whenever possible, we implemented this functionality. The new versions of these functions are available either in their latest CRAN releases (for sRRR and MBSP) or in our GitHub repository 3  (for GFLasso). The only method in which it was not possible to add this implementation was the MTL approach. In this case, therefore, it is advisable to conduct alternative ways of confounding treatment (such as residualization of predictors/outcomes on confounders (partialling-out approach))  (Demissie and Cupples 2011).
The methods deriving a less sparse model are GFlasso and mRRR; the ones with medium level are MTL and sRRR; while the rest are able to retrieve a highly sparse model (GroupRemMap, MBSP), with no need of postcuration of findings. Differences can also be found among methods in the type of relationships they look at. While GFLasso, GroupRemMap and MBSP allow intra-exposure sparsity (an exposure not necessarily need to affect all outcomes under study), the rest of the methods rely on the assumption all outcomes need to share the same set of associated exposures. Finally, it is important to state that, previous to data analysis, for all these methods it is critical to scale all predictor and outcomes to the same scale. This way, we avoid the bias of methods toward the selection variables with highest variance and wide domains, further allowing the inter-outcome comparison of coefficients.
2) Selection by row-wise norm: This approach tends to filter out exposures according to their estimated overall effect on all outcomes under the study, thereby promoting the selection of "master-regulators". This approach consists of calculating the Euclidean norm for each p th row of the coefficient matrix. The Euclidean norm (also called the vector magnitude, Euclidean length, or 2-norm) of a vector v with Q elements is defined by ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ ∑ Q k=1 |β k | 2 √. A higher value will indicate a higher overall effect of the p th exposure on the outcomes under study. Once this value is calculated for all exposures, one could select a number of exposures presenting the top values for the norm. The choice of the final set of selected exposures will depend on the initial dimensionality of the dataset and the desired level of interpretability.
Application of selected methods on a real exposome dataset: the helix study In this section, we apply all selected methods to a real exposome dataset demonstrating their suitability and giving recommendations for implementation. The research dataset employed here derives from the HELIX project (Human Early-Life Exposome). The HELIX project gathers data from 6 longitudinal European birth cohorts with the aim of evaluating the effect of environmental risk factors on mothers' and children's health. HELIX cohorts include the BIB (Born in Bradford) (United Kingdom), EDEN ( Étude des Déterminants pré et postnatals du développement et de la santé de l'ENfant) (France), INMA (INfancia y Medio Ambiente) (Spain), KANC (Kaunus Cohort) (Lithuania), MoBa (Norwegian Mother and Child Cohort Study) (Norway), and Rhea (Mother-Child Cohort in Crete) (Greece).
General details of the study design can be found elsewhere  (Maitre et al. 2018; Warembourg et al. 2019). Here, we focus on a HELIX subcohort of 881 children according to the following criteria of eligibility: 1) age 6 to 11 years at the moment of outcome evaluation; 2) complete address history; 3) no serious health problems that may affect the clinical testing or the child safety; and 4) having complete data on all health outcomes. In the 881 children, many environmental exposures were evaluated to define childhood exposome (age 6 to 11 years). Collected exposures comprise three main parts of the exposome: outdoor exposures, chemical exposures, and lifestyle and social factors. All variables incorporated in the dataset have been appropriately pre-processed previous to analysis (normalized and scaled, outliers removed, and missing values imputed) as described elsewhere  (Maitre et al.