dataesr/scientific-paragraphs-categorization huggingface dataset:
More about subsampling biases and tree depth In the previous section, simulations were conducted with representative subsamples and a maximum number of leaves equal to 500 for every tree in every forest. The present section pushes the analysis a bit further by specifically investigating the influence of these two features on the results. All simulations were performed with the same dataset and the same computing environment than in the previous section. Finally, the different parameters for the RF methods were fixed in light of the previous section: blbRF and dacRF were learned respectively with K = 10 and q = 50 and with K = 10, q = 100, whereas moonRF and sampRF were learned with total sampling fraction equal to 0. 1%. As explained in Section 3. 2, dacRF can be influenced by the lack of representativity of the data sent to the different chunks.
In this section, we evaluate the influence of such cases in two different directions. We have considered the non representativity of observations in the different chunks/sub-samples, firstly according to Y values using the unbalanced dataset and secondly, according to X values using the x-biases dataset (see Section 4. 1 for a description of these two datasets). For dacRF, this simulation corresponds to the case where the subforests built from the different chunks are very heterogeneous. This issue has been discussed in Section 3. 2. 3 and we will show that it indeed has a strong impact in practice. Results associated to the unbalanced case are presented in Figure  13. In The first fact worth noting in these results is again that errOOB and er-rTest are always very close, whereas BDerrForest is more and more biased as p decreases.
Moreover, BDerrForest appears to be a very bad estimation of the prediction error of the forest. Finally, the issue of tree depth is investigated more closely. As mentioned above, the maximum number of leaves was set to 500 in order to get comparable tree complexities. However homogeneity (in terms of classes) of leaves differs when a tree is built on the entire dataset or on a fraction of it. To illustrate this, the mean Gini index (over all leaves of a tree and over 100 trees) was computed (it is defined by 2p(1p), with p the proportion of observations of class 1 in a leaf). Results are reported in Table  6. For sampling fractions equal to 0. 1% or 1%, tree leaves are pure (i. e. , contain observations from only one class). But for sampling fractions equal to 100% and 10%, the heterogeneity of the leaves is more important.
Online random forest This section is dedicated to simulations with online RF. The simulations were performed with the method described in  [20]  which is available at https: //github. com/david-matheson/rftk (onRF). The method is implemented in python. Thus computational time cannot be directly compared to the computational described in the two previous sections (because of the programming language side effect). Similarly, the input hyperparameters of randomForest function in the R package randomForest are not exactly the same than the ones proposed in onRF: for instance, in the R package, the complexity of each tree is controlled by setting the maximum number of leaves in a tree whereas in onRF, it is controlled by setting the maximum depth of the trees. Additionally, the two tools are very differently documented: every function and option in the R package are described in details in the documentation whereas RFTK is not provided with a documentation.
However, the meaning of the different options and outputs of the library can be guessed from their names in most cases. When relevant, we discuss the comparison between the standard approaches tested in the two previous sections and the online RF tested in the current version but the reader must be aware that some of the differences might come directly from the method itself (standard or online), whereas others come from the implementation and programming languages and that it is impossible to distinguish between the two in most cases. The simulations in this section were performed on the datasets described in Section 4. 1. The training dataset (randomly ordered) took approximately 9 minutes to be loaded with the function loadtxt of the python library numpy, which is about 9 times larger than the time needed by the R package readr to perform the same task.
In the sequel, results about this dataset will be referred as standard. Moreover, simulations were also performed to study the effect of sampling (subsamples drawn at random with a sampling fraction in {0. 01, 0. 1, 1, 10}%) or of biased order of arrival of the observations (with the datasets unbalanced, with p = 0. 01, and x-biases with 15 parts). For xbiases the number of parts was chosen differently than in the Section 4. 2 (for dacRF) because only 2 parts would have led to a quite extreme situation for onRF, in which all data coming from submodel 1 are presented first, before all data coming from submodel 2 are presented. We have thus chosen a more moderate situations in which data from the two submodels are presented by blocks, alternating submodel 1 and submodel 2 blocks. Note that both simulation settings are similar, since dacRF processes the different (biased in X) blocks in parallel.
The forests were trained with a number of trees equal to 50 or 100 (for approximately 500 trees, the RAM capacity of the server was overloaded) and with a control of the complexity of the trees by their maximum depth which was varied in {5, 10, 15, 50}. RFTK does not provide the online approximation of OOB error so the accuracy was assessed by the computation of the prediction error on the same test dataset used in the previous two sections. The result shows that, contrary to the dacRF case, x-biases almost do not affect the accuracy of the results, even if the classifier always has a better accuracy when data are presented in random order. On the contrary, unbalanced has a strong negative impact on accuracy of the classifier. Finally, for the best case scenario (standard), the accuracy of onRF is not much affected by the number of trees in the forest but the accuracy tends to get even worse when increasing the number of trees in the worst case scenario (unbalanced).
Airline dataset In the present section, similar experiments are performed with a real world dataset related to flight delays. The data were first processed in  [6]  to illustrate the use of the R packages for Big Data computing bigmemory and foreach  [38]. In  [6] , the data were mainly used for description purpose (e. g. , quantile calculation), whereas we will be using it for prediction. More precisely, five variables based on the original variables included in the data set were used to predict if the flight was likely to arrive on time or with a delay larger than 15 minutes (flights with a delay smaller than 15 minutes were considered on time). The predictors were: the moment of the flight (two levels: night/daytime), the moment of the week (two levels: weekday/week-end), the departure time (in minutes, numeric) and distance (numeric).
The dataset used to make the simulations contained 120,748,239 observations (observations with missing values were filtered out) and had a size equal to 3. The same method than the one described in Section 4. 2 were compared: • a standard RF, seqRF, was computed sequentially. It contained 100 trees. The RF took 16 hours to be obtained and its OOB error was equal to 18. 32%; • sampRF was trained with a subsample of the total data (1% of all the observations were sampled at random without replacement). These RF were trained in parallel with 15 cores, each core building 7 trees from bootstrap samples coming from the common subsample (the final RF hence contained 105 trees); • a blbRF was also trained using K = 15 subsamples, each containing about 454,272 observations (about 0. 4% of the size of the total data set).
01%) of them. In terms of compromise between computational time and accuracy, using a small subsample is clearly the best strategy, provided that the user is able to obtain a representative subsample at a low computational cost. Also, contrary to what happened in the example described in Section 4. 3, BDerrForest is always a good approximation of errForest. An explanation of this result might be that for Airline dataset, prediction accuracy is quite poor and this might be due to explanatory variables that are not informative enough. Hence differences between BDerrForest and errForest may be hidden by the fact that the two estimations of the prediction error are quite high. In addition, the impact of the representativity, with respect to the target variable, of the samples on which the RF were trained was assessed: instead of using a representative (hence unbalanced) sample from the total dataset, a balanced subsample (for 50% of delayed flights and 50% of on time flights) was obtained and used as the input data to train the random forest.
Its size was equal to 10% of the total dataset size. This approach obtained an errForest equal to 33. 34% (and BDerrForest was equal to 39. 15%), which is strongly deteriorated compared to the previous misclassification rates. In this example, the representativity of the observations contained in the subsample strongly impacts the estimated model. The model with balanced data has a better ability to detect late flights and favors the sensitivity over the specificity.
Conclusions This paper aims at extending standard Random Forests in order to process Big Data. Indeed RF is an interesting example among the widely used statistical methods in machine learning since it already offers several ways to deal with massive data in offline or online contexts. Focusing on classification problems, we reviewed some of the available proposals about RF in parallel environments and online RF. We formulated various remarks for RF in the Big Data context, including approximations of out-of-bag type errors. We experimented on two massive datasets (15 and 120 millions of observations), a simulated one and real world data, five variants involving subsampling, adaptations of bootstrap to Big Data, a divide-and-conquer approach and online updates. Among the variants of RF that we tested, the fastest were sampRF with a small sampling fraction and blbRF.
This is a crucial difference with RF scheme for which the different training samples are obtained by independent bootstraps. But the aggregation part of the algorithm is interesting here: instead of taking the majority vote of the trees predictions as in the RF context, a weighted combination of trees is considered. The unnormalized weight of the tree t is simply α t = 1/2 ln( t /(1t )) where t is the misclassification error computed on the whole training sample L. This could be adapted by considering weighted forests using weights of such form, evaluated on a same (small) subset of observations supposed to be representative of the whole dataset.
1 and randomly ordered. With the readr package, loading this dataset took approximately one minute. As a baseline for comparison, a standard RF with 100 trees was trained in a sequential way with the R package randomForest. This package allows to control the complexity of the trees in the forest by setting a maximum number of terminal nodes (leaves). By default, fully developed trees are grown, with unlimited number of leaves, until all leaves are pure (i. e. composed of observations all belonging to the same class). Considering the very large number of observations, the number of leaves was limited to 500 in our experiments. The training of this forest took approximately 7 hours and the resulting OOB error was equal to 4. 564e -3 and has served as a baseline for the other experiments.
A large array of Ge detectors installed at ILL, around a 235 U target irradiated with cold neutrons, (EXILL) allowed measurement of prompt gamma-ray cascades occurring in fission fragments with an unambiguous determination of fragments. Here we present preliminary results of a systematic comparison between experimental γ -ray intensities and those obtained from the Monte-Carlo simulation code FIFRELIN, which is dedicated to the de-excitation of fission fragments. Major γ -ray intensities in the 142 Ba and 92 Kr fission products, extracted from EXILL data, were compared to FIFRELIN, as well as to reported values (when available) obtained with EUROGAM2 in the spontaneous fission of 248 Cm. The evolution of γ -ray intensities in 92 Kr versus the complementary partner in fission (i. e. versus the total number of evaporated neutrons by the fission pair) was then extracted and compared to FIFRELIN.
Experimental details A large array of sixteen HPGe detectors was placed at the end of the PF1b cold neutron guide, which has the highest neutron flux at the ILL. It included eight EXOGAM clovers, two smaller clovers from LOHENGRIN and six large efficiency detectors from GASP. The acronym EXILL refers to EXOGAM@ILL. All detectors, except the two LOHENGRIN clovers, were surrounded by a BGO active shielding to reduce Compton background. Overall, the array had a total efficiency of 6. 2% for γ rays of 1 MeV. The neutron beam was reduced by collimators to about 12 mm diameter spot at the target position. Its capture flux there was around 10 8 n/s/cm 2. The fission rate in the targets was estimated to around 10 5 fissions/s. Data were collected in a triggerless mode using a 100 MHz digital acquisition system for over 20 days with a 235 U target and 15 days with a 241 Pu target.
The goal is to study transitions between discrete levels, occurring in the deexcitation of the fragments, after the neutron evaporation process. A first step was to compare these transition intensities to the ones predicted by the current version of FIFRELIN simulation code. Then we look for any systematic correlations between the cascade in a fragment and its fission partner. At this stage of work, there was no adjustment on FIFRELIN parameters to match our results. The analysis was performed on the γ γ γ cube in a semiautomatic way using a home-made software developed on ROOT. To construct e. g. the cascade in 142 Ba, we searched for events in coincidence with a strong transition (at the energy E 1 ) in one of its fission partner (e. g. 92 Kr) and, in coincidence with a strong transition (at the energy E 2 ) in 142 Ba.
Preliminary results 142 Ba is a good study case. It is well produced in the thermal fission of 236 U (fission yield = 2. 8%) and its level scheme is well known. According to RIPL-3 database  [6] , the level scheme is complete up to 1848. 4 keV and 38 levels are known up to 5284 keV. In addition, the structure of this nucleus was studied by Urban and coworkers  [7]  in the spontaneous fission of 248 Cm. In that last case however, the exact gating conditions, and especially the fission partner used to build the level scheme, were not reported. Figure  1  shows the level scheme and γ -ray cascade simulated by FIFRELIN in 142 Ba, when this fission product (i. e. after neutron evaporation) is obtained in coincidence with 92 Kr (after neutron evaporation as well), compared to the experimental cascade observed within the EXILL data in the same conditions.
g. an inversion of the 1034 keV and 1297 keV intensities. The present FIFRELIN simulation uses the 2015 updated version of RIPL-3, which includes many new values or changes of level spin and parity for the 92 Kr. The partial experimental cascade obtained from EXILL data seems consistent with the results of Rzaca-Urban et al. (Fig. 4  in  [9] ) measured in the spontaneous fission of 248 Cm. However the values of the transition intensities were not reported in  [9] ). The EXILL data will allow systematic studies of the cascades to be performed. One question we are studying is whether and how the deexcitation cascade in a fragment evolves when its fragment partner in fission changes. FIFRELIN simulations were done to test such systematic trends. Figure  3  shows the (simulated) intensity of main transitions in 142 Ba versus possible Kr partners.
Other methods exist which do not use DFT but simulate interacting classical ions and electrons incorporating some quantum mechanical effects through the interaction pseudo-potential between ions and electrons  [3, 4]. We have also chosen to simulate the ions and electrons as classical particles and to incorporate a minimum of quantum information through a regularized potential allowing to model collisional ionization and recombination processes. The BinGo-TCP code has been designed to deal with neutral mixtures composed of ions of the same atom with different charge states and electrons  [5, 6]. Within the limits of classical mechanics, all charge-charge interactions are accounted for in the particle motion. In this article, after a brief recall of the modeling basis, the importance of such a model to get a good understanding in correlation effects will be emphasized through a study of the ionization potential depression (IPD) and some other statistical properties in dense plasmas.
Modeling A classical multi-component plasma molecular dynamics (MD) code, the BinGo-TCP code, has been developed to follow the evolution of plasmas involving ions of various charge states and electrons. The overall structure of the BinGo-TCP code is consistent with that of a standard code of classical MD simulation. Non-relativistic point-like charged particles are simulated in a cubic box. The system inside the simulation box is neutral. The periodic boundary conditions are used together with the minimum image convention and the Newton's equations are solved using a Velocity-Verlet algorithm. A molecular dynamics simulation requires the definition of the potential function that governs interactions between the particles inside the simulation box.
The results are shown in Figure  4  with the results obtained with two theoretical IPD models, the Stewart-Pyatt (SP)  [15]  and the Ecker-Kröll (EK)  [16]  model, which apply across a wide range of densities. In those conditions of plasma, the electron Figure  4. The ionization potential depression obtained with the BinGo-TCP code (down triangles) for an aluminum plasma at T e =T i =50eV and ρ = 2. 7g. cm -3 , compared with the IPD calculated with the SP (dot line) and EK (dot-dash line) models for the same conditions. The high density/low temperature limit of the SP model is also plotted (full line) for comparison. coupling plasma parameter is Γ e = 0. 35 and the measured mean charge is Z b = 7. 4 leading to a ion coupling plasma parameter, Γ i = 9. 96. The effects of the charge correlations on the lowering of the necessary energy to free an electron are very strong and cannot be ignored.
Simulation design For each round of simulation, a sample of n = 58 profiles with 22 chromosomes each is generated, with a number of genes per chromosome identical to the one observed in the reference dataset (see description in Section 5. 1). Each chromosome is split into K chr regions, according to the segmentation obtained in  Stransky et al. (2006). Each   2006 ) study. As it can be seen, this length varies substantially from a H 1 region to another. For ρ 1 , different values were considered (ranging from 0. 3 to 0. 9), all H 1 regions of all chromosomes sharing this same ρ 1 coefficient. For ρ 0 , two cases are considered : • Scenario 1 (Easy case): ρ 0 is the same for all H 0 regions of all chromosomes (3 different values considered: 0. 08, 0. 18, 0. 28). • Scenario 2 (Realistic case): ρ 0 is identical for all H 0 regions within a chromosome, but varies from a chromosome to another.
The specific value chosen for a given chromosome is the one obtained for this same chromosome on the reference dataset. The distribution of ρ 0 across chromosomes on the reference dataset is given in Figure  2. Additionally, correlations between genes from different regions were observed on the reference dataset. Thus, a similar extra block diagonal correlation pattern was generated in the simulations, with a level of background correlation between blocks similar to the one within blocks. Lastly, for each combination (ρ 0 , ρ 1 ) the simulation was replicated 20 times.
Bladder cancer data In this section, we apply SegCorr on the dataset described in Section 5. 1. It is now well known that copy number variation (CNV) impacts gene expression  (Sebat et al. , 2004). Here our goal is to detect regions where the correlation is not due to CNV. Therefore we correct the expression signal for CNV variation according to the strategy described in Sections 2. 1 and 5. 3. The effect of this correction is investigated in Section 5. 4. Lastly, Section 5. 5 illustrates the biological results obtained after correction for CNV.
Data presentation The dataset consists of n = 58 bladder tumors. Gene expression were measured using exon 1. 0 Affymetrix arrays and RMA normalisation  Irizarry et al. (2003)  was applied. The number of genes per chromosome ranges from a 293 to 2192 (with average 950). Additionally genomic and methylation data were collected for the same tumors. Genomic data were obtained with Illumina Human610-Quad SNP arrays and methylation data with Illumina Human methylation 450k arrays. For the latter, the normalization procedure proposed by  Teschendorff et al. (2013)  was used. The combination of probe positions was made according to the custom BrainArray EntrezGene annotation  (Dai et al. , 2005).
Procedure for CNV correction To correct the expression signal from CNV, one first needs to detect the CNV regions from the SNP signal. To this aim, we consider the segmentation method proposed by  Picard et al. (2011)  implemented in the R package cghseg. Denote SN P it the SNP signal of patient i at position t, the model writes SN P it = µ ik + E it if t ∈ I i k = [t i k-1 + 1, t i k ]. (11) where the E it are i. i. d centered Gaussian with variance σ 2. The method estimates the number of regions, the boundaries of the regions, denoted ti k and the signal mean within each region k in patient i, denoted μik. We then use the regression model (2) to make the correction where x ij is the mean μik obtained previously if the SNP position ti k corresponds to gene j of the expression signal in patient i.
The detected regions should be further investigated to better understand the underlying mechanism. While the expression data used here were acquired using the microarray technology, any other technology, including RNA-seq, can be used as well. In our analysis, we have assumed stretches of correlated contiguous neighboring genes. This is obviously a simplification. Within a correlated region, a gene (or a few genes) could exhibit a weak or even a negative correlation with the other genes. This could occur for different reasons: the gene can be not expressed; alternatively, the gene could be non affected by the regulation process that impacts the other ones; finally, the gene could be impacted in a opposite way compared with the other ones. Note that genes that exhibit no expression or no variation in the dataset can be detected and could be discarded before applying the analysis.
Results and discussion 3. 1 The MRF method allows for the network analysis of multivariate RNAi data and finds pertinent functional/GO enrichment We consider data from an RNAi screen aiming to identify genes implicated in DNA repair after induction of oxidative DNA damage  (Guyon et al. , 2015). Briefly, in the context of this screen, HeLa cells specifically engineered to express OGG1 [the initiating enzyme of the base excision repair (BER) of oxidized guanine] fused to a Green Fluorescent Protein (GFP), were systematically transfected with siRNAs targeting the 'druggable' subset of genes from human genome (3 siRNAs/gene; 7218 target genes). Three days posttransfection, 8-oxo-7,8-dihydro-guanine (8-oxoG) DNA base lesions were induced by cell exposure to potassium bromate (KBrO3) and the recruitment of chromatin-bound OGG1-GFP was quantified (subsequently the fluorescence intensity is averaged in each nucleus) by computer-assisted imaging performed on an automated epifluorescence microscope (Operetta, Perkin Elmer).