dataesr/scientific-paragraphs-categorization huggingface dataset:
The sample size is defined such that performance is within a given interval of the optimal. Application of the method to low-dimensional setting has not been evaluated by the authors and such an evaluation was out of the scope of our study. Regarding the sample size estimation of the test dataset, we propose to rely on the method devised by Shoenfeld for clinical trials  [16]. With the final objective to validate a two-groups prognostic index, the approach can be adapted to test the null hypothesis of a death hazard ratio equal to one between the two prognostic groups (instead of the two arms of a clinical trial). Following Schoenfeld, the number of events required to test the null hypothesis H 0 : = 1, under the alternative hypothesis H 1 : = 0 , with a significance level of α and the power β is: (z β + z 1-α ) 2 p A p B (ln( 0 )) 2 with z 1-α and z β the 1-α and β percentiles of the Normal distribution, 0 the hazard ratio for the good vs.
In particular, relative quantifications can be derived with respect to reference or housekeeping genes selected for their stable expression across samples  [18]. Second, maximizing the number of samples (instead of maximizing the number of genes) per PCR plate is recommended to favor the comparison of gene expression across patients  [18]. Ideally, this would allow us to measure only one candidate biomarker at a time within a PCR plate. Finally, we recommend that training and test datasets are acquired in a row, while processing training and test samples in different PCR batches, so that the technical/biological variability between the two datasets is not larger than the technical/biological variability within the training dataset.
Resampling strategy Resampling methods allow us to mimic the validation step within the training dataset only, in order to select models with good generalization performance. N-times k-fold cross-validation is a standard resampling method used to estimate model performance on the training dataset. Briefly, for each of the N repetitions, the training dataset is split into two parts: the training-fold and the test-fold dataset. (Please refer to the discussion for the rational for the choice of cross validation versus bootstrap as a resampling strategy in this context). The training-fold dataset includes (k -1)/k of the training dataset patients and the test-fold dataset includes the remaining 1/k of the training dataset patients. The procedure is repeated k times so that all of the patients are included once in a test-fold dataset.
Each time, the training-fold dataset is used to build the prognostic model (including data-preprocessing and normalization, variable selection and the estimations of the model parameters) and to estimate its apparent performance; then the corresponding test-fold dataset is used for (cross)-validated performance estimation. The apparent performance estimated on the training-fold dataset is expected to be overly-optimistic because the trainingfold dataset is both used to build the model and estimate its performance. The difference between the apparent and the validated performance corresponds to the model optimism. The more complex a model is, the larger the optimism is expected to be. Moreover, resampling methods should properly capture the intrinsic technical variability of the biomarker assay.
If the technical variability between the training and the test datasets is larger than the technical variability between the training samples, the performance estimated on the training samples is likely to be optimistic. The difference between the estimation of the performance in the training and the test datasets is called the "validation surprise" and reflects the lack of generalization of the model. This problem may happen with RT-qPCR assays or any assay that processes samples by batch. Hence, the within batch variability is likely to be smaller than the between batch variability. As stated in the introduction, we compared two resampling strategies: strategy A and B. Strategy A is a stratified cross-validation strategy where test-fold samples are randomly sampled across batches so that the proportion of events is the same in each fold.
Contrary to strategy A, strategy B tries to capture the technical and biological variability between the training and test datasets where entire PCR batches are randomly sampled and corresponding patients are set apart for test-fold data. This strategy does not ensure that the proportion of events is the same in each fold, except if the batches were designed to have the same proportion of events. With resampling strategy B, the variability between the training and the test fold datasets is expected to mimic the variability between the training and the test datasets. By doing so, cross-validated performance should better capture the between dataset variability with a lower risk of observing a significant validation surprise.
Model selection In practice, a large number of models need to be evaluated. Thus, small differences in cross-validated performances of the models have to be interpreted with caution. In order to narrow down the number of candidate models, we suggest a two-step model selection strategy. First, one needs to select a short-list of models with equally good performance and second apply additional model selection criteria to retain, ideally, a single model for the evaluation on the independent test dataset. The proposed additional selection criteria are: i) the stability of the selected variables across cross-validation repetitions, ii) the clinical interest in the selected variables, iii) the model optimism and iv) the impact of the omitted variables on the model performance.
Stability The stability of the selected variables should be evaluated for automatic variable selection methods (e. g. backward elimination, MFP, L1-penalized Cox models,. ); indeed, a selection method that selects more frequently the same set of covariates from the training-folds should be preferred, since such a selection method is less sensitive to the variability of the training dataset  [29]. For selection methods that constrain the number of variables included into the prognostic model, the Kuncheva index  [30]  could be used to assess the model stability. The Kuncheva index is a stability measure between -1 and 1, with larger values for more stable models: KI(modelfoldi, modelfoldj) = (r -(s 2 /N)/(s-(s 2 /N)), with s the number of predictors, r the number of shared predictors between the training folds i and j, and N the number of candidate covariates.
Model optimism The model optimism  [2]  is defined as the difference between the actual performance (actual performance in the targeted population) minus the apparent performance (the estimated performance from the training dataset). Model optimism is a consequence of model overfitting. Thus, we propose estimating the so-called validation surprise as the difference between the apparent performance estimated directly on the training-fold data (also used for model building) and the performance estimated on the test-fold data. Besides, the level of overfitting depends on the strength of selected predictors  [31] , the dependence between predictors, and the complexity of the model. Indeed, if a model is too complex, it will over-fit the training data and not generalize well to new data. Hence, models with high degree of optimism should be avoided because they are expected to present less generalization power.
Impact of omitted covariates Finally, some covariates could be excluded from the set of candidate covariates in case of limitations of the study sample size. The recommendation of 5-10 events per candidate covariate in the training dataset could lead to exclusion of some a priori less important variables from the set of candidate covariates. Hence, it is important to check i) whether some omitted covariates could still be associated with survival after adjustment on selected covariates, and ii) whether model performance could depend on the level of omitted covariates. First, similar to Therneau and Grambsch  [32] , we propose to use the Martingale residuals to test the residual association between the survival and each omitted covariate. In case of no association between the Martingale residuals and the omitted covariate, the covariate was considered not associated with the survival.
In practice, for each patient in the training-fold datasets, the Martingale residuals were averaged across all the folds and iterations. Second, we suggest to test whether the sensitivity and the specificity depend on omitted covariates using the following logistic models (in absence of censoring): logit(P(M i > c|T i ≤ t, Z i )) = Z i β sensi logit(P(M i ≤ c|T i > t, Z i )) = Z i β speci where, M i is the linear predictor for patient i of the Cox model without the omitted covariate, Z i is the omitted covariate for patient i, β sensi is the log odds ratio of the correct classification probability among deceased patients at time t for patients with covariate Z i = 1 vs Z i = 0, and β speci is the log odds ratio of the correct classification probability among survivor patients at time t for patients with covariate Z i = 1 vs Z i = 0.
Before RNA elution, the residual genomic DNA was digested using the Rnase-Free Dnase set (Qiagen, Hilden, Germany). Extracted RNA was reverse-transcribed into cDNA using SuperScript®VILO™ cDNA Synthesis Kit (Life Technologies, Chicago, IL). Finally, the PCR was performed on a LightCycler instrument using the standard Taqman Fast Advanced Master Mix PCR kit according to the manufacturer's instructions (Roche Molecular Biochemicals, Basel, Switzerland). PCR was performed with an initial denaturation step of 10 min at 95°C, followed by 45 cycles of a touchdown PCR protocol (10 sec at 95°C, 29 sec annealing at 68°C, and 1 sec extension at 72°C). The crossing point estimated by the LightCycler software was used as the unnormalized gene expression measure for each gene/sample combination. The gene expression data were normalized using a single reference gene.
79 [0. 74; 0. 84], and Kaplan-Meier estimated survival at day 14 was 0. 72 [0. 66;0. 77]. The training dataset included 156 patients and 44 death events (28%), with samples processed over 14 PCR batches. The test dataset included 95 patients and 27 death events (28%), with samples processed over 9 PCR batches. Unfortunately, the number of events was not balanced across batches as the mean and inter-quartile range for the proportion of events across batches in the training dataset were 0. 23 [0. 15;0. 30] and 0. 26 [0. 18;0. 31], respectively. As illustrated in Figure  1 , two resampling strategies were evaluated: A/20-times 5-fold stratified cross-validation with test-fold samples drawn randomly in the training dataset, but ensuring an equal number of events in each test-fold dataset; B/20-times 5-fold cross-validation with samples from entire PCR batches drawn randomly to be included in the test fold data.
The same approach was used for other covariates and no evidence of association was found for Lasso_Cox-4 model. Thus, we confirmed that this strategy is the best one here and can be retained for the subsequent validation step. Again, note that the p-values should be interpreted with caution due to the large number of test performed. Finally, the entire training dataset was used to build the prognostic model. The final Lasso_Cox-4 model included the following covariates: C1: HR= 1. Odds Ratio.
In case of predictor variables with missing values, it is possible to perform Multiple Imputations (MI). The idea of MI is to create m "completed datasets" with filled-in missing values. The imputation model for the missing values should contain all the candidate predictors, all the covariates that could be associated with the covariates with missing values, the event status and the survival time  [36]. The analysis is then performed on the m completed datasets using standard tools, and the results are combined into one final estimate using the Rubin's rule  [37]. In the context of prognostic model building, two difficulties remain: how to perform variable selection and how to estimate model performance based on the imputed datasets? Wood et al. [38]  compared different approaches to perform backward stepwise variable selection, and concluded that the only method that preserved the type I error was to select covariates based on the Rubin's rule estimated p-values at each step of the backward stepwise procedure.
Musoro et al. [39]  presented a procedure based on bootstrap resampling and multiple imputations to perform both variables selection via lasso and estimation of prognostic model performance. The authors recommend incorporating the MI procedure within the resampling step, i. e. to draw m completed datasets for each bootstrap sample. Briefly, for a given bootstrap sample, lasso estimates are averaged over all the imputed datasets to build a final model (meaning that zero and non-zero coefficients are averaged). Validated performance is then estimated on imputed datasets including all the samples (not only the bootstrap samples). Using this resampling framework in simulations, the internal estimated optimism was the closest to the external estimated optimism. An alternative approach is proposed by Chen et al.
[40]  who performed group-lasso to consistently select covariates across imputed datasets. Finally, additional future work would be needed to validate our strategy in high-dimensional settings (p >> n), in particular for microarray based prediction model building. In microarray studies, samples are also processed by batch, and it is also relevant to set apart entire batches in test-fold data. Of course, some statistical aspects may change to accommodate the specificity of the high-dimensional setting (e. g. the sizing strategy of the training dataset, see for instance  [15] ). As for the present RT-qPCR study, an evaluation based on real datasets will be required to validate our approach on other transcriptomic platforms.
Conclusions In conclusion, we presented resampling methods in order to estimate the performance of RT-qPCR prognostic models on a training dataset. These methods are expected to prevent any upward biases due to unaccounted technical and biological variability that may arise from the experimental and intrinsic design of the RT-qPCR assay. Although the estimation of the performance of the candidate models is a pivotal indicator to help select the best model, the final decision of selecting one or a few models requires consideration of additional criteria. We propose to use the stability of the selected variables, the model optimism, and the impact of the omitted variables on the model performance. These criteria should be weighted in accordance with the objective of the study. Model building strategies developed in highdimensional settings can also be very efficient in lowdimensional settings, in particular for covariate selection and coefficient penalization.
Framework Besides the usual tree-level Feynman rules, some extra ingredients have to be provided to the matrix-element generator in order to obtain the code for the simulation of a new physics process at NLO. These extra ingredients are the Ultra-Violet (UV) renormalisation counter-terms and a sub-class of the rational terms that enter the reduction of the virtual matrix elements, the so-called R 2 terms  [33]. UV and R 2 counter-terms can be computed starting from the model Lagrangian for any renormalisable theory via the NLOCT package  [34] , based on FeynRules  [35]  and FeynArts  [36]. The tree-level and NLO Feynman rules are exported in the Universal Feynrules Output (UFO)  [37]  format, as a Python module which can be loaded by matrix-element generators, such as MadGraph5 aMC@NLO. Finally, the UFO information is translated into helicity routines  [38]  by ALOHA  [39].
We remind the reader that MadGraph5 aMC@NLO is a meta-code that automatically generates the code for simulating any process at NLO(+PS) accuracy. MadGraph5 aMC@NLO adopts the FKS method  [40, 41]  for the subtraction of the singularities of the real-emission matrix elements, as automated in the MadFKS module  [42] ; virtual matrix elements are computed via the MadLoop module  [43] , which relies on the OPP  [44]  and Tensor Integral Reduction (TIR)  [45, 46]  methods, as implemented in CutTools  [47]  and IREGI  [48] , supplemented by an in-house implementation of the OpenLoops procedure  [49] ; finally, the generation of hard events and their matching to parton-shower simulations is performed à la MC@NLO  [50]. The matching to Her-wig6  [51] , Pythia6  [52]  (ordered in virtuality or in transverse momentum, the latter only for processes with no light partons in the final state), Herwig++  [53]  and Pythia8  [54]  is available.
As a consequence, the only inputs needed are the implementation of the model in FeynRules and the inclusion of the running of the bottom quark mass, which follows from our renormalisationscheme choice. More details are given in the next section. Both the 4FS and the 5FS computations have been performed in the framework specified above, with the additional advantage of ensuring the consistency of all settings and input parameters. To guarantee the validity of our analysis, we have performed a detailed comparison of the inclusive cross sections obtained with MadGraph5 aMC@NLO against previous results published in literature. The calculation in the 4FS has been compared to the private code of Ref. [10]  at the level of total rates, and a good agreement within the numerical uncertainty of the reference code has been found.
Therefore its Yukawa mass is set equal to the pole mass. On the contrary, the bottom quark Yukawa has been renormalised in the MS scheme, i. e. δy b = - √ 2 v g 2 s m y b 4π 2 ¯. ( 7 ) This scheme choice has the advantage of resumming potentially large logarithms log(µ R /m b ) (with µ R ∼ m H ± ) to all orders. The bottom Yukawa mass is set to the value of the running MS mass at the renormalisation scale. Besides the modifications at the level of the UFO model, also the code written by MadGraph5 aMC@NLO had to be changed in order to account for the additional scale dependence introduced by the b-quark Yukawa, in particular for what concerns the on-the-fly evaluation of scale uncertainties obtained via reweighting  [56]. This has been done in an analogous way as for bottom-associated Higgs production  [28] , by splitting the cross section in parts that factorise different powers of y b , i.
Acknowledgements We are indebted to  Fabio Maltoni  for his constant encouragement and his valuable suggestions. We thank  Michael Spira ,  Paolo Torrielli  and  Rikkert Frederix  for useful discussions and  Michael Krämer  for having provided the 4FS code that we used for comparison. Thanks to the SUSY working group in Cambridge for comments and discussions. Finally, we are grateful to  Liron Barack  and her experimental colleagues for their suggestions and requests. MW and MZ thank the  Cavendish Laboratory  and the  HEP group  for their hospitality in Cambridge during the course of this work. CD is a  Durham International Junior Research Fellow. The work of MU is supported by the  UK Science and Technology Facilities Council. MW is supported by the  Swiss National Science Foundation (SNF)  under contract  200020-141360.
The work of MZ is supported by the  ERC  grant " Higgs@LHC ", in part by the  Research Executive Agency (REA)  of the  European Union  under the Grant Agreement number  PITN-GA-2010-264564  (LHCPhenoNet), and in part by the  ILP LABEX  ( ANR-10-LABX-63 ), in turn supported by  French  state funds managed by the  ANR  within the " Investissements d'Avenir" programme  under reference  ANR-11-IDEX-0004-02. This work was supported in part by the  European Union  as part of the  FP7 Marie Curie Initial Training Network MCnetITN  ( PITN-GA-2012-315877 ).
The calculation times in this example are 70 s for SFEM and 15 s (h = 0. 3 m), 38 s (h = 0. 2 m), 233 s (h = 0. 1 m), 544 s (h = 0. 08 m) for the FEM. This includes 335 frequencies below 1000 Hz. It must be noted that SFEM time is constant while FEM time has an almost cubic grow with the inverse of the element size. The situation is even more favourable to SFEM if higher frequencies, requiring finer meshes would have been considered. To look at the relative times and the different evolution with frequency is much more important than the final number of seconds. On the one hand, the programs have been run in a simple desktop computer. On the other hand, the SFEM software is a home-made software that could be optimised while Code-Aster is a very big project. The differences are caused by the method.
The visual, auditory, and narrative elements were brought together in the Unity platform, software that is typically used by computer game developers. CAVE2-specific user interaction is scripted using the getReal3D plugin for Unity developed by Mechdyne Corporation. Because the project was developed in Unity, however, it is portable to other interactive environments. CAVE2 is powered by computer cluster connected to highspeed networks to enable users to better cope with informationintensive tasks. The CAVE immerses people into worlds too large, too small, too dangerous, too remote, or too complex to be viewed otherwise. CAVE2 is approximately 24 feet in diameter and 8 feet tall, and consists of 72 near-seamless passive stereo offaxis-optimized 3D LCD panels, a 36-node high-performance computer cluster, a 20-speaker surround audio system, a 10camera optical tracking system and a 100-Gigabit/second connection to the outside world.
1. Introduction 2RXP J130159. 6-635806, first discovered by the ROSAT observatory, was later rediscovered in hard X-rays by the INTEGRAL/IBIS telescope and designated with the name IGR J13020-6359  (Bird et al. 2006; Revnivtsev et al. 2006 ). The first comprehensive analysis of the temporal and spectral X-ray properties of this source was done by  Chernyakova et al. (2005)  using data from the ASCA, BeppoSAX, INTEGRAL and XMM-Newton observatories. In particular, XMM-Newton data showed coherent pulsations with a period of around 700 s. Joint spectral analysis of XMM-Newton and INTEGRAL data demonstrated that the spectral shape is very typical for accretion-powered X-ray pulsars (namely, an absorbed power law with a high-energy cut-off). Based on 2MASS archival data,  Chernyakova et al. (2005)  proposed that the binary companion to 2RXP J130159.
Observations 2RXP J130159. 6-635806 was initially serendipitously observed with Nuclear Spectroscopic Tele-scope Array (NuSTAR)  (Harrison et al. 2013)  during observations of the gamma-ray binary PSR B1259-63, with three data sets taken in 2014 May-June  (Chernyakova et al. , in prep. ). In one observation, 2RXP J130159. 6-635806 appears in the corner of ∼ 13 ′ × 13 ′ NuSTAR's field of view (FOV), and, in two more, the source is at the extreme edge of the FOV. Despite the large off-axis angles, the NuSTAR data were successfully used to extract coherent pulsations. This motivated the NuSTAR team to trigger an on-axis 30 ks observation of 2RXP J130159. 6-635806 in order to obtain high-quality data for spectral and timing analysis. Table  1  lists the NuSTAR observations used in this work. NuSTAR carries two co-aligned identical X-ray telescopes operating in wide energy band from 3 to 79 keV with angular resolution of 18" (FWHM) and half-power diameter (HPD) of 58".