dataesr/scientific-paragraphs-categorization huggingface dataset:
Study Design SP0993 (ClinicalTrials. gov identifier: NCT01243177) was a multicenter, double-blind, double-dummy, randomized, positive-controlled, non-inferiority trial comparing the efficacy and safety of LCM and CBZ-CR as monotherapy in patients with epilepsy  [7]. Target dose levels were 200, 400, or 600 mg/day for LCM, and 400, 800, or 1200 mg/day for CBZ-CR, respectively (twice-daily administration). Patients were C16 years of age and newly diagnosed with focal or generalized tonic-clonic seizures without signs of focal onset, provided they had no history or clinical or electroencephalographic findings suggestive of idiopathic generalized epilepsy. Patients with a history or a clinical or electroencephalographic finding suggestive of genetic generalized epilepsy were excluded  [7]. The dose-escalation procedure is depicted in Fig.
Modeling Strategy and Software As stated previously, step-wise model building, including the evaluation of covariates, of the dropout and the seizure model was carried out on a historic data set comparing CBZ-CR with levetiracetam (N01061; ClinicalTrials. gov identifier: NCT00150735). In a subsequent step, the resulting final models were applied to the new dataset from clinical trial SP0993. Finally, all model parameters were re-estimated on the SP0993 dataset. The turnaround time of the SP0993 analysis was critical to meet the project timelines, hence this two-step approach avoiding a full model development on SP0993 data. The parameter estimates of the dropout and seizure model developed on N01061 data are provided in Online Resource 1. The analyses and simulations were performed using NONMEM version 7.
2. 0 (ICON Development Solutions, Ellicott City, Maryland, USA)  [10] , supplemented with the PsN toolkit version 4. 2. 0 (https://uupharmacometrics. github. io/PsN/) [11] , and were further processed using R Version 3. 2. 0 (https://www. r-project. org/) [12]. In the pharmacokinetic analysis, the first-order conditional estimation method with interaction was used, while the Laplace's method was used for the time-to-event analysis.
The hazard h 0 (t) starts with a value of k1 until breakpoint 1 (BP1), where it switches to k2, until time reaches BP2. Automated stepwise covariate modeling using a forward addition/backward deletion procedure as implemented in PsN was applied on the N01061 dataset. The covariate effect was added to the hazard as: h t ð Þ ¼ h 0 t ð Þ Â expðCOVARÞ where COVAR describes the hazard-covariate relationship. Because this function assumes proportionality of the hazard with respect to the covariate effect, a hazard ratio of the covariate effect is simply: exp(COVAR). For example: COVAR ¼ h Â SEX; where SEX ¼ 1 if female; 0 if male In this case, exp(h) is the hazard ratio of female vs. male, with values[1 indicating a higher risk of dropout for women. For a continuous covariate COVAR, the relationship was as follows (e.
g. , age): COVAR ¼ h Â ðAGE À median ½AGEÞ The covariate effect relationship was considered statistically significant when p \ 0. 005 (DOFV = 7. 88, for 1 degree of freedom difference) for forward selection and p \ 0. 001 (DOFV = 10. 83) for backward deletion. The following covariates were tested: treatment (CBZ-CR, levetiracetam, or LCM), duration of epilepsy, number of AEDs taken before the trial, number of seizures in the previous 3 months (NSP3M), race, geographical region, sex, age, and body weight. The forward-backward covariate search was performed on the N01061 dataset but was not repeated on the SP0993 dataset in full. Any relationship found on the N01061 dataset was maintained for the SP0993 modeling with the coefficient re-estimated. Only relationships pertinent to the LCM treatment were revisited when analyzing the SP0993 data (treatment effect on dropout).
Seizure Modeling For the purpose of modeling, if a patient experienced multiple seizures on one day, this was counted as a single event and is referred to here as a seizure or seizure event. Although roughly 30% of the patients had multiple seizure events during the trial, a standard repeated-time-to-event model did not satisfactorily account for the low seizure rate overall in the population (more than 50% of the patients had no seizure at all). Instead, an approach first presented by Abrantes et al. [15]  was successfully employed, using two sub-models of the hazard: one for the time to the first seizure and one for the time to the second and subsequent seizures (time counted from the first dose): h 1st ðtÞ ¼ h 1st 0 ðtÞ; if SCOUNT\1ðhazard for 1st seizureÞ h 2ndþ ðtÞ ¼ h 2ndþ 0 ðtÞ Â expðgÞ; if SCOUNT ! 1ðhazard for 2nd and subsequent seizuresÞ where SCOUNT is a counter variable included in the NONMEM code that starts at 0 for every new patient and increases by one with every seizure that occurs in a patient as the estimation proceeds through the individual data records.
Owing to the skewed distribution of this variable (median 2, range 0-450), NSP3M was split into four categories: \2, 2-6, 7-50, and [50, with 2-6 being the reference category. The process for covariate modeling was similar to that described for the dropout model, i. e. , significant relationships found on the N01061 dataset were maintained and coefficients re-estimated. Only the agerelationship was revisited specifically because of the potential clinical interested. The same covariates tested in the dropout model were also tested in the seizure model.
Model Qualification To evaluate the predictive performance of the models, visual predictive checks  [16]  and posterior predictive checks  [17]  were performed using the final model parameter estimates. A simulation dataset was generated using a patient population with the same covariates as in the dataset used for model building. Every patient in the simulation dataset had dummy observation records to allow for seizure or dropout events being simulated at every day from 1 to 800. As the simulations should mimic the original trial design with dose escalation based on seizure events, the individual dose and AUC were assigned to patients 'on-the-fly' during the simulation. This means that the simulation code in NONMEM (see Online Resource 4) included flag and counter variables changing their value if a seizure (or dropout) event was simulated at a given point in time for a patient.
If, for example, a seizure event was simulated and the current time was within the evaluation period (days 21-203 since the start of a new dosing step), the flag for the dose escalation was changed and at the next iteration for this patient (next record in the simulation dataset) a higher dose was assigned and the AUC for this patient (based on individual pharmacokinetic parameters) was re-calculated. If a dropout event was simulated this was also flagged accordingly, thus records after the dropout day were filtered out in the post-processing step. The simulations were repeated 500 times and Kaplan-Meier curves comparing observed and predicted probabilities (95% prediction interval) were generated. The positive predictive check focused on two trial endpoints: percentage and the Kaplan-Meier estimated proportion of patients being seizure free for 6 months at the last evaluated dose.
Genetic discovery study participants In the genetic discovery analysis, we combined data from up to 159,940 participants (maximum number available for any variant) of European, African American, East Asian, and South Asian ancestry, including subsets from previous publications  [4, 5] (S1 Table, S2 Fig). All participants were free of diabetes defined by physician diagnosis, medication use, or fasting glucose (FG) ! 7 mmol/L. A small number of cohorts also removed individuals with 2hr glucose (2hrGlu) ! 11. 1 mmol/L, or HbA1c ! 6. 5%, where FG was not available (details of exclusions by individual cohorts, S1 Table ). Analysis followed the details in S1 Analysis Plans (Hemoglobin A1c Genetic Discovery Analysis Plan).
Genotyping and quality control Each cohort was genotyped on commercially available genome-wide arrays (for instance, the Affymetrix Genome-Wide Human SNP Assay 6. 0 or the Illumina Human610-Quad Bead-Chip) or the Illumina CardioMetabochip (Metabochip)  [15]. Variant and sample quality control (QC) was conducted within each cohort following a shared analysis plan (S1 Analysis Plans). Cohorts were advised to keep SNPs with hardy-weinberg-disequilibrium p-value ! 1 × 10 -6 , SNP genotyping call rate ! 95% and minor allele frequency (MAF) ! 1% (full details of SNP and sample QC can be found in S1 Table ). Following QC, studies with genome-wide array data were imputed (primarily using the Phase 2 of the International HapMap Project reference panel  [16] , see S1 Table, row 40), and poorly imputed variants (variants which could not reliably be inferred from surrounding variants) were excluded based on standard imputation quality thresholds (R-sq < 0.
0001). We retained the lead variant in the X-chromosome analysis of the African American ancestry data (rs1050828, G202A in G6PD) despite significant heterogeneity, as it was a strong biological candidate. Ancestry-specific meta-analysis results were conservatively corrected for a second round of genomic control by ancestry: European (λ GC = 1. 072); African American (λ GC = 1. 020); East Asian (λ GC = 1. 027); South Asian (λ GC = 1. 004); and combined using the Meta-Analysis of Transethnic Association (MANTRA) software that accounts for allelic heterogeneity across ancestry groups  [20].
Following the analysis plan detailed in S1 Analysis Plans (The Difference Between Fructosamine-inferred HbA1c and Measured HbA1c) we thus calculated the estimated residuals from a linear regression of HbA1c on fructosamine in ARIC African Americans (N = 1,676) to determine whether the T allele was associated with lower HbA1c than predicted by fructosamine, suggesting that the T allele artificially lowered HbA1c through a reduction in the average erythrocytic lifespan. We then reported the mean estimated residuals by genotype (women: CC, CT, TT; men: C, T). Estimated number of African Americans with T2D in the United States whose diagnosis would be missed due to the G6PD variant if screened with HbA1c. Using publicly-available data from the National Health and Nutrition Examination Survey (NHANES) 2013-2014 [34], a nationally representative sample of US residents, we calculated the proportion of African American adults (aged ! 18 years) with T2D who would be missed by not accounting for rs1050828 when using a single HbA1c diagnostic threshold of 6.
Public health implications of the G6PD variant on T2D screening Given the large effects of the G6PD G202A variant on HbA1c levels, we sought to investigate the impact this variant would have on diabetes detection if using HbA1c as a screening tool. To do this, we used publicly-available data from NHANES 2013-2014 [34], a nationally representative sample of the US, to calculate the proportion of African Americans adults with T2D who would be missed by not accounting for rs1050828 when using a single HbA1c diagnostic threshold of 6. 5%, assuming the observed effect size of rs1050828, allele frequency of 11%, and accounting for NHANES sampling design. In the NHANES sample of African Americans (N = 1,133), the mean age was 44. 2 years (standard error 0. 9), 55. 2% were women, and mean HbA1c, excluding those with physician-diagnosed T2D, was 5.
Acknowledgments Published data on glycemic traits were contributed by MAGIC investigators and have been downloaded from www. magicinvestigators. org. The ARIC authors thank the staff and participants of the ARIC study for their important contributions. The CAGE authors thank the participants who made this work possible and who gave it value. The CAGE authors would like to thank  Drs. Toshio Ogihara ,  Yukio Yamori ,  Akihiro Fujioka ,  Chikanori Makibayashi ,  Sekiharu Katsuya ,  Ken Sugimoto ,  Kei Kamide , and  Ryuichi Morishita  and the many physicians of the participating hospitals and medical institutions in Amagasaki Medical Association for their assistance in collecting the DNA samples and accompanying clinical information. The CROA-TIA authors would like to acknowledge the invaluable contributions of the  Institute for Antropological Research, Zagreb, Croatia , the administrative team in Split, and the people of Vis and Korcula.
Decoding, ie prediction from brain images or signals, calls for empirical evaluation of its predictive power. Such evaluation is achieved via cross-validation, a method also used to tune decoders' hyper-parameters. This paper is a review on cross-validation procedures for decoding in neuroimaging. It includes a didactic overview of the relevant theoretical considerations. Practical aspects are highlighted with an extensive empirical study of the common decoders in withinand across-subject predictions, on multiple datasets -anatomical and functional MRI and MEG-and simulations. Theory and experiments outline that the popular "leave-one-out" strategy leads to unstable and biased estimates, and a repeated random splits method should be preferred. Experiments outline the large error bars of cross-validation in neuroimaging settings: typical confidence intervals of 10%.
Test set Train set Full data and keeping enough training data to reach a good fit with the decoder. Indeed, theoretical results show that crossvalidation has a negative bias on small dataset [2, sec. 5. 1] as it involves fitting models on a fraction of the data. On the other hand, large test sets decrease the variance of the estimated accuracy [2, sec. 5. 2]. A good cross-validation strategy balances these two opposite effects. Neuroimaging papers often use leave one out cross-validation, leaving out a single sample at each split. While this provides ample data for training, it maximizes test-set variance and does not yield stable estimates of predictive accuracy  3. From a decision-theory standpoint, it is preferable to leave out 10% to 20% of the data, as in 10-fold cross-validation  [18, chap.
Experiments on real neuroimaging data A variety of decoding datasets. To achieve reliable empirical conclusions, it is important to consider a large number of different neuroimaging studies. We investigate crossvalidation in a large number of 2-class classification problems, from 7 different fMRI datasets (an exhaustive list can be found in Appendix E). We decode visual stimuli within subject (across sessions) in the classic Haxby dataset  [19] , and across subjects using data from  [10]. We discriminate across subjects i) affective content with data from  [59] , ii) visual from narrative with data from  [39] , iii) famous, familiar, and scrambled faces from a visual-presentations dataset  [22] , and iv) left and right saccades in data from  [26]. We also use a non-published dataset, ds009 from openfMRI  [47].
All the across-subject predictions are performed on trial-by-trial response (Z-score maps) computed in a first-level GLM. Finally, beyond fMRI, we perform prediction of gender from VBM maps using the OASIS data  [34]. Note that all these tasks cover many different settings, range from easy discriminations to hard ones, and (regarding fMRI) recruit very different systems with different effect size and variability. The number of observations available to the decoder varies between 80 (40 per class) and 400, with balanced classes. The results and figures reported below are for all these datasets. We use more inter-subject than intra-subject datasets. However 15 classification tasks out of 31 are intra-subject (see Tab. A1). In addition, when decoding is performed intra-subject, each subject gives rise to a crossvalidation.
We consider two-class decoding problems, focusing on either the image content (faces vs tools) or the functional role in the working memory task (target vs low-level and high-level distractors). This yields in total four classification analyzes per subject. For each trial, the feature set is a time window constrained to 50 ms before and 300 ms after event onset, emphasizing visual components. We use the cleaned single-trial outputs from the HCP "tmegpreproc" pipeline. MEG data analysis was performed with the MNE-Python software  [13, 14]. Full details on the analysis are given in Appendix F. 3. Experimental setup. Our experiments make use of nested cross-validation for an accurate measure of prediction power. As in figure  2 , we repeatedly split the data in a validation set and a decoding set passed on to the decoding procedure (including parameter-tuning for experiments in 3.
Results: cross-validation to assess predictive power Reliability of the cross-validation measure. Considering that prediction error on the large left-out validation set is a good estimate of predictive power, we use it to assess the quality of the estimate given by the nested crossvalidation loop. Figure  5  shows the prediction error measured by cross-validation as a function of the validationset error across all datasets and validation splits. It reveals a small negative bias: as predicted by the theory, 8 Similar decoders adding a regularization that captures spatiotemporal correlations among the voxels are well suited for neuroimaging  [15, 16, 25, 36]. Also, random forests, based on model averaging discussed above, have been used in fMRI  [29, 32]. However, this review focuses on the most common practice.
Experiments on MRI (anatomical and functional), MEG, and simulations consistently exhibit these large error bars due to data scarcity. Such limitations should be kept in mind for many MVPA practices that use predictive power as a form of hypothesis testing -eg searchlight  [28]  or testing for generalization  [26] -and it is recommended to use permutation to define the null hypothesis  [28]. In addition, in the light of cross-validation variance, methods publications should use several datasets to validate a new model. Guidelines on cross-validation. Leave-one-out crossvalidation should be avoided, as it yields more variable results. Leaving out blocks of correlated observations, rather than individual observations, is crucial for nonbiased estimates. Relying on repeated random splits with 20% of the data enables better estimates with less computation by increasing the number of cross-validations without shrinking the size of the test set.
Concluding remarks. Evaluating a decoder is hard. Crossvalidation should not be considered as a silver bullet. Neither should prediction performance be the only metric. To assess decoding accuracy, best practice is to use repeated learning-testing with 20% of the data left out, while keeping in mind the large variance of the procedure. Any parameter tuning should be performed in nested crossvalidation, to limit optimistic biases. Given the variance that arises from small samples, the choice of decoders and their parameters should be guided by several datasets. Our extensive empirical validation (31 decoding tasks, with 8 datasets and almost 1 000 validation splits with nested cross-validation) shows that sparse models, in particular 1 SVM with model averaging, give better prediction but worst weight-maps stability than non-sparse classifiers.
If stability of weight maps is important, non-sparse SVM with C = 1 appears to be a good choice. Further work calls for empirical studies of decoder performance with more datasets, to reveal factors of the dataset that could guide better the choice of a decoder for a given task. The classes are centered respectively on vectors (µ,. , µ) and (-µ,. , -µ) where µ is a parameter adjusted to control the separability of the classes. With larger µ the expected predictive accuracy would be higher. In addition, to mimic the time dependence in neuroimaging data we apply a Gaussian smoothing filter in the sample direction on the noise (σ = 2). Code to reproduce the simulations can be found on https: //github. com/GaelVaroquaux/cross_val_experiments. We produce different datasets with predefined separability by varying 16 µ in (.
Acknowledgments This work was supported by the  EU FP7/2007-2013  under grant agreement no. 604102  (HBP). Computing resource were provided by the  NiConnect  project ( ANR-11-BINF-0004  NiConnect) and an  Amazon Webservices Research Grant. The authors would like to thank the devel-opers of nilearn 13 , scikit-learn 14 and MNE-Python 15 for continuous efforts in producing high-quality tools crucial for this work. In addition, we acknowledge useful feedback from  Russ Poldrack  on the manuscript.
Log-reg 1 Figure 8 : 18 Figure 8: Tuning curves for SVM 2 , logistic regression 2 , SVM 1 , and logistic regression 1 , on the scissors / scramble discrimination for the Haxby dataset [19]. The thin colored lines are test scores for each of the internal cross-validation folds, the thick black line is the average of these test scores on all folds, and the thick dashed line is the score on left-out validation data. The vertical dashed line is the parameter selected on the inner cross-validation score.
Comparing parameter-tuning strategies on prediction accuracy. Relating pairs of tuning strategies. Each dot corresponds to a given dataset, task, and validation split. The line is an indication of the tendency, using a lowess non-parametric local regression. The top row summarizes results for non-sparse models, SVM 2 and logistic regression 2 ; while the bottom row gives results for sparse models, SVM 1 and logistic regression 1. 100% Non-sparse models 100% Accuracy with C=1 50% 60% 70% 80% 90% SVM logreg Accuracy with C=1000 50% 60% 70% 80% 90% SVM logreg 50% 60% 70% 80% 90% 100% Accuracy for crossval + refitting 50% 60% 70% 80% 90% 100% Accuracy for crossval + refitting Accuracy with C=1 50% 60% 70% 80% 90% 100% Sparse models SVM logreg Accuracy for crossval + refitting 50% 60% 70% 80% 90% 100% SVM logreg 50% 60% 70% 80% 90% 100% Accuracy for crossval + averaging 50% 60% 70% 80% 90% 100% Accuracy for crossval + averaging Decoder accuracy on validation set Figure A3: 50.
Prediction -stability tradeoff The figure reports for each dataset and task the stability of the weights as a function of the prediction accuracy measured on the validation set, with the different choices of decoders and parameter-tuning strategy. The line is an indication of the tendency, using a lowess local regression. The stability of the weights is the correlation across validation splits. +0. 4 Non-sparse models +0. 4 Sparse models +0. 3 +0. 3 Dataset Weight stability Description 0. 3 0. 4 0. 2 0. 1 0. 0 +0. 1 +0. 2 SVM; C=1 SVM; C=1000 SVM; CV + refitting SVM; CV + averaging # samples logreg; C=1 logreg; C=1000 logreg; CV + refitting logreg; CV + averaging # blocks (sess. /subj. ) Weight stability 0. 2 0. 1 0. 0 +0. 1 +0. 2 0. 3 bottle / scramble Task 0. 4 cat / bottle mean accuracy SVM 2 SVM 1 75% 86% 62% 69% 10% 5% Prediction accuracy 0% +5% 10% 5% cat / chair cat / face Prediction accuracy 0% 69% 80% +5% 65% 72% cat / house 86% 95% cat / scramble 83% 92% 0.
Results and discussion Expression of  19  F-labeled bPGM Incorporation of uorine into the bPGM enzyme structure was accomplished by use of 5-uorotryptophan (5FW). The bPGM structure contains two tryptophan residues, W24 and W216. Analysis of a previously recorded crystal structure of bPGM with a TSA complex (PDB ID code 4C4R) 7 reveals that W24 is on the enzyme cap domain and W216 is on the enzyme core domain. W24 is within $3 Å of the active site, allowing the indole nitrogen to form a hydrogen bond with the substrate during catalysis, while W216 is on the outer surface of the core domain, $22 Å from the active site, and is not in direct contact with the substrate (Fig. S1 † ). Since the two tryptophan residues are located on different domains, we anticipated that these differences in environment would result in distinct  19  F resonances by  19  F NMR spectroscopy.