dataesr/scientific-paragraphs-categorization huggingface dataset:
After measuring hit rates and false alarm rates for all possible statistical thresholds, an ROC curve was obtained, expressing hit rate as a function of false alarm rate. The area under this ROC curve (AUC) can be used as a threshold-independent measure of a tests' sensitivity. We computed this AUC for all seven tests (Figure  4D ). As could be expected from the p-value maps, all tests performed reliably, with AUC-values above 0. 8. The most sensitive test appeared to be POS, followed by POP, and then PBI. This ranking was the same, whether a standard permutation test or a hybrid permutation + z-score test was applied, though the latter performed consistently (albeit marginally) better than the former. Finally, the circular Watson-Williams test yielded the lowest AUC value (0. 82) 7. Nonetheless, as will be seen later (Section Artificial datasets), the circular Watson-Williams test is often among the most sensitive measures of phase opposition.
Artificial Datasets The confirmation of the adequacy of all seven tests on a given "reference" dataset was an essential step, but leaves open a number of questions. Would all tests prove equally robust to variations in specific properties of the dataset, such as the frequency of the rhythmic modulation, its magnitude, the number of trials collected, the presence and amplitude of poststimulus ERPs, and so on? Answering these questions with real experimental datasets could easily require several hundred experiments, and a lifetime of data collection. To address these issues, we created instead a large number of artificial datasets, for which various parameters could be precisely controlled (Figure  3 ). Each dataset included n trials with simulated electrophysiological signals (initialized as white noise) and an associated trial outcome A or B.
The trial outcome was decided with a probability that was a sinusoidal function of the signal phase angle at a specific time and frequency. Finally, noisy ERP waveforms, of possibly different amplitudes for the two trial types A or B, were added to the original electrophysiological signal. The number of trials (n), the average likelihood of outcome A (p A ), the depth (mod), frequency (f), and time (t) of phase modulation, as well as the average ERP amplitudes (ERP A and ERP B ) were parameters that could be varied in distinct simulations. For each fixed set of parameters, we generated 100 random artificial datasets and applied the seven phase opposition measures. The performance of each measure was quantified by their statistical power: the proportion of datasets (out of 100) yielding a successful statistical outcome, with significant phase opposition detected inside but not outside a target timefrequency region.
This statistical power was computed using a threshold-independent AUC procedure 8. Statistical power values above 0. 8 are conventionally considered appropriate. Therefore, a single value describing statistical power over the 100 simulated datasets can serve as a main criterion to assess the reliability of each measure (value > 0. 8). Nonetheless, for the additional purpose of comparing measures against each other, it is useful to keep in mind that differences of statistical power equal to or larger than 0. 06 (e. g. , 0. 87 vs. 0. 93) would normally be statistically significant at the p < 0. 05 level using a χ 2 -test.
Depth of Oscillatory Phase Modulation The first set of simulations investigated the effect of the depth of oscillatory phase modulation (Figure  5 ). Indeed, this manipulation is directly linked to the detectability of phase opposition: with a phase modulation of 1 the outcome likelihood changes from 100% A to 100% B between one oscillatory phase and its opposite, and thus phase opposition should reach its maximal value; whereas with 0 phase modulation the likelihood 8 Different from the previous analyses (Figure  4 ), in which only one dataset was available, here the hit and false alarm rates were not defined as proportions of time-frequency points, but as proportions of datasets (out of 100). This implies that a single significant p-value inside the target time-frequency region of all 100 datasets could suffice to yield 100% hit rate, and thus, that the all-or-none nature of the circular Watson-Williams test was less likely to impair its AUC performance here.
Number of Permutations While other parameters remained fixed as before, and the modulation depth parameter was set at mod = 0. 4, we varied the number of permutations from 10 to 10,000. Logically, this manipulation should only alter the non-parametric tests that rely on such permutations to estimate their null distribution. As expected, therefore, the parametric circ. W-W test was wholly unaffected by the variations (Figure  6 ). Although both types of non-parametric tests (permutation, or permutation + z-score) were expected to depend on permutation number, the hybrid permutation + z-score test appeared remarkably robust at low permutation numbers, with statistical power remaining above 0. 9 even down to only 10 permutations (Figure  6 ). This suggests that the mean and standard deviation computed across only 10 surrogate (permuted) datasets are sufficiently precise estimates of the shape of the null distribution.
43 with p A = 0. 8 (see bottom-right time-frequency map). This puzzling result (considering that A and B were arbitrarily assigned labels) can in fact be explained by Equation (  9 ): due to the cosine function, outcome A was always maximal around phase 0, and outcome B at the opposite phase. This arbitrary choice effectively broke the symmetry of the dataset when in the presence of ERPs: the phase of the ERP signal around the critical time, similar for the two trial groups, could help (top-left map) or impair phase opposition detection (bottom-right map), depending on which of the two trial groups had a phase closest to that of the ERP 11. Overall, the strong decrease of statistical power obtained with unbalanced datasets (Figure  10 ) has important implications for experimental design and data analyses practices.
DISCUSSION Seven measures of phase opposition were compared in a variety of real and artificial experimental situations. All measures proved their value on a previously published dataset  (Busch et al. , 2009 ; Figure  4 ). On a series of artificial datasets with systematic manipulations of experimental parameters, we were able to provide a detailed assessment of all methods, and understand their common or individual reactions to changes in certain key parameters. These results informed us about the conditions in which phase opposition experiments are most likely to succeed, and about the right measure(s) to use for every situation.
We thus recommend using the hybrid technique, or else using the conventional permutation method with sufficiently large numbers of permutations (at least 10,000). No single measure was systematically better than all the other ones; rather, the optimal measure was strongly tied to the exact parameter values of the artificial dataset generation process. Across all simulations (Figures 5-13), however, the winning measure was always either POS or circ. W-W, and never POP or PBI. We thus recommend abandoning these two measures, especially PBI which we had introduced in 2009  (Busch et al. , 2009)  and which has been independently employed in (at least) 13 publications since  (Hamm et al. , 2012; Ng et al. , 2012; Auksztulewicz and Blankenburg, 2013; Hanslmayr et al. , 2013; Manasseh et al. , 2013; Rana et al.
CONCLUSION Independent of the exact analysis procedure employed, our explorations have revealed that the likelihood for an experimenter to detect phase opposition in an electrophysiological dataset is affected by several factors. This was true, even using a constant magnitude of phasic modulation (for example, in most of our simulations a reversal of rhythmic phase produced a 40% modulation of the probability of task outcome A vs. B; Figures  6 7 8 9 10 11 12 13 ). Among the most important factors to consider for phase opposition analyses are the absolute and relative trial numbers (Figures 9, 10), the overall ERP amplitude (Figure  8 ) and any potential ERP amplitude difference between the trial groups (Figure  13 ), as well as the frequency (Figure  11 ) and exact time (Figure  12 ) of the phasic modulation.
B), and for each permutation recalculating POS, POP, and PBI. The ranking (percentile) of the original dataset values against these null-hypothesis distributions could be used as a p-value (right hand-side, green time-frequency maps). Alternately, the null-hypothesis distributions could be summarized by their mean and standard deviation (across permutations), the values of the original dataset could then be expressed as a standardized z-score against the null distribution, and a p-value assigned using the normal cumulative distribution function (right hand-side, red time-frequency maps).
Only a tiny fraction of the data and metadata produced by an fMRI study is finally conveyed to the community. This lack of transparency not only hinders the reproducibility of neuroimaging results but also impairs future meta-analyses. In this work we introduce NIDM-Results, a format specification providing a machine-readable description of neuroimaging statistical results along with key image data summarising the experiment. NIDM-Results provides a unified representation of mass univariate analyses including a level of detail consistent with available best practices. This standardized representation allows authors to relay methods and results in a platform-independent regularized format that is not tied to a particular neuroimaging software package. Tools are available to export NIDM-Result graphs and associated files from the widely used SPM and FSL software packages, and the NeuroVault repository can import NIDM-Results archives.
Databases have been built to provide metadata associated with published papers, either manually curated (e. g. , BrainMap  13, 14  ) or automatically-populated using text-mining algorithms (e. g. , NeuroSynth  15, 16  ), but, ideally, these metadata should be made available by the authors themselves at the time of the publication, together with the data. Additionally, searchable metadata, could help identify potential confounding factors that are currently being overlooked (e. g. , how different smoothing kernels impact the meta-analysis, or the influence of different processing strategies on the outcome of the analysis). In order to make neuroimaging results available in a machine-readable form a number of key technical issues have to be addressed. First, the scope of the metadata to be shared must be defined.
For example, FSL's parameter estimate maps (e. g. , pe1. nii. gz) are the equivalent of SPM's beta maps (e. g. , beta_0001. nii). They can also use the same term when referring to different concepts. For example, SPM and FSL both use a global scaling of the data to get 'percent BOLD signal change', but due to differences in how the brain mask and mean signal are computed, the data are scaled quite differently  23  and are not comparable. In order to fully describe an analysis, the sharing of software-specific batch scripts (e. g. , SPM matlabbatch files, FSL fsf files, or history stored in AFNI brick headers) would be a simple solution to provide all the parameters from an analysis, but the ability to compare and query across software would still be lacking. Pipeline systems like NiPype  24  , LONI Pipeline  25  and CBRAIN  26  do explicitly model analysis steps, but a large volume of research is still conducted directly with tools not embedded in pipelines.
XCEDE has been used by multiple sites across the United States and the United Kingdom in the context of the fBIRN project and is still in use by the Human Imaging Database  28, 31. An implementation was provided for SPM  32, 33  as well as a set of tools  34. However, the XCEDE model was not implemented by other imaging software, supported limited provenance information, and did not offer the ability to jointly share image data summarising the experiment. Beyond neuroimaging, encoding of provenance, i. e. , keeping track of the processes that were applied to the data, encompassing a description of the tools, data flow and workflow parameterization, is a topic of growing interest in science in general. A number of solutions have been proposed in order to support better documentation of research studies.
This agent is associated with all activities within the bundle. Provenance of the 'NIDM-Results bundle' is also recorded: the bundle was generated by a 'NIDM-Results Export' activity which was performed by a 'NIDM-Results Exporter' software agent corresponding to the software used to create the NIDM-Results document (e. g. , FSL's Python scripts, named 'nidmfsl' or SPM's  Updates. Each release of NIDM-Results is associated with a version number. Comments on the current version as well as suggestions of extension can be provided on the GitHub nidm repository: https:// github. com/incf-nidash/nidm. Each extension or proposition of update will be reviewed and discussed with the members of the INCF NIDASH task force.
Implementation SPM12 natively supports export of its results into a NIDM-Results pack, either by the use of a contextual menu in the results table or non-interactively via the batch interface as illustrated in Fig. 3. Export of FEAT results from FSL into a NIDM-results pack can be performed using the Python module nidmfsl (https://pypi. python. org/pypi/nidmfsl), as also illustrated in Fig. 3. nidmfsl was integrated in NeuroVault and as a plugin  39  of the CBRAIN web platform for high-performance computing (RRID:SCR_005513)  26. As a result, any FSL FEAT analysis uploaded to NeuroVault or performed in CBRAIN can be exported as a NIDM-Results pack. NeuroVault also accepts NIDM-Results packs as a mean to upload new data to a collection. The nidmresults Python library (https://pypi. python. org/pypi/nidmresults) and the provenance MATLAB toolbox (http://www.
Publically available NIDM-results packs A set of 244 NIDM-Results packs has been made publically available on NeuroVault at http://neurovault. org/collections/1435/. Those packs describe the results of fMRI analyses performed at the subject (232 packs) and group (12 packs) levels on six datasets downloaded from OpenfMRI (RRID:SCR_005031)  40, 41  $ nidmfsl fsl_ds107_group 49 -g Control.
The image-based meta-analysis was performed by combining the contrast estimate maps, along with their standard error, in a third-level mixed-effects general linear model (GLM). Each NIDM-Results pack was    queried to retrieve the image data needed for the meta-analysis (i. e. , the contrast image and contrast standard error image) along with the analysis mask. The query used to extract these data is displayed in Fig. 5. The name of the corresponding contrast was associated to each map to allow for the selection of the appropriate contrast. The neuroimaging software package used for the analysis was also extracted in order to identify which study estimates would need re-scaling. Second, the contrast and standard error estimates were selected according to the contrast name, re-scaled if needed and combined in a mixed-effects GLM.
We chose PROV as it is lightweight, focused only on provenance, and is easily extensible to provide domain-specific knowledge. Another recent effort to provide structured organisation of neuroimaging data is the Brain Imaging Data Structure (BIDS)  51. While NIDM-Results and BIDS both concern the organisation and description of neuroimaging data, they operate at very distinct domains of the analysis path. BIDS provides a mechanism for organising only the original raw data, and it does not cover any derived data nor the definition of particular statistical models or the outputs of those models. NIDM-Results, in contrast, works at the other end of the analysis pipeline, defining a framework to describe the statistical model, the statistical 'contrasts' that interrogate the model, and the resulting statistical maps and inferences obtained from each contrast.
Whereas BIDS was designed so that an end-user could manually create the files and directories of a BIDS-compliant data structure, NIDM-Results is intended to be automatically generated from analysis software and was therefore created using more expressive semantic web technologies. Of the larger NIDM project, it is the NIDM-Experiment portion that will have the greatest overlap with BIDS. By making the experimental metadata available as linked data, NIDM-Experiment will enable querying across the full neuroimaging data lifecycle, interrogating data possibly hosted on distributed resources. NIDM-Results is based on RDF and semantic web technologies. While a number of ontologies have been developed in relation with neuroimaging (e. g. , Cognitive Atlas 44 , CogPO 46 , OntoNeurolog  52  ), the use of controlled vocabularies and of linked data is not yet common practice in our community.
As more and more data become available online and as standardisation effort like the RII develops, we believe that these technologies will become more widespread. RDF was chosen as a basis for NIDM for the expressivity of its graph-based structures, the possibility to form intricate queries across datasets  37  , as well as for the extensibility of the created data models and the possibility to interconnect across knowledge domains (cf. ref. 53 for a review). One limitation of NIDM-Results is that only limited provenance is represented. For instance, computational environment, which has been shown to be source of undesired variability in neuroimaging results  54  , is not part of our model. NIDM-Results is part of a broader effort (NIDM) that aims at representing different levels of provenance in neuroimaging experiments.
A wider ecosystem is also under development (including a standalone viewer). Future work will focus on developing extensions for NIDM-Results to cover a larger spectrum of neuroimaging studies (e. g. , non-parametric analyses) as well as to stay up-to-date with emerging best practices. We will also sustain our effort on the development of tools that can read and write NIDM-Results. Finally we will focus on the specification of the other NIDM components to enable modelling of a complete fMRI experiment from raw data to statistical results. We believe NIDM-Results is an essential tool for the future of transparent, reproducible science using neuroimaging. If all research publications were accompanied by such a machine-readable description of the experiment, debates on the exact methodology used would be compressed or eliminated, and any replication efforts greatly facilitated.
Scope of the model NIDM-Results focuses on mass-univariate models based on a General Linear Model (at the subject or group level). To facilitate adoption, we restricted the scope of NIDM-Results to metadata that could be automatically extracted with limited user input, motivated by the specific metadata that is crucial for the application of meta-analysis. This had important practical consequences. Given that pre-processing and statistical analysis are sometimes done using separate pipelines, we focused on the statistical analysis only. The concepts to be represented in NIDM-Results were selected based on (1) meta-analysis best practices; (2) published guidelines to report fMRI studies  8  , and (3), in an effort to ensure continuity with current practice, we also considered the elements displayed as part of results reporting in different neuroimaging software (e.
Term re-use and definitions For each piece of information, we checked if an appropriate term was available in publicly available ontologies: in particular STATO for statistics term, PROV for provenance, NeuroLex for neuroscience terms, RRID for tools and also, to a lesser extent, Dublin Core, the NEPOMUK file ontology and the Cryptographic Hash Functions vocabulary. Namespaces of the re-used ontologies are provided in Table  2. More details on the re-used vocabularies are provided below. PROV. The W3C specification PROV  35  defines three types of objects: an Activity represents a process that was performed on some data (e. g. , a voxel-wise inference) and occurred over a fixed period of time; an Agent represents someone (human, organization, machine…) that takes responsibility for an activity (e.
g. , the SPM software) and, finally, an Entity represents any sort of data, parameters etc. that can be input or output of an activity (e. g. , a NIfTI image). PROV also defines a set of relations between those objects (e. g. , a voxel-wise inference Activity used a NIfTI image Entity; a voxel-wise inference Activity was associated with the SPM Agent and another NIfTI image Entity was generated by the segmentation Activity). NIDM-Results terms were defined as specialisations of PROV terms. STATO. GLM analyses of fMRI data rely on well-known statistical constructs (e. g. , one-sample T-test, two-sample T-test, F-tests, ANOVA, inference, ordinary least squares estimation, etc. ). The general-RDFlib library  72  to query the documents. The paragraph that was generated could, for instance, be used as part of the method section in a research paper.
Acknowledgements We gratefully acknowledge  Matthew Webster ,  Paul McCarthy ,  Eugene Duff  and  Steve Smith , from the  FMRIB  and  Robert Cox  and  Ziad Saad  from the  NIH , for their inputs on the integration of NIDM-Results within FSL and AFNI; as well as the NIDASH task force members for their inputs during the development of the model. We also gratefully acknowledge the  Tracey group at FMRIB  for sharing their pain datasets used in the meta-analysis. The INCF supported and organised the task force meetings in which the model was discussed. A. B. , C. M. and T. E. N. were supported by the  Wellcome Trust  [ 100309/Z/12/Z ]. T. A. was supported by the  Medical Research Council (United Kingdom)  [ MC-A060-53114 ]. BNN was supported by  NIH  grants [ AA012388 ,  AA021697 ,  AA021697-04S1 ].
1 Tween TBS for 2 h at room temperature. The membranes were washed after primary and secondary antibody incubation with 0. 3% Tween in PBS (p62), 0. 1% Tween in TBS (LC3-I/LC3-II) or 0. 05% Tween in PBS (alpha-Tubulin). After secondary incubation, the bands were visualized using Immobilon Western Chemiluminescent HRP Substrate (Millipore, Billerica, MA, USA) and an ImageQuant RT ECL system (GE Healthcare, Little Chalfont, UK). The band quantification was performed using ImageJ software (National Institutes of Health, Bethesda, MD, USA) and the band intensities of p62 and LC3-I/LC3-II were normalized against alpha-Tubulin.
The cells were photographed using fluorescence microscopy (Axio Imager with ApoTome. 2 (Carl Zeiss, Oberkochen, Germany) with Carl Zeiss ZEN imaging software) within 24 h. The samples were masked from the photographer. At neutral pH, both of the fluorophores fluoresce, indicating autophagosome presence (yellow), when a lysosome gets fused to an autophagosome during autolysosome maturation, the result is a lower pH and quenching of the GFP signal, which is seen as red fluorescence only (autolysosome). Autophagosome and autolysosome dots were then counted using ImageJ software.
Statistical Analyses Data are presented as mean ˘standard deviation (SD) (n = 3) for the Western blotting and cell viability experiments, and as mean ˘standard error of the mean (SEM), (n = 9-12) for the fluorescence microscopy experiment. Statistical analyses were done using IBM SPSS Statistics software version 21 (IBM, Armonk, NY, USA). All of the data were subjected to one-way analysis of variance (ANOVA), followed by Tukey's test for multiple comparison (p values < 0. 05 were considered significant).