dataesr/scientific-paragraphs-categorization huggingface dataset:
A. Perturbation Induced Differences Fig. 2  shows the percentage deviation for each simulation mode on 10 subjects. Introduced perturbations show highlyvariable changes in resulting connectomes across both the perturbation model and subject, ranging from no change to deviations equivalent to difference typically observed across subjects. For the 10 subjects tested, we see that the Pythoninstrumented MCA and RR pipelines resulted in the largest deviation from the reference connectome. In these cases we also see that the results are modal, where each subject has discrete states that may be settled in, some of which result in deviations comparable to subject-level noise. This modality is likely due to minor differences introduced at crucial branchpoints which then cascaded throughout the pipeline.
C. Structural Properties of Introduced Perturbation While the case investigated above notably showed a significant degradation of regional signal quality for Full Stack RR noise in a single subject, Fig. 4  explores the relative change in connectivity from the reference for each perturbation mode and subject. Edges in the presented graphs are weighted by their standard deviation across all simulations for that participant, and coloured as positive or negative deviations based on whether the mean weight for all simulations was greater or lower than the reference weight, respectively. All edges with a standard deviation of 0 across all simulations were greened out for clarity. For the Python instrumented MCA and RR implementations, edge weight was generally inflated non-specifically for existing edges in the reference connectome for all subjects.
The Full Stack RR implementation shows significant variability across subjects, where the number of affected edges ranges from none to all. In each case where there exists some deviation, intensities appear to be spatially linked, suggesting the differences may be due to variable quality in the underlying data. In this case, Monte Carlo Arithmetic may have served to shed light on poor signal-to-noise properties present within regions of the images being modelled. For 1-voxel noise, the differences introduced across independent injections impacted a larger portion of edges than As evaluated by the percent deviation from reference in the Frobenius Norm of a resulting connectome, each of the 10 processed subjects were re-processed 100 times for each perturbation method. We see that the MCA and RR (Python) methods resulted in distinct modes for the outputs in all cases reaching extreme deviations equivalent to cross-subject variation.
D. Aggregation Across Simulations For each simulation method there existed a graph nearly identical to the reference, but the variability introduced by these simulations were highly variable both in terms of the method of perturbation used and the dataset being processed. The aggregation of the simulated graphs into a consensus graph allows features of this variation to be encoded implicitly in connectomes which may be used for downstream analyses. Fig. 5  shows the relative percentage of added and missing edges for each setting across all subjects using a variety of such aggregation methods. By aggregating the simulated connectomes in a variety of methods, the resulting edges would be a product of applying some filter to the set of observed edges, and succinctly represented in a single graph.
The Python MCA and RR modes show a slight increase in computation time to the reference task, whereas the Full Stack version approaches a nearly 7× slowdown, on average. This discrepancy further supports the hypothesis stated above that floating point logic implemented directly in Python, without the use of Numpy or external libraries, account for a minor portion of the total floating point operations. As Verificarlo has been shown to increase the runtime of floating point operations by approximately 100×  [10] , this result suggests that the pipeline evaluated here is largely I/O limited. In the case of 1-voxel perturbations, we see a slowdown approximately equivalent to that of the Python instrumentation, not exceeding a 2× increase. Across all executions approximately 2000 CPU hours were consumed.
IV. DISCUSSION We have demonstrated through the application of multiple perturbation methods how noise can be effectively injected into neuroimaging pipelines enabling the exploration and evaluation of the stability of resulting derivatives. These methods operate by either perturbing the datasets or tools used in processing, resulting in a range of structurally distinct noise profiles and distributions which may each provide value when exploring the stability of analyses. While 1-voxel noise is injected directly into the datasets prior to analysis, MCA and RR methods iteratively add significantly smaller amounts of noise to each operation performed. In the case of partial (Python) instrumentation with MCA and RR, distinct and considerably distinct modes emerged in all tested subjects. We hypothesize that software branching likely played a role leading to this unexpected result.
As the majority of numerical analysis in Python is traditionally performed using the Numpy library, and therefore BLAS and LAPACK, it is possible that the error introduced by Python was allowed to cascade throughout the pipeline without correction, until the next Python branch point occured and this repeated, eventually growing to the often subjectlevel differences observed. These modes would then be the result of a small number of instrumented numerically-sensitive operations, leading to a bounded set of possible outcomes of an otherwise deterministic process. It is possible that these distinct modes could serve as upper-bounds for the deviation due to instabilities within a pipeline, and is an area for further exploration. Future work will also more closely instrument libraries with functionality that will enable the identification of crucial branch points, as this functionality is already present within Verificarlo.
The identified crucial branch points could be leveraged for the re-engineering of pipelines with more stable behaviour, and potentially shed light on new best practices. An exciting application of MCA and RR (Python) analyses in cases where pipeline modification is not feasible is the generation of synthetic datasets. Using each mode or an aggregated collection of modes as samples in the MCAboosted dataset could potentially increase the statistical power of analyses for datasets which may suffer from small samples, or be used to increase the robustness of derivatives by bagging Perturbation introduced structural differences. The variance of each edge is shown relative to the reference edge strength, and coloured either red or blue based on the mean perturbed weight was higher or lower than that of the reference, respectively.
Edges which experienced no variation were coloured as green to be distinct from all edges which experience any variation. the results using an appropriate averaging technique for the simulated derivatives. While the Python instrumentation with MCA and RR resulted in derivative modes, the Full Stack instrumentation with RR produced a continuous distribution of derivatives which were often less distinct from the reference results. Extending the hypothesis posited above, this continuous set of results may be due to a law of large numbers effect emerging when performing a considerable number of small perturbations, leading to a normalized error distribution and effectively a self-correction of deviations. Future work will test this hypothesis and consider the relationship between the fraction of instrumented floating point operations and modality, as well as through the incremental profiling and evaluation of tools for the comparison of intermediate derivatives and their deviation from a reference execution.
These experiments have the potential to provide more insight into the origin of instabilities in scientific pipelines and identify optimization targets. As the significance of RR (Full Stack) perturbation was highly variable across participants, this technique could also be used for automated quality control, flagging high-variance subjects for further inspection or exclusion from analyses. From the top level, inspecting the regional degradation of signal across these perturbations as shown in Fig. 3 , researchers could lead a targeted interrogation of their raw datasets to identify underlying causes of signal loss. Conversely, investigating which low-level BLAS operations contribute to the observed instabilities will allow researchers to clarify the link between ill-conditioning and so-called bad data directly within their pipelines.
Upon characterizing this relationship it would be valuable to identify the point (if any) at which targeted N-voxel perturbations become equivalent to MCA-induced variability. The differences observed when performing 1-voxel perturbations were often comparable in magnitude to the variation introduced across Operating Systems. As OS noise is not controlled and may differ greatly among distributions, package updates, etc. , it is likely an insufficiently descriptive evaluation method, and should be used as a reference alongside others. The level of control made available through 1-voxel perturbations in terms of both locality and strength of noise makes it a flexible option that could potentially be used to target known areas of key importance for subsequent analyses. Due to the fact that these perturbations introduce a minor change to input images, this method could also be used for estimating global pipeline stability in a classical sense (i.
e. conditioning). While each of the perturbation modes showed distinct differences with respect to the magnitude and continuity of their induced deviations, Fig. 4  illustrates that the structure of these deviations was also highly variable across both perturbation method and data. This suggests different applications and use cases for each perturbation method. While MCA and RR Python implementations impact connectomes globally, these could be applied to generate synthetic datasets. Full Stack RR mean percentile (0) percentile  (10)   Fig. 5. Gain and loss of edges in aggregation of simulations. The relative gain and loss of edges is shown for each aggregation method and perturbation method in terms of binary count. The volatile aggregation is the difference between percentile (100) and percentile (0) aggregates, and is contains all edges which do not appear in every graph.
The volatile set of edges for each of MCA (Python), RR(Python), RR (Full Stack), 1-voxel (independent), and 1-voxel (single) contain 2. 5%, 2. 5%, 18. 5%, 43. 0%, and 1. 7% of the number of edges found in the reference, respectively. In the worst case, 1-voxel (independent), this means that the existence of nearly half the edges in the graph fail to have consensus across the simulations. is highly variable with respect to dataset, suggesting possible applications in quality control, granted further work is performed to more fully understand the effect observed between this and the Python-only case. Both 1-voxel methods add noise locally, and can test the sensitivity of specific pipeline components or regions of interest to variation. Other methods, such as automatic differentiation, could also be explored as possible avenues leading towards an understanding of the endto-end conditioning of pipelines.
In addition to generating unstable derivatives which could be looked at or analyzed independently, this type of perturbation analyses enables the aggregation of derivatives. As is summarized in Fig. 5 , the method by which graphs or Relative Mean Time per Simulation Fig. 6. Computation time for each perturbation method. Shown in relative time to the reference execution, plotted is the average execution time for the perturbation methods. MCA and RR (Python) have a small increase in computation time per run, as few floating point operations were instrumented in these settings. The RR (Full Stack) method has nearly a 7× slowdown. In this case, all floating point operations were instrumented, but the slowdown of less than the estimated 100× would suggest that the bulk of computation time is not spent on floating point arithmetic.
While this slowdown is observed for the the Full Stack implementation, the Python-only implementation is negligibly slower than the reference execution, suggesting that even fewer of the floating point logic is directly written in Python. The slowdown in the 1-voxel setting is of a similar scale to that of the Python-only implementation, with the slowdown likely caused by the addition of 2 read and 1 write operations to the pipeline's execution (reading of simulation parameters and original image, application of simulation, and subsequent writing of perturbed image to temporary storage). Note that the figures shown in Fig. 6  are for a single simulation, and real relative CPU time in each case would be 100× larger for the experimental application of these methods. The work presented here demonstrates that even low order computational models such as a 6-component tensor used in diffusion modelling are susceptible to noise.
V. CONCLUSION All pipeline perturbation methods showed unique nonzero output noise patterns in low-order diffusion modeling, demonstrating their viability for exploring numerical stability of pipelines in neuroimaging. MCA and RR (Python) instrumented pipelines resulted in a wide range of variability, sometimes equivalent to subject-level differences, and are recommended as possible methods to estimate the lower-bound of stability of analyses, generation of synthetic datasets, and possible identification of Python-introduced critical branch points. RR (Full Stack) perturbations resulted in continuously distributed connectomes that were highly variable across datasets, ranging from negligible deviations to complete regional signal degradation. We provisionally recommend the use of RR (Full Stack) noise for automated quality control and identifying global pipeline stability.
We recommend a percentile approach to aggregation, where the threshold can be adjusted based on the desired robustness of the resulting graphs. An advantage of percentile approaches is also that composite aggregates can be formed, isolating edges based on their prevalence across simulations. Further exploration of the distribution of perturbed results should be performed to conclude on the relevance of the aggregation used, as the desired aggregate should be close to the expected value of the distribution. While both MCA and random-injection simulations are computationally expensive in that they require the evaluation of many simulations, they provide an opportunity to characterize processing modes that may emerge when analyzing either noisy datasets or unstable tools. This work also highlighted an important relationship between the noise properties of an incoming dataset and the tool, validating the need to jointly evaluate the stability of tool-dataset combinations.
ACKNOWLEDGMENTS Support for this research was provided by The  Natural Sciences and Engineering Research Council of Canada (NSERC)  (Award #:  CGSD3-519497-2018 ). This research was enabled in part by support provided by  Calcul Quebec  (http://www. calculquebec. ca) and  Compute Canada  (http://www. computecanada. ca). We would also like to thank  Dell and Intel  for their collaboration and contribution of computing infrastructure. The authors would also like to thank their reviewers for thoughtful and insightful comments and suggestions.
| Design & stimuli We constructed 200 sentences in French, each containing between 6 and 11 words (M = 7. 39; SD = 1. 01). The sentences had an average length of 42. 16 characters (including spaces; SD = 6. 29) and the average frequency of all words in the sentences was 5,233 occurrences per million which is equivalent to 6. 71 Zipf  (van Heuven, Mandera, Keuleers, & Brysbaert, 2014). Word frequencies were the film subtitle frequencies of the Lexique2 database  (New, Pallier, Brysbaert, & Ferrand, 2004). We manipulated the nature of the parafoveal word (n + 1) that was present at that location before the eyes left word n. Word n + 1 could either be the same word as word n or a completely unrelated word paired in length and in frequency (see Table  1 ). In both cases, the words were an illegal continuation of the sentence, and once participants' eyes crossed an invisible boundary between words n and n + 1, the word at location n + 1 was changed into a word that formed a syntactically correct continuation of the sentence.
A chin-rest was used to minimize head movements. The scalp electrical activity was recorded with the ActiveTwo BioSemi system from a 64-electrode head cap (Electro-Cap Inc. ) and positioned according to the 10-20 international system. Two additional electrodes (CMS/DRL) were used as an online reference (for a complete description, see  Schutter, Leitner, Kenemans, & van Honk, 2006). The montage included 10 midline sites and 27 sites over each hemisphere. Four additional electrodes were used to monitor eye movements and blinks (two placed at lateral canthi and two below the eyes), and two additional electrodes were used for an offline re-referencing (placed behind the ears on the mastoid bone). Continuous EEG was digitized at 1,024 Hertz. The EyeLink and BioSemi systems were jointly controlled using OpenSesame on the master computer which sent triggers to the EyeLink through an ethernet cable and to the BioSemi software via the parallel port.
| Preprocessing of eye movement data The raw data were preprocessed by EyeLink algorithms that detect saccades, fixations, and eye-blinks. We excluded trials on which blinks occurred during the fixation of the target word (0. 32%), and we only analyzed first-pass reading measures. That is, trials, where the target word was skipped during first-pass reading, were removed from the fixation duration analyses. The resulting output was then analyzed using scripts written in R data analysis software.
| Preprocessing of EEG data We used the EEGLAB toolbox (version 14. 1. 2b; Delorme & Makeig, 2004)  for MATLAB (version 2018b; The MathWorks) to preprocess the EEG data. In preparation for independent components analysis (ICA), the EEG data were initially downsampled to 500 Hz, re-referenced to the averaged mastoids, 6 and synchronized to the eye-tracking data using the EYE-EEG toolbox  (Dimigen et al. , 2011) , then filtered between 2. 5 and 100 Hz. Blinks, as detected by the eye-tracker, were removed Based on blinks and other ocular artifacts, 22. 65% of trials were removed. ICA was trained on this data set which over-weighted presaccadic potentials per the procedure in  Dimigen (2018). ICA otherwise used the default settings in EEGLAB. Separately, each data set was filtered between 0. 1 and 40 Hz.
The ICA weights from the corresponding training set were then applied to this set. The automatic component rejection was conducted according to the procedure set forth in  Plöchl, Ossandon, and König (2012)  as implemented in the EYE-EEG toolbox using the default threshold of 1. 1. An average of 1. 6 components corresponding to ocular artifacts were removed per participant. Following the component rejection, the data were separated into epochs of -100 to 800 ms post-onset of the fixation on the target word n, as well as for word n + 1. Epochs were then baseline corrected using the 100ms prefixation baseline. An additional 8. 76% of trials were removed due to residual EEG artifacts. This left 4,664 total trials included in the final analysis of the EEG time-locked to word n.
| EEG analyses We conducted a mass-univariate analysis using the clustermass permutation test in the Mass Univariate ERP Toolbox  (Groppe, Urbach, & Kutas, 2011)  in MATLAB. This test was run on the t-statistic for the difference between the averaged ERPs to the repetition and non-repetition conditions (different--repeated) for all 64 scalp electrodes. The time window for the test was 0-550 ms, and 2,500 permutations were used.
| Eye movement analyses For each eye movement measure (first fixation duration (FFD), gaze duration (GD), skipping rate, refixation rate, initial landing position [ILP]), we used LME analyses (for FFD, GD, and ILP) and GLME (for skipping and refixation rates) with main effects of condition (repeated parafoveal word, different parafoveal word), and included random intercepts for participants and items. All measures corresponded to firstpass reading. The averages values and 95% CIs were computed using R. All LMEs/GLMEs were implemented using the lme4 package  (Bates, Maechler, Bolker, & Walker, 2015)  also in R (R Core Team, 2013), and were compared using a likelihood ratio test using the ANOVA function in R. This test compares a full model fit with all parameters to a simplified one fit without the parameter to be tested to estimate whether that parameter provides an improvement in goodness of fit to the data greater than sampling error.
in developing new, more active washcoat, there is a lack of rigorous mathematical knowledge for the performance of these advanced washcoats, for which the reaction can dominate the diffusion and/or convection. We observe that the industry is still broadly using heuristically derived models, or models derived for the diffusion dominated reactive transport regimes. In many cases such models are postulated using the large experimental knowledge and the similarity with other processes. Obviously, such models have been validated (often after calibration) for many particular regimes and have played an important role in the design and optimization of catalytic filters. Special software tools like DETCHEM, www. detchem. com have been developed based on such models, and general purpose software tools like Comsol  Multiphysics, www.
804. For the validation, we simulate the time evolution of the microscale model on a geometry consisting of a repetition of   = 5 , 10 and 20 periodic cells in x-direction. Varying the number of the periodic cells in the row allows to study the convergence with respect to. Since the length of the macroscale domain is fixed to the unit interval, the length of the periodic cells is given by l mp = 1  Software and discretization The problems considered here, were solved using the commercial software tool COMSOL Multiphysics at Version 5. 1. The laminar flow model was used for the flow problem and the cell problems and the microscale reactive transport problems were solved using the coefficient form PDE module of COMSOL Multiphysics. Steady state problems were solved for the flow and for the cell problems.
Acknowledgements We are indebted to the anonymous reviewers for their comments and help to improve significantly the article. The work of O. I. has been supported by  BMBF  project  ML-MORE 05M20AMD. The research of A. M. was supported in part by the project  UPGEO  ⟨  ANR-19-CU05-032  ⟩ of the  French National Research Agency (ANR)  and by the  LABEX MILYON  ( ANR-10-LABX-0070 ) of  Université de Lyon , within the program " Investissements d'Avenir "  ( ANR-11-IDEX-0007 ) operated by the  French National Research Agency (ANR).
|--center01 | |--healthySubject01 | | `--month0 | |--healthySubject02 | | `--month0 | |--patient01 | | |--month0 | | `--month12 | `--patient02 | |--month0 | `--month12 `--center02 |--healthySubject01 | |--month0 | `--month12 |--healthySubject02 | |--month0 | `--month12 |--patient01 | |--month0 | `--month12 `--patient02 |--month0 `--month12 Each of the folders at the bottom of the hierarchy (here month0 or month12) contains a set of files with MRI volumes for the given subject (that may be, for example, either a healthy subject or a patient) at the given time of their follow-up, acquired in the given center. These volumes may consist of a set of raw data as well as post-processed data such as segmentation masks, co-registered volumes, etc. that are produced from image processing tools. Obviously, the same dataset could have been structured in many other ways.