dataesr/scientific-paragraphs-categorization huggingface dataset:
Plate Hole DIC Challenge The first set of input images was selected from the Society for Experimental Mechanics (SEM) 2D-DIC challenge simulated datasets  [38, 39]  (Bethel, CT, USA). The DIC challenge provides common image datasets that can be used to validate and improve both commercial and academic DIC software solutions. Specifically, in order to make a comparison between py2DIC, Ncorr, Vic-2D and DICe, the "plate hole" sample images (Sample 12) from SEM DIC challenge were analysed. Sample 12 is an experimental set of 12 images (resolution of 400 × 1040 pixels) of a steel plate with a hole in the middle being loaded in tension (Figure  3 ). Regarding the specimen, a painted speckle pattern was used during the tensile test. As this is an experimental image series, there are no ground truth data; this is the reason why it is useful for round-robin type tests where different codes for DIC implementation are compared  [39].
Results and Discussion py2DIC was validated using the two mentioned sets of images which were processed to obtain horizontal (u) and vertical (v) displacement fields. For the first image set, also the Green Lagrangian strain fields (E xx , E xy and E yy ) were computed. Furthermore, the same sets of images were used as input for both Ncorr, Vic-2D and DICe software applications. The differences between the raw displacements (not smoothed) computed with py2DIC and those obtained with each of the compared software were also calculated as follows: ∆u = u py2DIC i,j -u Vic-2D/Ncorr/DICe i,j ∆v = v py2DIC i,j -v Vic-2D/Ncorr/DICe i,j (3) where i and j are the row and column pixel locations, respectively. Referring to displacement differences ∆u and ∆v (Equation (  3 )), a global approach was performed for all the points of the grid, considering the following standard statistical parameters in order to summarize the error of full-field displacements: • ∆u ∆v: mean value of the horizontal and vertical displacement differences • ∆u ∆v: median value of the horizontal and vertical displacement differences • Std.
Plate Hole DIC Challenge Displacement Field Comparison The first and the last image of the experimental set with a resolution of 400 × 1040 pixels were processed using a local approach for all the software applications. For this set of experimental images, the horizontal and vertical displacement values were calculated in pixels, as no conversion pixels-to-unit length was available. py2DIC results were calculated at every pixel of the considered images, using a 11 × 11 pixel template, a step size of 1 pixel and oversampling factor of 20 and 10 for the u and v displacement computation respectively. Firstly, the py2DIC and Ncorr comparison was performed; the obtained displacement fields are shown in Figures  5  and 6. Ncorr returned the displacements on a 2-pixel spaced grid. Regarding this comparison, the results are in reasonable agreement with each other.
The obtained displacement fields are shown in Figures  5  and 6 , whereas Table  4  reports the statistics of the differences among the displacements computed with the two software applications. Quite similar results to the previous comparison were obtained in this case too, together with the slightly worse behaviour of py2DIC on the vertical displacements. Furthermore, it is highlighted in Figure  6  that Vic-2D is not able to compute the displacements near the hole and this is probably due to the handling of the Nan values. Thirdly, py2DIC results were compared with those obtained using DICe in terms of displacement fields (Figures  5  and 6 ) and statistical analysis (Table  5 ). DICe returned the displacements at every pixel of the considered images, as for py2DIC. Again, the quite low mean differences highlight the absence of significant systematic errors in both the directions; the normal distribution of the differences and the slightly worse behaviour of vertical displacements are also confirmed in this test.
Plate Hole DIC Challenge Strain Field Comparison Starting from the obtained displacement fields, the Green Lagrangian strains E xx , E xy , E yy were computed using the open source software py2DIC, Ncorr and DICe. Regarding py2DIC, in order to reduce the noise, both Gaussian and spline smoothing methods were applied and then the convolution procedure was performed. As for Ncorr, a strain radius equal to 5 pixels was chosen to define a local group of displacement data points and perform a least squares plane fit, as proposed in Reference  [40]. This method is used to estimate the plane parameters and retrieve the displacement gradients and the subsequent Green Lagrangian strains. Finally, a gauge size of 15 pixels was used to define a displacement subset for DICe strain computation. The obtained E xy and E yy strain fields are reported in Figures  8  and 9.
Conclusions and Prospects py2DIC, a free and open source 2D DIC software was developed at the Geodesy and Geomatics Division of the Sapienza University of Rome. The potentialities of the developed software were investigated by processing two different sets of experimental images and comparing the results with those supplied by the three well known software packages, one commercial (Vic-2D) and two free and open source (Ncorr and DICe) and with those coming from independent measurements by a strain gauge fixed on one of the samples. The first set of images was selected from the SEM DIC challenge simulated datasets; the second one was acquired during an experimental campaign performed in collaboration with the Lab of Structural Engineering of the Sapienza University of Rome and it is available at http://github.
com/Geod-Geom/py2DIC/tree/master/LabTest together with the strain gauge measurements. The obtained results point out the very good py2DIC performances in successfully characterizing the deformation mechanism of material subjected to mechanical loading: the new software is indeed able to estimate displacements with an agreement at the level of few hundredths of a pixel or few microns; also, the compared strain fields are globally consistent, even if in this case only a graphical comparison was possible, since each software adopts a proper grid. In conclusion, py2DIC represents a reliable free and open source alternative software for 2D DIC applications in the field of solid mechanics. In the future, this research project will aim to investigate the potentialities of py2DIC outside the elementary lab conditions, in order to bring the methodology and the software at a full 2D operational level in the widest practical cases, releasing as much as possible the acquisition constraints related to the testing setup.
A wealth of analysis tools are available to fMRI researchers in order to extract patterns of task variation and, ultimately, understand cognitive function. However, this "methodological plurality" comes with a drawback. While conceptually similar, two different analysis pipelines applied on the same dataset may not produce the same scientific results. Differences in methods, implementations across software, and even operating systems or software versions all contribute to this variability. Consequently, attention in the field has recently been directed to reproducibility and data sharing. In this work, our goal is to understand how choice of software package impacts on analysis results. We use publicly shared data from three published task fMRI neuroimaging studies, reanalyzing each study using the three main neuroimaging software packages, AFNI, FSL, and SPM, using parametric and nonparametric inference.
We obtain all information on how to process, analyse, and model each dataset from the publications. We make quantitative and qualitative comparisons between our replications to gauge the scale of variability in our results and assess the fundamental differences between each software package. Qualitatively we find similarities between packages, backed up by Neurosynth association analyses that correlate similar words and phrases to all three software package's unthresholded results for each of the studies we reanalyse. However, we also discover marked differences, such as Dice similarity coefficients ranging from 0. 000 to 0. 684 in comparisons of thresholded statistic maps between software. We discuss the challenges involved in trying to reanalyse the published studies, and highlight our efforts to make this research reproducible.
choice of analysis pipeline, operating system, and even software version may influence the final research outcome of a study. The extent to which varying processing conditions can lead to discrepancies in observed results has been highlighted throughout the neuroimaging literature. In a study examining the use of FreeSurfer to measure the cortical thickness and volume of structural brain images  (Gronenschild et al. , 2012) , a change in software version was shown to lead to increases of over 10% in observed anatomical measurement; a switch in workstation from which the software was run also manifested significant deviations in the final result. In related work  (Glatard et al. , 2015) , changes in operating system lead to differences in the results of an independent component analysis of resting state fMRI data carried out using FSL.
Here, disparities in both the number of components determined as well as information between matched components were found when the analysis was conducted on two separate computing clusters. For task-based fMRI, the impact of methodological choices has been investigated extensively. Choices for each individual procedure in the analysis pipeline (e. g. , head-motion regression  (Lund et al. , 2005) , temporal filtering  (Skudlarski et al. , 1999) , and autocorrelation correction  (Woolrich et al. , 2001) ) alongside the order in which these procedures are conducted  (Carp, 2013)  can deeply influence the final determined areas of brain activation. In perhaps the most comprehensive of such studies  (Carp, 2012a) , a single publicly available fMRI dataset was analysed using over 6,000 unique analysis pipelines, generating 34,560 unique thresholded activation images.
These results displayed a substantial degree of flexibility in both the sizes and locations of significant activation. In combination, these examples of research shape a sombre picture for the possibility of study reproducibility. While each of the aforementioned studies investigated the effect of either software version, operating system, or analysis pipeline on analytic variability, the choice of software package for carrying out the analysis remained fixed in each study. This is despite a vast array of analysis packages that are now freely available to researchers. The three most popular of these packages for fMRI data analysis are AFNI (RRID:SCR_005927;  [Cox, 1996] ), FSL (RRID:SCR_002823;  [Jenkinson et al. , 2012] ), and SPM (RRID:SCR_007037;  [Penny et al. , 2011] ). While SPM is the oldest, FSL has grown in popularity and together the three packages have been estimated to account for 80% of published functional neuroimaging results  (Carp, 2012b).
Although there are differences in how each software package models and processes data, the analysis framework for task fMRI-now a mature research area-is expected to be similar across software, and hence the results yielded from each package should be comparable. We therefore seek to answer the question: How much of the variability in neuroimaging results is attributable to the choice of analysis software package? In this work we reanalyse data from three published neuroimaging studies using each of the three main software packages and quantify differences in the results. We choose three publications with data that have been made publicly available on the OpenfMRI database (RRID: SCR_005031, http://openfmri. org; [Poldrack et al. , 2013] ), recently relaunched as OpenNeuro (http://openneuro.
| Study description and data source We selected three functional fMRI studies for reanalysis from the publicly accessible OpenfMRI data repository: ds000001 (Revision: 2. 0. 4; [Schonberg et al. , 2012] ), ds000109 (Revision 2. 0. 2; [Moran et al. , 2012] ), and ds000120 (Revision 1. 0. 0; [Padmanabhan et al. , 2011] ). Each of the datasets have been organised in compliance with the Brain Imaging Data Structure (BIDS, RRID:SCR_016124;  [Gorgolewski et al. , 2016] ). These datasets were chosen following an extensive selection procedure (carried out between May 2016 and November 2016), whereby we vetted the associated publication for each dataset stored in the repository. We sought studies with simple analysis pipelines and clearly reported regions of brain activation that would be easily comparable to our own results.
A total of 48 subjects were scanned, although 43 had acceptable data for the false belief task-29 younger adults and 14 older adults. In this task participants listened to either a "false belief" or "false photo" story. A false belief story would entail an object being moved from one place to another, with certain characters witnessing the change in location while others were unaware. False photo stories were similar except involved some physical representation, such as a photo of an object in a location from which it had been subsequently removed. The task had a block design where stories were represented for 10 s, after which participants had to answer a question about one of the character's perceptions of the location of the object. We reproduce the contrast map of false belief versus false photo activations for the young adults, corresponding to Figure  5a  and Table  3  from the original publication.
| Data analyses All data analyses were conducted using AFNI (version AFNI_18. 1. 09), FSL  (version 5. 0. 10), and SPM (version SPM12, v6906). Computation was performed on a cluster comprised of 12 Dell PowerEdge servers (6 R410, 12 core 2. 40 GHz processors, 6 R420, 12 core 2. 80 GHz processors) running CentOS 7. 3.
| Common processing steps A number of processing steps for each package were included in all of our analyses, regardless of whether they had been implemented in the original study. While this meant deviating from an exact replication of the original pipeline, these processing steps were either fundamental to ensure that results from each software package could be compared objectively, or steps that are widely accepted as best practice within the community. In this section we describe these steps. Successful coregistration of the functional data to the structural brain images-and subsequently-registration to the MNI template, was of paramount importance to us for fair comparability of the results. During our first attempt at analysing the ds000001 dataset we discovered that seven subjects had essential orientation information missing from the NIfTI header fields of their functional and structural data.
As the source DICOM files were no longer available, the original position matrices for this dataset were unable to be retrieved. This caused coregistration to fail for several subjects across all three software packages in our initial analysis of this data. We rectified the issue by manually setting the origins of the functional and structural data. OpenfMRI released a revision (Revision: 2. 0. 4) of our amended dataset which we used for the analysis. Further to this, we also set a number of common preprocessing steps within each package to be applied in all our analyses. Firstly, brain extraction was conducted on the structural image in all software. We did this to improve registration and segmentation. In AFNI, brain extraction was carried out using 3dskullstrip, that was called implicitly from within the @SSwarper program.
The skullstripped anatomical volume obtained here was inputted into our afni_proc. py scripts where further preprocessing and first-level analyses were carried out. In FSL, brain extraction was performed on both the functional and structural data. The Brain Extraction Tool (BET;  [Smith, 2002] ) was applied to each structural image from the command line before preprocessing, and for functional data with the BET option within the Prestats module of FEAT. In SPM, brain extraction was implemented via the segmented structural images. Grey matter, white matter and CSF images were summed and binarised at 0. 5 to create a brain mask, which was applied to the bias corrected structural image using the Image Calculator. Coregistration of the functional data to the anatomy was carried out for the most part using the default settings in each software.
In AFNI, alignment of the data was conducted using the align_epi_anat. py program called implicitly from the align block within the afni_proc. py scripts. We included the -volreg_align_e2a option within our scripts to specify alignment of the functional data onto the anatomy, as by default AFNI conducts the inverse transformation of anatomy onto functional. Further to this, we also added the -align_opts_aea program to all of our scripts with the -giant_move and -check_flip options to allow for larger transformations between the images. In FSL, coregistration was carried out within FEAT using the default linear registration methods with a Boundary-Based Registration (BBR) cost function. The default methods were also applied within SPM's Coregister: Estimate module, using a normalised mutual information cost function.
Registration of the structural and functional data to the anatomical template was executed using each packages nonlinear settings. In AFNI, nonlinear registration of the anatomical data to the MNI template was conducted as part of the @SSwarper program ran prior to the afni_proc. py script. The warps computed by @SSwarper were passed to afni_proc. py using the -tlrc_NL_warpred_dsets option, and applied to the functional data within the tlrc block using the -volreg_tlrc_warp option. By default, the resampled functional data in MNI space has voxel size determined from the raw 4D data; we forced 2 mm cubic voxels with the -volreg_warp_dxyz option for compatibility with FSL and SPM's 2 mm default. In FSL, registration to the MNI template was conducted using FMRIB's Nonlinear Image Registration Tool (FNIRT;  [Andersson et al.
, 2007] ), controlling the degrees of freedom of the transformation with a warp resolution of 10 mm. In SPM, the nonlinear deformations to MNI space were obtained as part of the Segment module and then applied to the structural and functional data within the Normalise: Write module. As a form of quality control, we created mean and standard deviation images of the subject-level MNI-transformed anatomical and mean functional images. Alongside the subject-level data, these images were assessed to check that registration to MNI space had been successful. When intersubject registration failed remedial steps were taken within each software; these are described in the software implementation parts of the following study-specific analysis sections. Across all software packages six motion regressors were included in the analysis design matrix to regress out motion-related fluctuations in the BOLD signal.
Use of six or more derived motion regressors is commonly recommended as good practice, and we chose to use just six regressors as this could be easily implemented across software. Finally, we note that each software package uses a different default connectivity criterion for determining significant clusters: 6-connectivity for AFNI, 18-connectivity for SPM, and 28-connectivity for FSL. Since these settings are not typically modified we have kept these defaults in all of our analyses to reflect standard practices carried out within each software. We now describe the task-specific analysis procedures for each of the three studies as carried out in the original publications, and how these methods were implemented within each package. While we decided to keep the above steps of the analysis pipelines fixed, for all remaining procedures we attempted to remain true to the original study.
| ds000001 analyses In the publication associated with the ds000001 study all preprocessing and analysis was conducted within FSL  (version 4. 1. 6). Data on all 16 subjects were available to us on OpenfMRI. In the original preprocessing, the first two volumes of the functional data were discarded and the highpass-filter was set to a sigma of 50. 0 s. Motion correction was conducted using MCFLIRT and brain extraction of the functional data was applied with BET, after which FSL's standard three-step registration procedure was carried out to align the functional images to the structural scan. Spatial normalisation was implemented with FMRIB's Linear Image Registration Tool (FLIRT;  [Jenkinson et al. , 2002] ), and data were smoothed using a 5 mm full-width-half-maximum (FWHM) Gaussian kernel.
AFNI implementation Using our default procedure for the AFNI analysis, we found that coregistration of the functional scans onto the anatomy failed for four subjects. To remedy this issue, for this study we modified our afni_proc. py scripts: Within the -align_opts_aea module, the "-ginormous move" option was added to align centers of the functional and anatomical volumes, and the "-cost lpc + ZZ" option was used to apply a weighted combination of cost functionals. Both of these changes are recommended for data with little structural detail. Following these modifications all coregistrations were successful. To replicate the orthogonalization methods from the original study, a separate orthogonalization script was ran for each subject prior to the first-level analyses. Within this script, the (un-orthogonalized) regressors were generated by passing the event timing files to 3dDeconvolve, after which the 3dTproject command was used to obtain the desired projections.
| ds000109 analyses The original preprocessing and statistical analysis for the ds000109 study was carried out using SPM8. Data were shared on 36 of the 40 subjects, 21 of which were young adult subjects that had fMRI data compatible for our reanalysis. First, functional data were realigned and unwarped to correct for head motion and geometric distortions. After transforming the data into a standardised space, the normalised data were smoothed with an 8 mm FWHM Gaussian kernel. Further to this, custom software was applied to exclude functional volumes where head motion had exceeded a certain limit; however, this process was omitted from our pipelines since this feature was not available in any of the software packages. The preprocessed data were entered into a GLM for first level analysis where trials were modelled using a block design and convolved using SPM's canonical HRF.
The degree of disagreement is therefore evaluated by the perpendicular distance of points from the x-axis; for example, for a "AFNI-FSL" Bland-Altman plot, points above x-axis are where AFNI's statistic is larger than FSL's. With the difference plotted against the average, general patterns of disagreement can be discerned. In addition to this, we also created Bland-Altman plots to compare percentage BOLD change maps (for ds000120, partial R 2 maps) between software. For each package, an appropriate normalisation of the grouplevel beta maps was conducted to convert to percentage BOLD change units. Due to differences in how each package scales the data, a different normalisation was required for each of the three packages. For ds000120, the partial R 2 maps were computed via a transformation of the group-level F-statistic images.
| Permutation test methods For ds000001 and ds000109, in parallel to our replication analyses we computed an additional set of group-level results applying nonparametric permutation test inference procedures available within each software package (a one-sample repeated measures permutation test needed for ds000120 was not available in AFNI). The first level contrast maps obtained from our initial replications for each subject were entered into a group-level one-sample T-test where clusterwise inference was conducted using the same cluster-forming thresholds, and then 5% level FWE corrected thresholds were computed by permutation, using 10,000 permutations. F I G U R E 1 (a).