dataesr/scientific-paragraphs-categorization huggingface dataset:
2. 3. 2. 1. Core sequences preprocessing pipeline. The structural module of ExploreASL v1. 0. 2, described in  (Mutsaerts et al. , 2020) , was used in combination with the Bayesian Model Selection (BaMoS) for WMH segmentation (Carole H. Sudre et al. , 2015)  to preprocess 3D T1 and 3D FLAIR images. Other 2D core sequences were only used for radiological assessment of patient eligibility and not preprocessed. Preprocessing began with the registration of the 3D FLAIR to the 3D T1w (rigid-body) and the 3D T1w to the MNI center of mass (rigidbody). WMH segmentation was computed using BaMoS (Carole H. Sudre et al. , 2015) , a hierarchical unsupervised model selection framework simultaneously accounting for healthy tissue and unexpected observations. The resulting WMH segmentations were then used with the Lesion Segmentation Toolbox (LST) v2.
0. 15  (Gaser 2009)  to fill these lesion areas on the T1w image, which can be present as hypointensities and affect subsequent segmentation and non-linear registration  (Schmidt et al. , 2019). Tissue segmentation was then performed with the Computational Anatomy Toolbox (CAT) 12 (r1363), which estimates and corrects the bias field inhomogeneity in 3D T1w images, and iteratively improves the non-linear registration to MNI standard space and the creation of partial volume maps of gray matter (GM), white matter (WM) and CSF. All the described transformations were combined in a single transformation, to avoid multiple interpolations. ASL were handled by different preprocessing submodules and underwent three common steps: geometric distortion correction, motion correction, and registration with the structural reference image T1w.
D) dMRI Tensor Fitting. In the dMRI preprocessing submodule the registration output was fed into the FSL Brain Extraction Toolbox (BET) (Stephen M. Smith 2002)  and then into FSL DTIFIT, to fit the diffusion tensor model to the data and produce diffusion tensor imaging (DTI) scalars maps (fractional anisotropy (FA), and mean (MD), axial (AD) and radial (RD) diffusivity).
Semi-automatic QC To control the quality of the EPAD imaging cohort, we created an inhouse workflow to perform semi-automatic QC of MRI data. This set of QC functionalities was written as an extension to ExploreASL called ExploreQC. The semi-automated QC procedure was based on two steps: feature estimation and visualization (Fig. 3 ). ExploreQC code availability and software specifications are listed in section 4 of the supplementary material. 2. 3. 3. 1. QCfeature estimation. Image quality features were computed from five image feature domains: motion, noise, inhomogeneity, asymmetry, and descriptives, in line with recent MRI QC studies  (Esteban et al. , 2017a,b; Fidel Alfaro-Almagro et al. , 2018; Zarrar et al. , 2015; Bastiani et al. , 2019) ). Definitions of domains and individual features are provided in Table  S2.
Advanced IDPs 2. 3. 4. 2. 1. rs-fMRI. Temporally correlated low-frequency (<0. 1 Hz) fluctuations in the rsfMRI signal are defined as functional resting-state networks (RSNs) (S. M. Smith et al. , 2009). To identify RSNs in fMRI time series, a group-level independent component analysis (ICA) was performed by FSL Melodic  (Beckmann and Smith 2004 ) on the preprocessed fMRI datasets. Following previous works, scans with a mean framewise displacement (FD)  (Power et al. , 2012)  of more than 2 standard deviations (SD) from the group average measured over time were excluded from this analysis. Two atlases of RSNs with a predefined number of independent components were generated: a low dimensional atlas with 20 and a higher dimensional atlas with 50 independent components. A dual regression approach was then used to obtain subject-specific RSNs  (Nickerson et al.
The EPAD LCS baseline imaging dataset Of the 1500 screened participants, 144 did not fulfill the EPAD LCS eligibility criteria and were excluded from the analysis  (Solomon et al. , 2019). The resulting dataset has a complete set of core sequences for 1356 participants. Advanced sequences were performed at thirteen sites, with 756 SWI, 842 fMRI, 831 dMRI, and 858 ASL scans acquired. The mean age was 65. 46 ± 7. 14 (ranging from 50 to 88) and 775 (55. 71 %) were female. All participants were without dementia at inclusion with an average MMSE of 28. 58 ± 1. 66, and 259 (19. 11%) had a CDR of 0. 5. A total of 1246 participants had CSF measurements, of which 845 (67. 8 %) individuals were CSF amyloid negative and 401 (32. 2 %) CSF amyloid positive, following previously defined study-specific cutoffs  (Ingala et al.
, 2021). A detailed description of the baseline clinical and demographic characteristics of the EPAD cohort can be found in  (Ingala et al. , 2021). Pre-processing steps were successful for all the T1w images, while failed for 8. 6 % of fMRI and 8. 6% of dMRI data. These failures were mostly due to lack of necessary files for pre-processing (e. g. reversephase, b-values or vectors for dMRI). A high percentage (62. 3%) of the ASL sequences showed fat-saturation artifacts and are currently excluded from the dataset; preprocessing failure rate for the remaining ASL sequences was 0%. The final preprocessed EPAD dataset resulted in 1356 T1w and FLAIR, 770 fMRI, 759 dMRI, and 237 ASL scans. Details on the number of scanned and available processed data are provided in Fig. 4.
Overall, RSN showed mild positive relationship with age while decreased connectivity was observed in amyloid positive participants, most markedly in the default mode, visual and executive networks (all p < 0. 001; Fig. 7  and Table  S6 ). Visual inspection of group-aligned and skeletonised FA maps led to exclusion of 139 scans, resulting in n = 626 FA maps for IDP computation. Fig. S6  shows examples of included and excluded FA maps. Significant negative correlations of age with both global and regional WM integrity were found for all tracts, and lower FA values were specifically observed in CDR = 0. 5 participants (Fig. 8  and Table  S7 ). Usable ASL data (without fat-saturation artifact) were derived from 237 participants. Mean CBF maps across participants showed considerable regional variability across the brain with high perfusion in the cingulate and precuneus GM and lower perfusion in basal ganglia (Fig.
Discussion and future directions Here, we provided a detailed description of the EPAD MRI dataset and the semi-automatic pipeline developed to process raw multimodal multicenter imaging data from the v1500 baseline data release, and illustrate its IDP feature extraction. We proposed to use a combination of distribution-based statistics and subpopulation visual assessments to help identify low-quality images in EPAD. Finally, we described the computation of MRI summary measures (IDPs) from the core and advanced sequences and provided evidence of meaningful associations between brain phenotypes and other non-imaging metadata. We anticipate that this work may benefit both EPAD investigators by providing a structured, organized, and well-characterized neuroimaging dataset.
MRI cohorts research data management The proposed pipeline for the EPAD study represents an attempt to deliver standardized data in a multicenter MRI study, in a similar way to what has been previously done in other cohorts. Among publicly available neuroimaging cohorts, extensive work on the documentation of MRI data management strategies has been recently published by two large MRI cohort studies  (Sudlow et al. , 2015; Bookheimer et al. , 2019). These efforts entail fully automated pipelines for MRI processing, QC, and computation of IDPs from multimodal single-scanner imaging datasets (F. Alfaro-Almagro and Jenkinson 2016;  Glasser et al. , 2013). Compared to these studies, a unique feature of our pipeline is that it caters multiple scanners from multiple vendors and harmonizes differences between different scanner outputs to allow for automated processing and semi-automated QC.
during the preparation phase  (Jack et al. , 2008)  where QC and preprocessing were done centrally and also provided extracted MRI metrics to investigators. The Preprocessed Connectome Project  (Cameron et al. , 2013)  is a community effort to systematically process data from several available multicenter datasets including structural and functional MRI workflows and comparing outputs of different processing choices and toolboxes. As an example, the Autism Brain Imaging Data Exchange (ABIDE) dataset (Di  Martino et al. , 2014) , which includes 1114 participants with structural (3D T1w) and rs-fMRI data from 16 sites, was processed using three different structural and four functional pipelines, and corresponding derivatives were made available. While both these studies and our EPAD pipeline managed to deal with such diverse datasets, we specifically aimed at creating a unified workflow by delivering a uniform and QCed shared outcome, promoting reproducibility, and avoiding redundancy and variability of results.
Image-derived phenotypes In agreement with current literature (Damoiseaux 2017), we showed an overall effect of age in modulating brain IDPs across different MRI modalities in the EPAD-LCS data set. However, different from more clinically oriented papers, we here only reported basic association without investigating the effect of covariates correction and multiple testing adjustment on the studied associations. Similar to our findings using FreeSurfer and CAT12 segmentations, studies on structural brains converge on a gradual loss of brain volume with advancing age and demonstrate significant regional overlap with our results  (Salat et al. , 2004). Notably, in line with our results, hippocampus and lateral ventricles have previously shown the strongest association with preclinical AD and cognitive decline  (Marizzoni et al.
Limitations Although we acknowledge the heterogeneity of existing MRI processing methodologies and implementations, we focused here on a purely descriptive overview of the procedures implemented for the EPAD study. As we followed generally accepted and standard pipelines, a potential limitation of this study could be the lack of more novel and AI-based processing routines. For example, performing denoising of signal drift correction is becoming a preferable procedure for dMRI preprocessing  (Vos et al. , 2017) , even if few implementations exist. It is important to stress that the aim of this work was to distribute openaccess data and IDPs. We anticipate that the reuse of the same processing pipeline and QC procedures by investigators would result in more comparable EPAD studies than reusing the EPAD data only.
Reflective of EPAD's multi-center and multi-vendor design, ExploreQC was tailored to this dataset and requires further validation and testing before it can be generalized to other studies. Eventually, the lack of visual QC standards for functional MRI sequence motivates the development of more standardized approaches in future works and hampers the interpretation of fMRI QC metrics informative character. However, as shown in supplementary Fig. 3 , a mean frame-wise motion parameter was able to detect outliers and clear differences could be visualized in the SD over time images of those scans, proposing this to be a possible standard for fMRI visual QC in future works. The IDPs found in the EPAD v1500. 0 data sample cover a broad range of structural and functional brain phenotypes. However, several possibilities for summary brain measures exist  (Gong, Beckmann, and Smith 2020).
Conclusion We provide a detailed description of the baseline EPAD LCS MRI dataset including the processing and QC procedures and details of the computation of derived data from core and advanced MRI sequences yielding biologically plausible IDPs. The introduced procedures and results may serve as a reference point for future developments and promote replicability and stability of results on the EPAD cohort. We made the pipeline available for external investigators aiming at the comparability of outcomes between different cohorts. We anticipate that this work will help both imaging and non-imaging researchers working on EPAD for an easier understanding and use of the shared data.
Specifications and Goal In this paper, a magneto-dielectric material will be used in order to shift the resonance frequency of a 20 cm height, 3 cm width, and 1 mm thick antenna (Figure  2a ) from 350 MHz (Figure  2b ) to 200 MHz. During this study, the omnidirectional radiation pattern of the antenna has to be kept (Figure  2c ). The realized gain value will be maximized. The main goal is to maximize this ratio and keep the antenna's properties reasonable (bandwidth, gain, efficiency, and matching). All the results presented before the section «Antenna prototype realization» are EM simulations done with HFSS (High Frequency Structure Simulator) Software.
Specifications and Goal In this paper, a magneto-dielectric material will be used in order to shift the resonance frequency of a 20 cm height, 3 cm width, and 1 mm thick antenna (Figure  2a ) from 350 MHz (Figure  2b ) to 200 MHz. During this study, the omnidirectional radiation pattern of the antenna has to be kept (Figure  2c ). The realized gain value will be maximized. The main goal is to maximize this ratio and keep the antenna's properties reasonable (bandwidth, gain, efficiency, and matching). All the results presented before the section «Antenna prototype realization» are EM simulations done with HFSS (High Frequency Structure Simulator) Software. Magnetism 2022, 2, FOR PEER  REVIEW 3  band from 10 MHz to 400 MHz and will have same factor of reduction in comparison to pure dielectric substrate material.
Matching Network and Antenna Prototype Design 4. 1. Matching Network Design with Real Frequency Technic (RFT) Due to the antenna's impedance modifications by adding parasitic metal plates, the antenna's reflection coefficient has to be improved in order to make the antenna work in the aimed bandwidth. The matching circuit is optimized to match the antenna in the [185-215] MHz bandwidth at -5 dB according to the aimed application. To design such a matching circuit, optimization is performed with OptenniLab software. This software allows the use of all of the available circuit topologies (series and parallel) with capacitance and inductance. It is possible to optimize it in order to obtain the maximum available bandwidth and keep good circuit efficiency. In our case, to limit the circuit complexity, the component number is limited to six to avoid as much as possible the effect of the component tolerances.
LOBSTER (LOss-Based SensiTivity rEgulaRization) is a method for training neural networks having a sparse topology. Let the sensitivity of a network parameter be the variation of the loss function with respect to the variation of the parameter. Parameters with low sensitivity, i. e. having little impact on the loss when perturbed, are shrunk and then pruned to sparsify the network. Our method allows to train a network from scratch, i. e. without preliminary learning or rewinding. Experiments on multiple architectures and datasets show competitive compression ratios with minimal computational overhead.
Practically, LOBSTER shrinks to zero parameters with low sensitivity with a regularizeand-prune approach, achieving a sparse network topology. With respect to similar literature  (Han, Mao, and Dally 2016; Guo, Yao, and Chen 2016; Gomez et al. 2019) , LOB-STER does not require a preliminary training stage to learn the dense reference topology to prune. Moreover, differently to other sensitivity-based approaches, LOBSTER computes the sensitivity exploiting the already available gradient of the loss function, avoiding additional derivative computations  (Mozer and Smolensky 1989; Tartaglione et al. 2018) , or second-order derivatives  (LeCun, Denker, and Solla 1990). Our experiments, performed over different network topologies and datasets, show that LOBSTER outperforms several competitors in multiple tasks.
The rest of this paper is organized as follows. In Sec. 2 we review the relevant literature concerning sparse neural architectures. Next, in Sec. 3 we describe our method for training a neural network such that its topology is sparse. We provide a general overview on the technique in Sec. 4. Then, in Sec. 5 we experiment with our proposed training scheme over some deep ANNs on a number of different datasets. Finally, Sec. 6 draws the conclusions while providing further directions for future research.
Results In this section we experimentally evaluate LOBSTER over multiple architectures and datasets commonly used as benchmark in the literature: • LeNet-300 on MNIST (Fig. 3a ), • LeNet-5 on MNIST (Fig. 3b ), • LeNet-5 on Fashion-MNIST (Fig. 3c ), • ResNet-32 on CIFAR-10 (Fig. 3d ), • ResNet-18 on ImageNet (Fig. 3e ), • ResNet-101 on ImageNet (Fig. 3f ). We compare with other state-of-the-art approaches introduced in Sec. 2 wherever numbers are publicly available. Besides these, we also perform an ablation study with a 2based regularizer and our proposed pruning strategy (as discussed in Sec. 4). Performance is measured as the achieved model sparsity versus classification error (Top-1 or Top-5 error). The network sparsity is defined here ad the percentage of pruned parameters in the ANN model.
LeNet-300 on MNIST As a first experiment, we train a sparse LeNet-300 (Le-Cun et al. 1998) architecture, which consists of three fullyconnected layers with 300, 100 and 10 neurons respectively. We trained the network on the MNIST dataset, made of 60k training images and 10k test gray-scale 28×28 pixels large images, depicting handwritten digits. Starting from a randomly initialized network, we trained LeNet-300 via SGD with learning rate η = 0. 1, λ = 10 -4 , P W E = 20 epochs and T W T = 0. 05. The related literature reports several compression results that can be clustered in two groups corresponding to classification error rates of about 1. 65% and 1. 95%, respectively. Fig. 3a  provides results for the proposed procedure. Our method reaches higher sparsity than the the approaches found in literature.
We hypothesize that, in the case of Sparse VD and for this particular dataset, extracting a larger variety of features at the first convolutional layer, both eases the classification task (hence the lower Top-1 error) and allows to drop more parameters in the next layers (a slightly improved sparsity). However, since we are above 99% of sparsity, the difference between the two techniques is minimal. To scale-up the difficulty of the training task, we ex-perimented on the classification of the Fashion-MNIST dataset  (Xiao, Rasul, and Vollgraf 2017) , using again LeNet5. This dataset has the same size and image format of the MNIST dataset, yet it contains images of clothing items, resulting in a non-sparse distribution of the pixel intensity value. Since the images are not as sparse, such dataset is notoriously harder to classify than MNIST.
For this experiment, we trained the network from scratch using SGD with η = 0. 1, λ = 10 -4 , P W E = 20 epochs and T W T = 0. 1. The results are shown in Fig. 3c. F-MNIST is an inherently more challenging dataset than MNIST, so the achievable sparsity is lower. Nevertheless, the proposed method still reaches higher sparsity than other approaches, removing an higher percentage of parameters, especially in the fully connected layers, while maintaining good generalization. In this case, we observe that the first layer is the least sparsified: this is an effect of the higher complexity of the classification task, which requires more features to be extracted.
ResNet-32 on CIFAR-10 To evaluate how our method scales to deeper, modern architectures, we applied it on a PyTorch implementation of the ResNet-32 network  (He et al. 2015 ) that classifies the CIFAR-10 dataset. 2 This dataset consists of 60k 32×32 RGB images divided in 10 classes (50k training images and 10k test images). We trained the network using SGD with momentum β = 0. 9, λ = 10 -6 , P W E = 10 and T W T = 0. The full training is performed for 11k epochs. Our method performs well on this task and outperforms other state-ofthe-art techniques. Furthermore, LOBSTER improves the network generalization ability reducing the baseline Top-1 error from 7. 37% to 7. 33% of the sparsified network while removing 80. 11% of the parameters. This effect is most likely due to the LOBSTER technique itself, which self- tunes the regularization on the parameters as explained in Sec.
ResNet on ImageNet Finally, we further scale-up both the output and the complexity of the classification problem testing the proposed method on network over the well-known ImageNet dataset (ILSVRC-2012), composed of more than 1. 2 million train images, for a total of 1k classes. For this test we used SGD with momentum β = 0. 9, λ = 10 -6 and T W T = 0. The full training lasts 95 epochs. Due to time constraints, we decided to use the pre-trained network offered by the torchvision library. 3 Fig. 3e  shows the results for ResNet-18 while Fig. 3f  shows the results for ResNet-101. Even in this scenario, LOBSTER proves to be particularly efficient: we are able to remove, with no performance loss, 37. 04% of the parameters from ResNet-18 and 81. 58% from ResNet-101.
Conclusion We presented LOBSTER, a regularization method suitable to train neural networks with a sparse topology without a preliminary training. Differently from 2 regularization, LOBSTER is aware of the global contribution of the parameter on the loss function and self-tunes the regularization effect on the parameter depending on factors like the ANN architecture or the training problem itself (in other words, the dataset). Moreover, tuning its hyper-parameters is easy and the optimal threshold for parameter pruning is self-determined by the proposed approach employing a validation set. LOBSTER achieves competitive results from shallow architectures like LeNet-300 and LeNet-5 to deeper topologies like ResNet over ImageNet. In these scenarios we have observed the boost provided by the proposed regularization approach towards less-unaware approaches like 2 regularization, in terms of achieved sparsity.
Computational Methods The DFT simulations herein were performed using the Vienna Ab-initio Simulation Package  [20, 21]. The Perdew-Burke-Ernzerhof (PBE) and generalized gradient approximation (GGA) were employed to treat the effect of exchange-correlation between the electrons along with DFT-D3  [22]  van der Waals (vdW) dispersion correction. The plane wave cutoff energy was set to 500 eV with the Monkhorst-Pack  [23]  k-point method. A vacuum spacing of 15 Å was kept between the two nearest nanosheets to prevent interaction between them in the three-dimensional periodic systems. Atomic positions and simulation box sizes were relaxed until the fulfilment of the energy and force convergence of 10 -5 eV and 0. 01 eV/Å, respectively. For the geometry optimization, we used a 3 × 3 × 1 k-point grid.
The absorption spectrum was subsequently calculated using the following relation: α(ω) = √ 2ω   ε 2 1 + ε 2 2 -ε 1 2   1/2 (1) Here, ε 2 and ε 1 are the imaginary and real parts of the dielectric function, respectively. In this work, we passively trained moment tensor potentials (MTPs)  [26]  to investigate the thermal and mechanical properties at finite temperatures utilizing the MLIP package  [27]. The required datasets for the fitting of MTPs were prepared by conducting ab-initio molecular dynamics (AIMD) simulations within the VASP package. AIMD calculations were conducted for strain-free and for four uniaxially loaded unitcells with strains of 10 and 20% for the loading along armchair and zigzag directions, in which the temperature of the systems varied from 300 to 1500 K over 400 time steps.