dataesr/scientific-paragraphs-categorization huggingface dataset:
Sampling The dataset of our experiment is highly unbalanced, with high inconclusive and positive extreme polarity outnumbering the small class of extreme negative terms. Training without balancing the data will cause inaccurate results, i. e. , predicting well for inconclusive and positive extreme while poorly for negative extreme. Negative extreme happens to have the essential kind of terms which we cannot ignore. Therefore, undersampling and a mix of under and oversampling techniques are used to get the balanced dataset with an equal number of polarity classes. The sampling technique differs for each dataset, and it is explained in Sect. 5 under the heading of each dataset.
Training and validation We use the "bert-base-cased" pre-trained model as most of our data are in English. The model runs six epochs and a different number of batches for training with each dataset. The batch number varies as the higher number for batch size will cause GPU memory constraints. To overcome this limitation, we keep batch size around 6, 8, or 10 depending on the training data size and use validation data to analyze the training accuracy and loss. The hyper-tuning parameters include optimizer and learning rate. AdamW optimizer is used in the experiment which was also used in the original BERT model for pre-training. The learning rate used in the model is 2e-5 which is necessary to make BERT overcome the catastrophic forgetting problem. However, higher learning rates can be used such as 4e-4 which can cause the failure of converge on the training set  (Sun et al.
The imbalanced labels are not taken into account. Weighted average determines metrics for each label. It finds their average weighted by support. This changes the macro average to reckon unbalanced label which can lead to F-score that is distinct from precision and recall. Support is the number of actual occurrences of the class in the specified dataset. Unbalanced support in the training data may indicate structural weaknesses in the scores of the reported classifiers and could indicate the need for stratified sampling or re-balancing. The support does not variate between the models but rather diagnoses the evaluation process. 1 3 Page 9 of 16 55.
Results and discussion In this section, the results obtained for the detection of extreme sentiments by using BERT are presented. Apart from using the five datasets (RT-polarity, T4SA, Senti-ment140, TurntoIslam, and Ansar1), we have also combined these datasets to build a new dataset, comb_all to analyze the overall performance of the proposed model. Hence, six experimental results and findings are outlined for each dataset. The datasets used in this experiment are acquired from our previous work, which we refer to as the extended lexicon  (Pais et al. 2020 ). The comprehensive lexicon data are highly unbalanced, as shown in Table  2. The number of negative extremes is very low compared to inconclusive and positive extremes. The split of these data into train, validation, and test sets causes further isolation, which results in higher accuracy of dominant classes while ignoring the minor class, which is negative extreme.
RT-polarity This is a dataset of classified movies, containing the polarity of tweets calculated in the first phase (Sect. 3). It is highly skewed toward the inconclusive and the positive extreme classes, with a tiny percentage of negative extreme. The total records are 26108 for inconclusive, 5596 for positive Extreme, and 282 for negative extreme. The undersampling technique is used to prepare data for training. The equal number of data is taken randomly from the class of inconclusive and positive extreme with regard to the negative extreme which implies 282 entries for each class. The train set contains 20% of data, while 80% of the data is used for the test and validation set. After running three epochs of training, the validation accuracy of the model becomes flat at 67% approximately.
T4SA The next dataset we use is T4SA (Twitter for sentiment analysis). The text in the T4SA dataset is in byte string format, and we have applied a special functionality to clean and decode the text in order to finalize it for training as it contains emojis. The number of entries is 3134881 for inconclusive, 293568 and positive extreme and 24114 for negative extreme. As the negative extreme is the lowest class with 24114 entries, the same number of entries are randomly selected for the inconclusive and positive extreme class while using the undersampling technique. The train set contains 20% of the data, while 80% of the data is used for the test and validation sets. Sentiment analysis is widely used on Twitter datasets because of its usefulness. Therefore, the BERT model gives 99% training, 98% validation, and 98% test accuracy after running six epochs, supposedly because BERT is pre-trained on a large corpus of unlabeled text.
Sentiment140 The sentiment140 data contain two files. The sentiment140_ test is used for creating comb_all dataset, and the senti-ment140_train file is discarded because of the absence of "extreme negative" class in polarity. The text, which is the training variable, is the message that needs preprocessing. The sentiment_train140 dataset is tiny, and the number of entries is 1808 inconclusive, 168 for positive extreme and 12 for negative extreme. Since the negative extreme class is extremely small with 12 entries in total, the probability of data leakage and overfitting is high regardless of sampling technique used. Therefore, the experiment is not done on this dataset; rather, it is experimented indirectly by combining it with other datasets in comb_all.
TurntoIslam The TurntoIslam dataset contains two extra classes of our target variable, "polarity," similar to Ansar1. Since the main goal of our work is to find extreme positive and extreme negative terms, we regard the negative non-extreme and positive non-extreme as inconclusive or neutral. The number of records is 690902 for inconclusive, 302178 for positive extreme, 3828 for positive non-extreme, 3574 for    5. The confusion matrix of TurntoIslam given in Fig. 5  indicates that the model works well for the negative extreme, which is the most important class of our dataset.
Ansar1 Ansar1 is a dark web forum that contains mixed language discussions on forum. Ansar1 dataset yields lower accuracy compared to other datasets and it can be attributed to the presence of foreign languages other than the English language in sentences. While the central part consists of English, the mixture of another language, especially Arabic, can influence the model, as we are using the BERT pre-trained model for English. Although multilingual pretrained models are available for BERT, they do not provide better results with the type of data we are using. We split the more extensive texts into smaller chunks to feed the BERT model. The number of entries is 65301 for inconclusive, 21334 for positive extreme, 796 for negative extreme, 520 for positive non-extreme, and 491 for negative nonextreme.
comb_all The last dataset for the experiment is made by combining all the previous five datasets we tested on. All datasets are concatenated based on the common classes of polarity present in all datasets to preserve the data consistency, which is "inconclusive, positive extreme and negative extreme. " This dataset is referred to as comb_all. The sampling technique is not required here as the data used to create this dataset are already passed through the stages of the sampling phase. A total of 20% data are used for the training set, and the rest of the 80% data are used for test and validation set. This combination of data into a new dataset provides a good insight into the efficacy of using BERT, which seems to perform well. It proves the reliability of BERT for our tasks and related ones.
Also, it verifies our approach to classify into extreme positive and extreme negative classes. The training accuracy of the model reaches around 98%, while the validation accuracy of our model is around 95%. The accuracy we get on the test set is also 95%, which confirms the validation accuracy. The classification report of comb_all is shown in Table  7. Figure  7  shows the confusion matrix of our model, which achieves good results for extreme values, especially for negative extreme. For a comprehensive analysis and visualization of the difference between the original lexicon and the expanded one, using BERT, we can look at the results presented in Table  8 , for each dataset. The results are also shown in form of graphs in Figs. 8, 9, and 10. However, the Senti-ment140 dataset results are presented here only for the sake of comparison and to highlight the issue of data  leakage and overfitting caused by small and imbalanced data (Table  9 ).
In terms of overall accuracy, BERT shows promising results for the datasets RT polarity, Sentiment140 and T4SA. Mainly, it successfully classifies the Extreme Negative (EN) after fine-tuning the base model of BERT as seen in Table  8. It yields higher result values for the dataset T4SA which can be because of being a massive training data and the fact that it contains common topics and themes. In regard to extreme negative sentiments, it shows limited ability to classify the sentiments. Our proposed method outperforms BERT in certain cases and highlights the limitation of BERT for detecting extreme sentiments. BERT is one of the top models for the purpose of general sentiment analysis, while for very specific tasks of extreme sentiments, it requires further enhancements or other models on top of the base BERT model for better detection.
Conclusion and future work In this paper, we validated an unsupervised and languageindependent approach for detecting people's extreme sentiments on social media platforms using BERT. The proposed approach is based on defining extreme polarity for terms and generating extreme sentiment lexicon by relying upon two standard lexical resources, i. e. , Senti-WordNet 3. 0 and SenticNet 5. This work provides a standard lexicon consisting of extreme positive and negative terms polarity. We implemented a prototype system with two different components ESG and ESC. The proposed system experimented on five social networks and media data lexicons to analyze its accuracy, effectiveness, and   efficiency. Moreover, word embeddings have also been utilized to extend the lexicon to analyze the improvement in the system's performance.
Acknowledgements This work was supported by  National Founding  from the  FCT-Fundacão  para a Ciência e a Tecnologia, through the  MOVES  Project -PTD/EEI-AUT/28918/2017  and by operation  Centro-01-0145-FEDER-000019-C4 - Centro de Competências em Cloud Computing , co-financed by the  European Regional Development Fund (ERDF)  through the  Programa Operacional Regional do Centro  ( Centro 2020 ), in the scope of the  Sistema de Apoio à Investigação Científica e Tecnológica.
Brain charts for the human lifespan Over the past few decades, neuroimaging has become a ubiquitous tool in basic research and clinical studies of the human brain. However, no reference standards currently exist to quantify individual differences in neuroimaging metrics over time, in contrast to growth charts for anthropometric traits such as height and weight  1. Here we assemble an interactive open resource to benchmark brain morphology derived from any current or future sample of MRI data (http://www. brainchart. io/). With the goal of basing these reference charts on the largest and most inclusive dataset available, acknowledging limitations due to known biases of MRI studies relative to the diversity of the global population, we aggregated 123,984 MRI scans, across more than 100 primary studies, from 101,457 human participants between 115 days post-conception to 100 years of age.
Primary case-control studies are usually focused on a single disorder despite evidence of trans-diagnostically shared risk factors and pathogenic mechanisms, especially in psychiatry  16, 17. Harmonization of MRI data across primary studies to address these and other deficiencies in the extant literature is challenged by methodological and technical heterogeneity. Compared with relatively simple anthropometric measurements such as height or weight, brain morphometrics are known to be highly sensitive to variation in scanner platforms and sequences, data quality control, pre-processing and statistical analysis  18  , thus severely limiting the generalizability of trajectories estimated from any individual study  19. Collaborative initiatives spurring collection of large-scale datasets  20, 21  , recent advances in neuroimaging data processing  22, 23  and proven statistical frameworks for modelling biological growth curves  2, 24, 25  provide the building blocks for a more comprehensive and generalizable approach to age-normed quantification of MRI phenotypes over the entire lifespan (see Supplementary Information 1 for details and consideration of previous work focused on the related but distinct objective of inferring brain age from MRI data).
Mapping normative brain growth We created brain charts for the human lifespan using generalized additive models for location, scale and shape  2, 24  (GAMLSS), a robust and flexible framework for modelling non-linear growth trajectories recommended by the World Health Organization  24. GAMLSS and related statistical frameworks have previously been applied to developmental  Fig. 1  | Human brain charts. a, MRI data were aggregated from over 100 primary studies comprising 123,984 scans that collectively spanned the age range from mid-gestation to 100 postnatal years. Box-violin plots show the age distribution for each study coloured by its relative sample size (log-scaled using the natural logarithm for visualization purposes). b, Non-centiled, 'raw' bilateral cerebrum tissue volumes for grey matter, white matter, subcortical grey matter and ventricles are plotted for each cross-sectional control scan as a function of age (log-scaled); points are coloured by sex.
c, Normative brain-volume trajectories were estimated using GAMLSS, accounting for site-and study-specific batch effects, and stratified by sex (female, red; male, blue). All four cerebrum tissue volumes demonstrated distinct, non-linear trajectories of their medians (with 2. 5% and 97. 5% centiles denoted as dotted lines) as a function of age over the lifespan. Demographics for each cross-sectional sample of healthy controls included in the reference dataset for normative GAMLSS modelling of each MRI phenotype are detailed in Supplementary Table  1. 2-1. 8. d, Trajectories of median between-subject variability and 95% confidence intervals for four cerebrum tissue volumes were estimated by sex-stratified bootstrapping (see Supplementary Information 3 for details). e, Rates of volumetric change across the lifespan for each tissue volume, stratified by sex, were estimated by the first derivatives of the median volumetric trajectories.
For solid (parenchymal) tissue volumes, the horizontal line (y = 0) indicates when the volume at which each tissue stops growing and starts shrinking and the solid vertical line indicates the age of maximum growth of each tissue. See Supplementary Table  2. 1 for all neurodevelopmental milestones and their confidence intervals. Note that y axes in b-e are scaled in units of 10,000 mm 3 (10 ml). modelling of brain structural and functional MRI phenotypes in open datasets  19, [26] [27] [28] [29] [30] [31]. Our approach to GAMLSS modelling leveraged the greater scale of data available to optimize model selection empirically, to estimate non-linear age-related trends (in median and variance) stratified by sex over the entire lifespan, and to account for site-or study-specific 'batch effects' on MRI phenotypes in terms of multiple random effect parameters.
Specifically, GAMLSS models were fitted to structural MRI data from control subjects for the four main tissue volumes of the cerebrum (total cortical grey matter volume (GMV), total white matter volume (WMV), total subcortical grey matter volume (sGMV) and total ventricular cerebrospinal fluid volume (ventricles or CSF)). Supplementary Tables 1. 1-1. 8 present details on acquisition, processing and demographics of the dataset; see Methods, 'Model generation and specification' and Supplementary Information 1 for further details regarding GAMLSS model specification and estimation; image quality control, which used a combination of expert visual curation and automated metrics of image quality (Supplementary Information 2); model stability and robustness (Supplementary Information 3, 4); phenotypic validation against non-imaging metrics (Supplementary Information 3 and 5.
2); inter-study harmonization (Supplementary Information 5); and assessment of cohort effects (Supplementary Information 6). See Supplementary Information 19 for details on all primary studies contributing to the reference dataset, including multiple publicly available open MRI datasets  [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42]. Lifespan curves (Fig. 1 , Supplementary Table  2. 1) showed an initial strong increase in GMV from mid-gestation onwards, peaking at 5. 9 years (95% bootstrap confidence interval (CI) 5. 8-6. 1), followed by a near-linear decrease. This peak was observed 2 to 3 years later than previous reports relying on smaller, more age-restricted samples  43, 44. WMV also increased rapidly from mid-gestation to early childhood, peaking at 28. 7 years (95% bootstrap CI 28.
001). d, Heritability of raw volumetric phenotypes and their centile scores across two twin studies (Adolescent Brain Cognitive Development (ABCD) and Human Connectome Project (HCP)); Supplementary Information 19), see Supplementary Information 13 for a full overview of statistics for each individual feature in each dataset. Data are mean ± s. e. m. (although some confidence intervals are too narrow to be seen). MCI, mild cognitive impairment. See Fig. 3  for other diagnostic abbreviations. FDR-corrected significance: *P < 0. 05, **P < 0. 01, ***P < 0. 001.
This epoch of GMV:WMV differentiation, which may reflect underlying changes in myelination and synaptic proliferation 4,55-58 , has not been demarcated in previous studies  45, 59. It was probably identified in this study owing to the substantial amount of early developmental MRI data available for analysis in the aggregated dataset (in total across all primary studies, N = 2,571 and N = 1,484 participants aged less than 2 years were available for analysis of cerebrum tissue volumes and extended global MRI phenotypes, respectively). The period of GMV:WMV differentiation encompasses dynamic changes in brain metabolites  60  (0-3 months), resting metabolic rate  61  (RMR) (minimum = 7 months, maximum = 4. 2 years), the typical period of acquisition of motor capabilities and other early paediatric milestones  62  , and the most rapid change in TCV (Fig.
Individualized centile scores We computed individualized centile scores that benchmarked each individual scan in the context of normative age-related trends (Methods, 'Centile scores and case-control differences' and Supplementary Information 1-6 for further details). This approach is conceptually similar to quantile rank mapping, as previously reported  26, 28, 29  , where the typicality or atypicality of each phenotype in each scan is quantified by its score on the distribution of phenotypic parameters in the normative or reference sample of scans, with more atypical phenotypes having more extreme centile (or quantile) scores. The clinical diversity of the aggregated dataset enabled us to comprehensively investigate casecontrol differences in individually specific centile scores across a range of conditions.
Longitudinal centile changes Owing to the relative paucity of longitudinal imaging data (about 10% of the reference dataset), normative models were estimated from cross-sectional data collected at a single time point. However, the generalizability of cross-sectional models to longitudinal assessment is important for future research. Within-subject variability of centile scores derived from longitudinally repeated scans, measured with the interquartile range (IQR) (Methods, 'Longitudinal stability', Supplementary Information 1. 7), was low across both clinical and CN groups (all median IQR < 0. 05 centile points), indicating that centile scoring of brain structure was generally stable over time, although there was also some evidence of between-study and cross-disorder differences in within-subject variability (Supplementary Information 14).
Centile scoring of new MRI data A key challenge for brain charts is the accurate centile scoring of out-of-sample MRI data, not represented in the reference dataset used to estimate normative trajectories. We therefore carefully evaluated the reliability and validity of brain charts for centile scoring of such 'new' scans. For each new MRI study, we used maximum likelihood to estimate study-specific statistical offsets from the age-appropriate epoch of the normative trajectory; we then estimated centile scores for each individual in the new study benchmarked against the offset trajectory (Fig. 5 , Methods, 'Data-sharing and out-of-sample estimation', Supplementary Information 1. 8). Extensive jack-knife and leave-one-study-out analyses indicated that a study size of N > 100 scans was sufficient for stable and unbiased estimation of out-of-sample centile scores (Supplementary Information 4).
Discussion We have aggregated the largest neuroimaging dataset to date to modernize the concept of growth charts for mapping typical and atypical human brain development and ageing. The approximately 100-year age range enabled the delineation of milestones and critical periods in maturation of the human brain, revealing an early growth epoch across its constituent tissue classes-beginning before 17 post-conception weeks, when the brain is at approximately 10% of its maximum size, and ending by age 3, when the brain is at approximately 80% of the maximum size. Individual centile scores benchmarked by normative neurodevelopmental trajectories were significantly associated with neuropsychiatric disorders as well as with dimensional phenotypes (Supplementary Information 5. 2, 12). Furthermore, imaging-genetics studies  65  may benefit from the increased heritability of centile scores compared with raw volumetric data (Supplementary Information 13).
However, the current results bode well for future progress towards digital diagnosis of atypical brain structure and development  67. By providing an age-and sex-normalized metric, centile scores enable trans-diagnostic comparisons between disorders that emerge at different stages of the lifespan (Supplementary Information 10, 11). The generally high stability of centile scores across longitudinal measurements also enabled assessment of brain changes related to diagnostic transition from mild cognitive impairment to Alzheimer's disease (Supplementary Information 14), which provides one example of how centile scoring could be clinically useful in quantitatively predicting or diagnosing progressive neurodegenerative disorders in the future. Our provision of appropriate normative growth charts and online tools also creates an immediate opportunity to quantify atypical brain structure in clinical research samples, to leverage available legacy neuroimaging datasets, and to enhance ongoing studies.