dataesr/scientific-paragraphs-categorization huggingface dataset:
Different methodologies and epistemologies can be considered to approach and study narrative engagement. For instance, the method Busselle and Bilandzic (ibid) relied upon was to measure the audience's attentional focus and viewing experience by mean of self-report. Another perspective is the one of neuroscience, where one pays attention to brain processes when an audience perceives narrative messages  (Grall et al. , 2021). For instance, shared brain responses toward movie narratives can be highlighted by inter-subject correlation (ISC) using functional Magnetic Resonance Imaging (fMRI) across brain regions  (Schmälzle and Grall, 2020; Grall et al. , 2021). While it might seem easier to collect self-reports than recording brain responses when dealing with many subjects, large-scale approaches are getting more common in neuroimaging, with for instance nearly 350 subjects for the "Narrative" fMRI dataset of brain activity when listening to spoken stories  (Nastase et al.
Data collection of time-synchronic movie comments We collected time-synchronic comments in Chinese from a video-streaming platform: Tencent Video. 2 Most of the movies there are paid content, which means only users who pay for a given movie -or for unlimited access -can watch it and send out comments. We wrote a Python script to crawl 3,831,222 comments from 240 movies. Each time-synchronic comment contains the user ID of the commenter, the calendar time when it was sent, the video time it corresponds to, and its content. We kept only the video time and content of each comment. We deleted non-Chinese characters to minimize noise in the data. Some numbers have a special meaning on Chinese social media, for instance "2333"when something is amusing -or "6666, " which means "cool" / "amazing. " However, as such significations were not covered by our main analytical tool (the Simplified Chinese version of the Linguistic Inquiry and Word Count or SCLIWC, see below), we chose to discard them.
After data cleaning, the total number of comments was 3,220,079. The code written to conduct our research and sample time-synchronic comments, as well as a number of outputs, are available in a GitHub repository. 3 o better understand the effect of time on viewers' responses, we segmented the comments of each movie into 50 consecutive subgroups, each corresponding to the audience's comments when viewing 2% of the movie. This segmentation does not accurately divide film narratives according to setup, complications, climax, or epilog (which would have been much more difficult to achieve); however, it allows us to track the chronological changes in viewers' commenting behavior and underlying evolving psychological activity. To understand the effect of the movie genre on viewers' reactions, we chose to study 12 of the genres of the classification provided by the Internet Movie Database (IMDB for short).
4 These 12 genres are Action, Adventure, Crime, Comedy, Drama, Fantasy, Horror, Mystery, Romance, SciFi, Thriller, and War. For each genre, we selected 20 movies from IMDB TOP250 most rated movies within that genre, to ensure that each movie was of significant popularity, for the less popular movies may have had fewer viewers, resulting in a lack of ample comments for analysis. We moved down each list, starting from the most popular movies and skipping those unavailable on Tencent Video. Many movies actually fall into two or three of the genres we chose to study. We thus counted every movie one to three times, which resulted in different numbers of movies eventually falling into each category. We finally collected ratings for each movie in our dataset from both IMDB and Tencent Video (Full list of movies is provided as Supplementary Material S1), and the year in which the movie was produced.
Identifying linguistic and psychological aspects of the comments To identify the linguistics features viewers' comments as well as the psychological dimensions they point to, we relied on the Simplified Chinese version of the Linguistic Inquiry and Word Count (SCLIWC) software  (Gao et al. , 2013). The Linguistic Inquiry and Word Count system (LIWC;  Pennebaker et al. , 2007)  is a transparent text analysis program that counts words in linguistically or psychologically meaningful categories  (Tausczik and Pennebaker, 2010). The 2007 version of the LIWC system contains a dictionary with words falling into 80 categories  (Tausczik and Pennebaker, 2010; Zhao et al. , 2016) , and the SCLIWC expands it into 102 categories to reflect the special characteristics of Chinese. For example, auxiliary verbs contain words such as "am, " "will, " and "have, " while the category of prepositions gathers words such as "to, " "which, " and "above.
The sensitivity of the clustering of time series to the clustering technique used is a current limitation of our work, and can be further studied by comparing various algorithms. Such algorithms do not yield a "correct answer" but instead offer a view on the data which must be considered as such. The reversed u-shape trajectory of cognition-related word rates for some of the movies (ScrCogCluster2) corresponds to the shape of the so-called cognitive tension curve in novel narratives discussed by  Boyd et al. (2020)  (this curve was also generated by calculating cognition-related word rates with LIWC on a series of text segments). However, our second cluster, ScrCogCluster1 (106 out of 179 movies), displays an upward trend with two periods of increase at the beginning and at the end of the movie, separated by a period of near stability.
Structure of the paper This paper is organized with respect to subtopics that are most relevant for the analysis of HDD, particularly motivated by typical aims of biomedical studies but also applicable more generally. These subtopics are initial data analysis (IDA and Preprocessing, section "IDA: Initial data analysis and preprocessing"), exploratory data analysis (EDA, section "EDA: Exploratory data analysis"), multiple testing (section "TEST: Identification of informative variables and multiple testing"), and prediction (section "PRED: Prediction"). For each subtopic, we discuss a list of main analytical goals. For each goal, basic explanations, at a minimally technical level, are provided for some commonly used analysis methods. Situations are identified where performance of some traditional, possibly more familiar, statistical methods might break down in the HDD setting or might not be possible to apply at all when p is larger than n.
IDA2. 2: Tabulation of missing data Missing values are ubiquitous in real-world data and may have major implications for choice of analysis methods and interpretation of results  [36]. In fact, most multivariable and multivariate analyses methods have as their default requirement that values of all variables are available for all subjects, i. e. , all observations are "complete. " An important early step in any analysis is tabulation of the missing values, i. e. , the identification of the number of missing values per subject and per variable, respectively, to provide an overview of the missing data structure. In multi-omics integrative studies, high-dimensional data from different data types are collected for the same subjects. In such studies, small sample size caused by experimental and financial constraints, which can also vary between data types, can be the reason for missing data, the absence of which has to be taken into account in the subsequent statistical analysis.
IDA3. 1: Background subtraction and normalization Omics data are prone to perturbations due to systematic effects induced by the measurement technology, also referred to as the assay platform. Many of these effects are unique to the assay platform, but there are some commonalities. A biological sample may have its gene expression profile measured using a single microarray or gene chip or its protein profile measured using a mass spectrometry system. The set of experimental conditions that gives rise to profiles such as these will be referred to here as an experimental run. However, even for the same sample, measurements obtained in different runs may differ due to factors such as different amounts of biological material input to the measurement system, Fig. 4  Comparison of a scatterplot (left) and a Bland-Altman plot (right, also MA plot for omics data) of the same data.
The samples have been annotated in the plot according to pathologic subtype: 11 B-cell chronic lymphocytic leukemia (B-CLL; blue squares), 9 follicular lymphoma (FL; black triangles), and 42 diffuse large B-cell lymphoma (DLCL; red dots). Source:  [39]  Fig. 7  Biplot constructed by superimposing a PCA plot of 62 lymphoma samples (see Fig. 6 ) onto a PCA plot of genes where first and second principal component for the genes are calculated from gene expression profiles comprising expression levels of the 62 samples for each gene. Genes are represented in the plot as small green dots. Genes representing the three classes well are indicated by numbers. Source:  [39]  with nuisance batch effects, as this could make it impossible to remove nuisance batch effects adequately. Preprocessing of omics data aimed at removal of the aforementioned artifact effects poses several challenges.
Source:  [44]  IDA4. 3: Construction of new variables Sometimes it is useful to construct new variables as an initial step of data analysis by combining the variables that are available in the dataset in a meaningful way, using expert knowledge. For example, in medical studies investigating factors affecting health, often, overweight status is an important variable to consider in the analysis. Because weight and height must be considered together in assessing whether an individual is overweight, constructed variables like body mass index (BMI) have been used. The importance of fat distribution has also been recognized, and it has motivated the combined measured of waist-hip ratio (WHR). Instead of relying on the ability of statistical methods and algorithms to construct such variables implicitly, e.
IDA4. 4: Removal of variables or observations due to missing values The simplest approach to deal with missing data is a "complete case analysis. " That is, if a single variable is missing for an observation, the observation is fully excluded from the dataset. Basing analyses on only complete cases at best only leads to loss of statistical power, but at worst can lead to substantially biased analyses. Impact of missing data will depend on how many cases have missing data, how many variables have missing values, how many values are missing, and whether the likelihood of missing values in a variable is related to the value of that variable or other variables. When few observations have missing values for few variables, then the impact on results of subsequent analyses may be limited, but when the number is large, the impact can be substantial.
Multiple imputation Multiple imputation is a widely used approach for handling missing data under the MAR scenario. It uses a regression model based on the available variables to predict the missing values. In an iterative fashion, missing values of a specific variable are predicted using a regression model that depends on the other observed variables, and the resulting predicted value is used in the main regression model. To account for the uncertainty in the imputation, multiple imputed datasets are generated and then analyzed, and the results are summarized according to "Rubin's rule"  [61]. Software for multiple imputation is widespread in major statistical packages. As described above, for HDD, before applying multiple imputation, often a pre-selection of variables is advisable Future directions for HDD analysis include a more detailed look at MAR settings (as all procedures provided so far are fully justified only when the MCAR assumption is tenable), the addition of auxiliary information for specifying the imputation model, and development of analysis methods that can directly cope with missing values, such as robust PCA and random forests.
For example, if outlier observations are removed from a dataset, the performance of a prediction model built only on the remaining observations is most probably an overly optimistic estimate of what the model performance would be on an independent dataset, which might contain different outliers. Two major analytical goals for EDA are (1) to identify interesting data characteristics such as variables with extreme values, associations between variables, or representative subjects with usual values of variables, and (2) to gain insight into the structure of the data. Note that many of the methods used in EDA are also applied in IDA (like PCA; see section "IDA2. 4: Graphical displays"). In this section, we focus on methods that are more specific to EDA. Note that many of the methods described in this section are generally designed and suitable for continuous data; only some can also be applied for discrete data.
EDA2: Gain insight into the data structure A global depiction of data to identify structure, including patterns or motifs, is another major goal of exploratory data analysis for HDD. Here, data structure is understood in a general sense, it refers to many aspects of the data that concern the arrangement or interrelation of the observations or variables of a dataset. Although a natural first step is to look at marginal distributions (e. g. , univariate and bivariate) of all variables across observations, this approach is generally not feasible for HDD for reasons discussed above. Further, some structure may involve many different variables and not be discernible by examination of univariate, bivariate, or even trivariate distributions. The data visualization techniques described in section "EDA1.
Therefore, the distance metrics may be prone to exaggeration by a few distant objects. Strategies to help avoid this problem include use of data reduction or variable selection before clustering (see section "IDA2. 4: Graphical displays" for graphical displays for dimension reduction and section "PRED1. 2: Variable selection. " for variable selection and dimension reduction in the context of improving prediction models). Clustering algorithms can be divided into hierarchical and partitioning methods. In hierarchical clustering, observations are iteratively grouped together into larger clusters (agglomerative hierarchical clustering) or clusters are subdivided into smaller clusters (divisive hierarchical clustering). Centroid-based so-called partitioning Fig. 10  Two-dimensional visualization of a high-dimensional dataset using t-SNE.
The dataset consists of 2700 single cells (peripheral blood mononuclear cells) that were sequenced on an Illumina NextSeq 500. The dataset is freely available from 10X Genomics. Points are colored by cell type. The plot shows that the cell types are locally well separated. Source:  [71]  algorithms aggregate the observations around specific points (the centroids) such that observations related to the same centroid are as similar as possible, and observations related to different centroids as different as possible. Hierarchical clustering algorithms provide a clustering for any number of clusters, whereas partitioning methods require an initial choice about the number of clusters present in the data. The most popular clustering algorithms are described in Table  13. Other methods for cluster analysis applied to biomedical data include fuzzy clustering and SOMs (self-organizing maps).
This enforces stringent control on the probability that any of the hundreds of thousands of genomic variants typically studied is falsely identified as associated with the phenotype of interest. Second, a simple Bonferroni correction is conservative in that it does not leverage information about Table  16  Methods for hypothesis testing for multiple variables in HDD: Limma, edgeR, DEseq2 Limma Linear Models for Microarray Data (limma) developed by Smyth and colleagues  [95, 96]  and implemented in the R package limma was developed to address several challenges of multiple testing for HDD. Limma offers a unifying, statistically based framework for multiple testing that uses empirical Bayes shrinkage methods in the context of linear models. Initially popularized in the context of traditional gene expression analysis with microarrays, limma is based on normal distribution theory.
g. ,  [128, 129] ). The most straightforward statistical approach to construct a prediction model using several predictor variables simultaneously while taking into account their correlation is fitting a multivariable (generalized) regression model, for example a simple linear regression model in the case of an approximately normally distributed target variable. In linear regression, the regression coefficients are fitted such that the sum (for the n observations) of squared errors (i. e. , of squared differences between the true value of the target variable and the predicted value) is minimal. Mathematically, this basic linear regression amounts to solving a system of n equations with p + 1 unknowns, where p stands for the number of predictor variables. Such a regression model, however, cannot be fitted if the number p + 1 of coefficients to fit (the intercept and one coefficient for each variable) exceeds the dataset size n.
Standardization Another variable transformation often performed in high-dimensional contexts is standardization. Here, the variable is centered (for each value of the variable the mean of the variable is subtracted) and scaled (each centered value is divided by the standard deviation of the variable). This procedure has advantages from an interpretation point of view. For example, the intercept of a linear model including age would represent a person of average age instead of a hypothetical person of age 0. Further, standardization is crucial for the correct implementation of many regularized methods (e. g. , lasso and ridge regression, see section "PRED1. 4: Statistical modelling"). Note that standardization can cause problems when applying a prediction model to a new dataset. In this case, one either has to use the correction factors calculated from the original dataset or recompute them on the new dataset, which is problematic because then individual predictions depend on other observations that happened to be included in the new dataset.
Regarding horizontal integration, the meta-analytic approach of pooling summary measures of association is the most used approach. For other applications, such as clustering, in order to deal with different normalizations and platforms for the different datasets, centering, and standardization  [161]  or specific methods should be considered; for clustering, see for example Huo et al. [162]. Vertical data integration is typically model-based and the model used considers the specific characteristics of the data to be integrated and of the research question (whether exploratory or predictive). In biomedicine, integration of multiple omics data types can provide deeper biological insights compared to individual omics in terms of disease subtyping, biomarker identification, and understanding of molecular Table  23  Methods for statistical modelling with constraints on regression coefficients: Ridge regression, lasso regression, elastic net, boosting.
Mathematical theorems show that increasing the number of hidden layers and decreasing the number of neurons in each layer can improve the prediction performance of neural networks. Specific neural networks with many hidden layers are called deep learning. The choice of the tuning parameters (activation function, number of hidden layers, and number of neurons per layer) characterizes the different kinds of neural networks (and deep learning algorithms). In the high-dimensional contexts, special approaches (e. g. , selecting variables or setting weights to zero) are used to avoid overfitting. Deep learning methods are extremely successful in the situation of a very large number of observations (as in image classification and speech recognition based on huge databases). However, they tend to generate overfitted models for typical biomedical applications in which the number of observations (e.
Such measures can be used to quantify the performance of a model (or algorithm) or to compare different models constructed using the same dataset. In most biomedical applications, the goal of a comparative assessment is to select a final model  [177, 178]. Models of absolute risk that depend on covariates have been used to design intervention studies, to counsel patients regarding their risks of disease or future disease-related events, and to inform clinical decisions. Several criteria related to "calibration" and "discriminatory power" have been proposed  [179, 180]. Often the main interest will be in the added value of biomarkers or gene signatures relative to an existing clinical prediction model. Several performance measures are available to quantify the added value  [181]. For a clinical task, several very different models with equivalent prediction performance may be available.
ROC curves and AUC A receiver operating characteristic (ROC) curve is a graphical plot that facilitates visualization of the discrimination ability of a binary classification method. Many statistical methods classify observations into two classes based on estimated probabilities of their membership. If the probability is larger than a threshold, then the response is classified as positive (e. g. , sick), otherwise as negative (e. g. , healthy). This threshold is mostly set to 0. 5 or to the prevalence of the positive cases in the dataset. Choosing a lower threshold corresponds to more positive predictions, with the consequence of increasing the percentage of observations correctly classified positive among those actually positive (sensitivity) with the potential cost of decreasing the percentage of observations correctly classified negative among those actually negative (specificity).
The specific choice of the information criterion is difficult and depends, e. g. , for classification tasks, also on the relative importance of sensitivity and specificity  [188]  PRED2. 2: Internal and external validation Whatever measure of model performance has been chosen, computing it on the same dataset that was used for constructing the model may lead to a dramatic over-estimation of the performance. Instead, one should assess prediction performance using independent data, i. e. , data not used to construct the model  [189] [190] [191]. One classical procedure is to split the given dataset into a training set and a test set, and then to construct the model using only the training set and evaluate the model using only the test set. This is one type of "internal validation, " in contrast to "external validation, " where data from independent patient cohorts are used  [192].
Due to the typical instability of predictors developed using HDD, this sample splitting procedure is very risky, as in most cases the specific split heavily influences the result. Resampling techniques, such as cross-validation, subsampling and bootstrapping, can be less risky, although even those methods cannot avoid impact of biases in the data introduced by faulty designs such as those that would confound batch effects with outcome variables. The common idea behind these procedures is to repeatedly use a part of the dataset as training dataset, i. e. , to construct a prediction model, and the other (non-overlapping) part as test dataset to evaluate the constructed model. This process is repeated several times for different splits into training and test data to produce a more stable and representative estimate of model performance.