dataesr/scientific-paragraphs-categorization huggingface dataset:
Brain Imaging Datasets We work with two datasets: Pereira and Narratives-Pieman. Results on Narratives-Lucy and Narratives-SlumLord show similar trends. Hence, we also show results on Narratives-Lucy and Narratives-SlumLord in the appendix. Pereira Dataset (Reading Sentences from Passages) For the Pereira dataset, similar to earlier work  (Sun et al. , 2019 (Sun et al. , , 2020)) , we combine the data from sentence-based experiments (experiments-2 and 3) from  Pereira et al. (2018). Five subjects were presented a total of 627 sentences from 48 broad topics, spanning over 168 passages, where each passage consists of 3-4 sentences. As in  (Pereira et al. , 2018) , we focused on nine brain ROIs (regions of interest) corresponding to four brain networks: (i) Default Mode Network (DMN) (linked to the functionality of semantic processing), (ii) Language Network (related to language processing, understanding, word meaning, and sentence comprehension), (iii) Task Positive Network (TP) (related to attention, salience information), and (iv) Visual Network (related to the processing of visual objects, object recognition).
We briefly summarize the details of the dataset and the number of voxels corresponding to each ROI in Table  1   Narratives-Pieman (Listening to Stories) The "Narratives" collection aggregates a variety of fMRI datasets collected while human subjects listened to naturalistic spoken stories. The Narratives dataset that includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ∼4. 6 hours of unique stimuli (∼43,000 words) was proposed in  (Nastase et al. , 2021). Similar to earlier works  (Caucheteux et al. , 2021b) , we analyze data from 82 subjects listening to the story titled 'PieMan' with 259 TRs (repetition time -fMRI recorded every 1. 5 sec. ). We list number of voxels per ROI in this dataset in Table  2. We use the multimodal parcellation of the human cerebral cortex (Glassar Atlas: consists of 180 ROIs in each hemisphere) to display the brain maps  (Glasser et al.
, 2016) , since Narratives dataset contains annotations tied to this atlas. The data covers ten brain ROIs in the human brain, i. e. , Left hemisphere (L), and Right hemisphere (R) for each of the following: (i) early auditory cortex (EAC: A1, LBelt, MBelt, PBelt, and R1) which plays a key role for sound perception since it represents one of the first cortical processing stations for sounds; (ii) auditory association cortex (AAC: A4, A5, STSdp, STSda, STSvp, STSva, STGa, and TA2) which is concerned with the memory and classification of sounds; (iii) posterior medial cortex (PMC: POS1, POS2, v23ab, d23ab, 31pv, 31pd, 7m); (iv) the temporo parieto occipital junction (TPOJ: TPOJ1, TPOJ2, TPOJ3, STV, PSL) which is a complex brain territory heavily involved in several high-level neurological functions, such as language, visuo-spatial recognition, writing, reading, symbol processing, calculation, self-processing, working memory, musical memory, and face and object recognition; and (v) the dorsal frontal lobe (DFL: L_55b, SFL, L_44, L_45, IFJA, IFSP) which covers the aspects of pragmatic processing such as discourse management, integration of prosody, interpretation of nonliteral meanings, inference making, ambiguity resolution, and error repair.
Encoding Model To explore how and where contextual language features are represented in the brain when reading sentences and listening to stories, we extract different features spaces describing each stimulus sentence and use them in an encoding model to predict brain responses. Our reasoning is as follows. If a feature is a good predictor of a specific brain region, information about that feature is likely encoded in that region. In this paper, for both datasets, we train fMRI encoding models using Ridge regression on stimuli representations obtained using a variety of NLP tasks. The main goal of each fMRI encoder model is to predict brain responses associated with each brain region given a stimuli. In all cases, we train a model per subject separately. Following literature on brain encoding  (Caucheteux et al.
, 2021b; Toneva et al. , 2020) , we choose to use a ridge regression model instead of more complicated models. We plan to explore more such models as part of future work. We follow K-fold (K=10) cross-validation. All the data samples from K-1 folds were used for training, and the model was tested on samples of the left-out fold. We used sklearn's ridge-regression with default parameters, 10-fold cross-validation, Stochastic-Average-Gradient Descent Optimizer, Huggingface for Transformer models, MSE loss function, and L2-decay (λ) as 1. 0. We used BERT Word-Piece tokenizer for the linguistic Transformer input. All experiments were conducted on a machine with 1 NVIDIA GEFORCE-GTX GPU with 16GB GPU RAM. We make the code publicly available 1.
Neural Language Tasks Similarity Computation To estimate the similarity between 10 language tasks, we took the prediction performance scores across all the voxels in Pereira (97,539) and Narratives-Pieman datasets (10,732). To analyze the relationship between tasks based on neural representations, we calculated the Pearson correlation between predicted voxels of each task with the remaining tasks. These Pearson correlation values were used to construct heatmaps and the task similarity trees(dendograms) using hierarchical clustering for Pereira and Narratives-Pieman datasets.
These Pearson correlation (τ ) results are comparatively much higher compared to those obtained using pretrained (task-agnostic) GPT2 model in  (Caucheteux et al. , 2021a ) (τ ranging from 0. 02 -0. 06). As shown in Fig. 2 , our method obtains much higher correlations (τ ranging from 0. 02 -0. 229). Similar to the Pereira dataset, we estimate the statistical significance of the performance differences using the one-way ANOVA test. The main effect of task was significant for all the ROIs with p≤ 10 -3 with confidence 95% (see Appendix for detailed ANOVA results). Also, Post hoc pairwise comparisons  (Ruxton and Beauchamp, 2008)  revealed that on both 2V2 accuracy and Pearson correlation measures, tasks such as PD, Sum, and NLI performed significantly better compared to other tasks (see Appendix for pairwise comparison results).
Language Task Similarity Computation Pearson correlation values between predicted responses for each pair of tasks were used to con-  struct the similarity matrix with heatmap for both Pereira and Narratives-Pieman datasets, as shown in Figs. 3  and 4. We observe that the following task pairs are highly correlated for the Pereira dataset: (NER and CR), (SS and CR) and (PD and Sum). Also these task pairs are highly correlated for the Narratives-Pieman dataset: (CR and NLI), (NLI and SA) and (PD and Sum). Similarities are relatively higher for Narratives-Pieman compared to the Pereira dataset. Surprisingly, the (NLI, SA) pair has lowest similarity for Pereira (reading) and close to highest in Narratives-Pieman (listening). We hypothesize that this is because sentiment is best conveyed while the subject is listening.
Reading sentences  (Pereira) : The stimulus sentences from the Pereira dataset were fed as input to each of the 10 task Transformers. The similarity among the resulting representations was analyzed using hierarchical clustering, and the clusters are visualized as dendrograms in Fig. 5  (left). We observe that the tasks are clustered into three groups denoted using red, green, and blue colors. Next, we wished to check if similar task grouping is observed on brain activations predicted by ridge regression trained on task-specific representations. Hence, similar clustering analysis was conducted on the neural space representations, and the clusters are visualized as dendrograms in Fig. 5  (right)  across all subjects. Interestingly, the tree derived from brain representation also shows a similar distribution of tasks across the three groups.
Brain maps for whole brain predictions The mean absolute error (MAE) between predictive and actual responses is obtained using individual task features from the taskonomy. MAE values are obtained for all the voxels in the brain for both the reading (Fig. 7 ) and listening datasets (Fig. 8 ). In the reading task, we observe from Fig. 7  that CR has lower MAE compared to PD which in turn has lower MAE compared to the NLI task (brain maps for the other tasks are reported in Fig. 17  in the Appendix). Overall, for the reading stimuli, tasks such as NLI, QA, and SA display higher MAE values. To further investigate which sub ROIs (LPTG, LMTG, LATG, LFus, Lpar, Lang, LIFGorb, LIFG, LaMFG, LpMFG, and LmMFG) of the Language network are related to the predictive task features, we train encoding models for all the sub ROIs for the best encoding task, i.
Discussion (1) We used a ridge regression model instead of more complicated models for encoding. We believe that more complex models can lead to further exciting insights. (2) We experimented with 10 NLP tasks. Models can be pretrained for more such tasks to check if other tasks are better predictive of voxel activations. (3) We leveraged models finetuned using datasets of different sizes across tasks. While a fair comparison of dataset sizes across tasks is impossible, we understand that this could have resulted in some bias in our results. (4) We used a different dataset for reading vs listening. While we believe that the differences in task-specific model performances across reading and listening are mainly due to the learned stimulus representations, but they could also arise from other factors such as experimental conditions, the text domain of the stimuli or number of voxels, etc.
Conclusion In this paper, we studied the effectiveness of task specific NLP models for brain encoding. We observe that building individual encoding models and exploiting existing relationships among models can provide a more in-depth understanding of the neural representation of language information. Our experiments on Pereira and Narrative datasets lead to interesting cognitive insights. A Details of the Finetuned Models We selected tasks for which BERT-base-cased finetuned models were available. Note that we did not finetune any of these models ourselves but leveraged the state-of-the-art finetuned models available on Huggingface. Details of the specific finetuned model checkpoints are mentioned in Table  3. B ANOVA test results.
B. 1 Pereira dataset The main effect of model was significant for the ROIs with 95% confidence with these statistics: • The main effect of model was significant for the ROIs with 95% confidence with these statistics: • EAC_L [F(9,810)=3. 88, p=. 00009] • EAC_R [F(9,810)=3. 34, p=. 00055] • AAC_L [F(9,810)=5. 37, p=. 0000007] • AAC_R [F(9,810)=6. 955, p=. 00000] • PMC_L [F(9,810)=37. 21, p=. 00000] • PMC_R [F(9,810)=31. 62, p=. 00000] • TPOJ_L [F(9,810)=9. 166, p=. 00000] • TPOJ_R [F(9,810)=7. 797, p=. 00000] • DFL_L [F(9,810)=12. 445, p=. 00000] • DFL_R [F(9,810)=12. 27, p=. 00000].
Objectives The hermeneutical objectives of computational literary scholars are at least as diverse as those of traditional literary studies scholars. Just as the pool of technological developments is quickly expanding, so too is the list of possible uses. Rather than try to provide an exhaustive inventory of paths and trends, we would like to mention a few prominent approaches and orientations, to catch a glimpse of the extraordinarily wide array of specific processes and results that are branching out in unforeseen directions and combining to form new protocols. Digital humanities conferences and publications make a habit of the unexpected, with scholars finding new, ingenious ways to explore datasets, collect overlooked data, undertake new research endeavours or perform seemingly impossible tasks.
, 2020; Byszuk et al. , 2020; Schöch et al. , 2016). And although quantitative analysis, by breaking texts down to sums of measurable elements, at first appears to be blind to the interpretative meaning of literary works, a number of paths do lead to insights about what texts actually mean, including, for instance, sentiment analysis  (Kim, 2020; Klinger et al. , 2020). The prospects of advancing literary history and literary theory are improved greatly both by approaching texts as boxfuls of words to create language models and by approaching literary movements as boxfuls of books to produce literary evolution models, thus following in the footsteps of Franco Moretti by looking at more than just the works that have kept being read. The statistical strategies of computational literary studies are equally wideranging, from descriptive statistics, which examines the characteristics of a dataset, through inferential statistics, in which these characteristics are tentatively generalized into a broader, virtual dataset, all the way to advanced machine learning, including topic modeling and deep learning, among others.
Moreover, many of them can be combined with a quantitative twist. But because poetry, among other literary genres, is such an intrinsically regulated device, we should never ignore its quantifiable aspects when examining it. It thus comes as no surprise that the field has been quick to accommodate computational approaches with very little friction: they have come as a welcome relief and have facilitated and furthered existing counting practices. Card files, where the researchers kept their records, have been replaced with modern databases, simple paper-and-pencil computations with more advanced methods and models. The goals have remained largely unchanged: identifying patterns within the multiplicity of forms, using these patterns to follow trends throughout literary history, identifying what sets one style apart from another, and testing hypotheses about what might actually constitute the efficacy or the uniqueness of a poetic work.
'" (Really, one should never speak of "given" but always of "obtained" [our translation]. )  Molière has been at the center of a very active debate around whether and why he did (or did not) write some of his plays (cf. Labbé, Labbé [2001] ;  Cafiero, Camps [2019] ;  Labbé [2019] ).  See https://tei-c. org/release/doc/tei-p5-doc/en/html/AB. html (accessed May 2, 2022). full and exact text itself; it also involves adding various annotations, automatically, manually, or both. The annotations transform a literary work into a dataset, operationalize it, and allow for a radically changed view, bringing together very localized phenomena and the scale of an entire corpus. When building a corpus, one can lean toward an immaculate, fully interoperable, detailed, and exhaustive result, nearing a digital edition of texts, or opt for a more robust, goal-oriented corpus, with just the level of granularity and exactness needed for the project at hand.
This can be managed for reasonably sized corpora provided one commits to a spirit of economy, only including the precise level of granularity required to fit one's specific needs. These corpora, which are snugly customized to one's own needs, are less likely to be reused by others but still require  Kraken OCR models: https://zenodo. org/communities/ocr_models(accessed May 2, 2022).  https://www. abbyy. com/cloud-ocr-sdk/, https://acrobat. adobe. com, https://readcoop. eu/ transkribus/(accessed May 2, 2022).  https://www. distant-reading. net/eltec/(accessed May 2, 2022). The Polite Revolution of Computational Literary Studies considerable effort. It is essential to refrain from collecting any more data than actually useful for the hermeneutic question at hand and to take care to only collect it in a way that provides satisfactory interoperability.
Noise When speaking of data, mention must be made of noise, as what is noise for some can be a signal for others or simply lead to the ruin of one's own data, depending on the quality of the dataset. When noise is made by outliers or by non-representative occurrences, or when the sheer volume of a very frequent phenomena is overpowering and obscures a less frequent but interesting trait, then this noise is actually data. One can choose to keep it or to remove it, one can also choose to extract it and study it specifically. But when noise is the result of too high a proportion of errors in the data, of an inadequate collection protocol, then noise is indeed just pollution. Efforts are then made to remove it or to account for it through error bars, but what it signals is really poor data quality.
Is it bad faith to admit large yet nonexhaustive corpora as reliable bodies when their content has not yet been disclosed? Is there imprecision in corpus delineation? Are their results reproducible? The reproducibility of  Moretti's findings (2005)  about the evolution of British novelistic genres has been questioned  (Riddell, 2013: 47-49; Shalizi, 2011: 118, 130-131). Other scholars have tested the intuitions presented by Moretti on the basis of a smaller dataset by trying them out on a larger scale. Picking up on  Moretti's (2011)  use of character networks as a way to study plots, Algee-Hewitt (2017), for instance, examines plot evolution in English drama since the sixteenth century. But whereas Moretti based his study on single plays, Algee-Hewitt offers a viable, large-scale implementation based on centrality measures in character networks for a corpus of 3,895 plays.
The Polite Revolution of Computational Literary Studies While some habits are being challenged or broken, many obstacles and perils are being maintained. Basing our assumptions on quantifiable, reproducible observations does not necessarily afford our arguments a higher degree of truth. At every step, from corpus preparation to criteria definition, from annotating to extracting, from statistics to interpretations, we have to make choices that are informed but inevitably subjective. This is how  Drucker (2011)  argued that "data are capta," highlighting their constructed and interpretive nature. Any research effort calls for individual sensibility, all the more so when it examines literature. The necessary economy in data collection requires a hierarchization of equally real phenomena. Literature research datasets, and the analyses drawn from them, are inevitably informed and colored by a team's assumptions, hypotheses, and way of thinking.
g. , the CATMA annotation platform discussed in this volume) or by making their approach known and potentially reproducible. Indeed, it is not uncommon for conference submissions in digital humanities and computational humanities journals, particularly in natural language processing, to publish accompanying code and datasets. Many of us are putting together patchworks of borrowed and custom-built devices to do one job, making modest contributions to the pooled resources of computational literary scholars. In a similar movement of methodological hybridization, computational, distant reading methods and more traditional, close reading can be intertwined, the hermeneutical gain deriving from the diversification of angles and focal points, a mixing favorable to the emergence of unexpected observations.
We were trying out tools that became much easier to use (for me at least) with the program Hyperbase. From 2001, Hyperbase became my main tool, which I have used on many corpora since, whereas Deconinck-Brossard applied different tools to her homiletic corpora. Hyperbase is the brainchild of Etienne Brunet, a French scholar and developer, who had started with published frequency indexes of an author or a period: Le vocabulaire de Jean Giraudoux: structure et évolution (1978), Le vocabulaire de  Proust (1983) , Le vocabulaire français de 1789 à nos jours (1981). 5 I invited him to my master's seminar in 2007, and my students presented computer-aided analyses they had prepared on a corpus of modernist short stories as a basis for discussion. In a few clicks, Hyperbase provides user-friendly graphs.
This "ongoing project [aims at] making freely accessible full-text transcripts of some of the rarest works in the Chawton House library collection," which consists of "works by women, mostly in English, and mostly within the period 1600-1830. " 17 I had found my second treasure trove, thanks to which I could combine all my experience, working with a large quantity of reliable electronic texts both canonical and non-canonical. I set up a project entitled "Strategies of Writing: Women's Fiction in the Long Eighteenth-Century, a Corpus-Based Stylistic Analysis" and read up on critical work by scholars linked to Chawton as well as recent advances in what was now being called corpus stylistics  (Mahlberg, 2007; Fischer-Starcke, 2010; Jockers, 2013) , and the Stanford Literary Lab pamphlets (https://litlab.
Conclusion What Corpus Stylistics can do beyond the obvious provision of quantitative data is help with the analysis of an individual text by providing various options for the comparison of one text with groups of other texts to identify tendencies, intertextual relationships, or reflections of social and cultural contexts. (Mahlberg, 2007: 221)  When I came upon Mahlberg's work, I related strongly to the new name given to the kind of research I had been doing all along: it asserts equal status with corpus linguistics and shows the evolution of the work carried out since the 1950s and 1960s by the pioneers who called it lexicometry or stylostatistics. The recent definition of style by Herrmann et al. takes corpus stylistics into account, but not exclusively: "Style is a property of texts constituted by an ensemble of formal features which can be observed quantitatively or qualitatively"  (Herrmann et al.
One such tool is the annotation software CATMA, which I discuss briefly as "proof of concept" for a new practice of digital text analysis termed "scalable reading. " This epistemic practice explores the continuum between "close" and "distant" reading approaches in a methodical and controlled fashion that is designed to bring into fruitful contact the phenomenological and the phenomenal, i. e. , the empirical perspective on poetic texts and poetry and their conceptualization. Setting such an ambitious objective has significant consequences for the design and development of software like CATMA, for it requires us to approach our task with a data and workflow model in mind that can map the fuzzy yet highly productive logic of the discursive hermeneutic circle onto the rigid base layer of non-ambiguous binary code.
Building CATMA as a hermeneutic annotation platform Developing such an annotation tool has been the goal of the CATMA project that my team and I started in 2008. CATMAshort for Computer Assisted Text Markup and Analysisbuilds on John Bradley's DOS-program suite TACT (Textual Analysis Computing Tools) from the mid-1980s, one of the first desktop applications for digital text annotation. TACT soon became popular in the new field of humanities computing that had just begun to evolve at the time. Today, its successor CATMA is an open-source, collaborative digital text annotation and analysis platform made freely available as a web service (at https://catma. de), working with digital texts in any (UTF-8-encoded) language and used by individual researchers and in research projects worldwide. The technology and architecture of the CATMA web service programmed in JAVA might now be considered cutting edge.