# A large-scale multi-disciplinary analysis of uncertainty in research articles

Nicolas Gutehrlé, Panggih Kusuma Ningrum, Iana Atanassova

► To cite this version:

Nicolas Gutehrlé, Panggih Kusuma Ningrum, Iana Atanassova. A large-scale multi-disciplinary analysis of uncertainty in research articles. Data and Knowledge Engineering, 2026, 163, 10.1016/j.datak.2026.102561. hal-05527274

HAL Id: hal-05527274

https://hal.science/hal-05527274v1

Submitted on 25 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ELSEVIER

# A large-scale multi-disciplinary analysis of uncertainty in research articles☆

Nicolas Gutehrlé<sup>a, ID</sup>, Panggih Kusuma Ningrum<sup>a, ID</sup>, Iana Atanassova<sup>a,b ID,*</sup>

## ARTICLE INFO

Dataset link: https://github.com/Project-InSciM/Scientific-Uncertainty-Extractor

**Keywords:**

Uncertainty

Research article

Multi-disciplinary analysis

Semantic annotation

Full text

## ABSTRACT

Scientific uncertainty is inherent to the research process and to the production of new knowledge. In this paper, we present a large-scale analysis of how scientific uncertainty is expressed in research articles. To perform this study, we analyze the Const-L dataset, which consists in 31,849 research articles across 16 disciplines published over more than two decades. To identify and categorize uncertainty expressions, we employ the UnScientify annotation system, a linguistically informed, rule-based approach. We examine the distribution of uncertainty across disciplines, over time, and within the structure of articles, and we analyze its contexts and objects. The results show that the Social Sciences and Humanities (SSH) tend to have a higher frequency of uncertainty expressions than other fields. Overall, uncertainty tends to decrease over time, though this trend varies across disciplines. Moreover, correlations can be observed between the uncertainty expressions and both article structure and length. Finally, our findings provide new insights into scientific communication, by indicating distinctive disciplinary patterns in the ways uncertainty is expressed, as well as shared and field-specific research objects associated with uncertainty.

## 1. Introduction

Uncertainty in science is an integral part of the research process and of the production of new knowledge. Research generates uncertainties through various processes, such as the use of tools or observations with margins of error, abductive and inductive reasoning, as well as the expression of researchers' intuitions and interpretations of data.

In this paper, we propose a large-scale linguistic analysis of scientific uncertainty in a dataset of full text research articles. More specifically, we focus on the distribution of uncertainty across disciplines and time, and investigate its correlations with the structure of articles. Finally, we propose a comparative linguistic analysis of the contexts and objects of uncertainty across the various disciplines, by examining the noun groups occurring close to uncertainty expressions.

Uncertainty is a complex concept. The literature provides a broad range of meanings and interpretations of the term, such as lack of knowledge, doubt, inconsistency, ambiguity, vagueness, subjectivity, error, risk, ignorance, and contradiction [1–4]. For the purposes of this study, we focus on scientific uncertainty that is explicitly expressed in articles as a lack of knowledge or lack of information. More specifically, our aim is to identify sentences in articles that contain scientific uncertainty, defined as follows: *the expression of a lack of knowledge or a lack of precision in the information on a subject or a concept identified in a scientific text.*

<footnote><sup>a</sup> Université Marie et Louis Pasteur, CRIT, Besançon, F-25000, France</footnote>

<footnote><sup>b</sup> Institut Universitaire de France (IUF), Paris, France</footnote>

<footnote>☆ This article is part of a Special issue entitled: 'EGC 2025: Akoka' published in Data & Knowledge Engineering.</footnote>

<footnote>* Corresponding author at: Université Marie et Louis Pasteur, CRIT, Besançon, F-25000, France.</footnote>

<footnote>E-mail address: iana.atanassova@umlp.fr (I. Atanassova).</footnote>
To conduct this study, we introduce Const-L, a new dataset consisting of 31,849 research articles in 16 different disciplines and published between 2000 and 2021. To select an annotation method for this dataset, we conducted a comparative evaluation of several Small and Large Language Models as well as a rule-based model, assessing their performance using F-measures and Accuracy scores. We adopt as the primary annotation system UnScientify [5,6], a weakly supervised annotation system that uses linguistic knowledge to identify verbally expressed uncertainty in texts. This choice is motivated by the comparative evaluation of several different annotation methods for this task. We then annotate the Const-L dataset and subsequently analyze the distributions of uncertainty across disciplines and time, as well as its evolution along text progression and linguistic contexts.

The rest of the article is organized as follows: in Section 2, we present the concept of uncertainty, as well as a state of the art of works on its identification in texts. In Section 3, we describe the AURORA-MESS dataset which is used for the evaluations and the new Const-L dataset, with its sources, preparation process and characteristics. In Section 4, we explain first the evaluations that led to the choice of the annotation system, and then we present the overall methodology for the analysis of the distributions of uncertainty in the Const-L dataset. The results are presented in Section 5 on the distribution of uncertainty across disciplines and publication years, its correlations with article length and text progression, as well as the contexts and objects of uncertainty. We discuss the results in Section 6 and finally provide a conclusion in Section 7.

## 2. Related works

Uncertainty is a complex concept, and the literature offers several definitions of the term (see for example [7–9]). Uncertainty is one of the components of the results of scientific research, with multiple dimensions in the different disciplinary fields. Conceptually, uncertainty needs to be differentiated from subjectivity, while taking into account the epistemic modality. However, the concepts and relations around the notion of modality are not consensual [10]. In scientific texts, Ningrum et al. [6] propose a conceptual definition of scientific uncertainty, as follows:

> A sentence expresses uncertainty if it (explicitly) expresses a state of absence of knowledge, insufficient knowledge, or lack of precision in information about a concept or subject.

The identification of textual segments expressing uncertainty has been the focus of several works. For example, the identification of speculative phrases in texts by machine learning approaches has been addressed by Moncecchi et al. [11], which highlights the specificity of the subjectivity problem. A state of the art in this field is presented by Diaz and López [12]. The study by Chen et al. [13] proposes to identify expressions for introducing uncertainty by expanding a restricted set of expressions. However, the latter work expresses a binary vision of uncertainty, and does not address the different levels and dimensions of uncertainty in order to capture the complexity of this notion. To go further, Atanassova et al. [14] construct a list of strong indicators of uncertainty and observe their distributions within the rhetorical structure of articles in Biomedicine and Physics.

Further, the problem of interdisciplinary and conceptual understanding of the notion of uncertainty was addressed by Rey et al. [15], through the study of a corpus of scientific articles dealing with global warming. They propose a relational schema of scientific uncertainty, where the uncertainties expressed are organized in the form of classes according to the type of reasoning used (abductive, inductive, deductive) and the presence or absence of quantitative indications of uncertainty.

However, identifying and categorizing the uncertainty associated with scientific knowledge through the study of textual content remains a challenge. The fundamental problem lies in the difficulty of working with textual data present in scientific articles. The majority of previous studies have focused on detecting and identifying a specific set of uncertainty cues or markers in abstracts of scientific articles only [16,17] or in full text articles [14,18,19]. A wide range of techniques has been explored to detect and analyze uncertainty or related phenomena such as hedging, modality, and factuality in text. These techniques generally focus on two main objectives: treating uncertainty detection as a sentence-level classification task, where the text is categorized as either expressing uncertainty or not, and identifying specific linguistic cues or markers of uncertainty. The approaches used in previous studies can be grouped into three categories: rule-based methods, machine learning approaches, and deep learning or Transformer-based methods. Each offers unique advantages and faces distinct limitations.

Rule-based techniques generally rely on predefined cues, markers, and linguistic patterns to detect uncertainty in text. The earliest approaches have in that regard typically employed handcrafted lexicons to identify linguistic indicators of uncertainty [20,21]. However, as the contextual meaning of cues alone can significantly vary, simple cue matching is not sufficiently reliable to detect the expression of uncertainty [22]. To address these limitations and capture more complex linguistic structures, works such as Chapman et al. [23], Atanassova et al. [24] and Rey [25], have incorporated syntactic patterns and regular expressions (regexes), whereas works such as Omero et al. [26], Müller et al. [27] and Müller [28] have leveraged linguistic insights, such as frame-linguistic modeling or corpus-linguistics. However, rule-based methods are usually difficult to generalize due to being confined to specific domains and contexts. Moreover, the rigidity of regexes limit their ability to capture the variability inherent to natural languages.

Machine-learning (ML) approaches, on the other hand, generally rely on feature-rich representations that include surface-level attributes, part-of-speech (POS) tags, and syntactic relations to perform tasks such as text classification or sequence labeling [29–36]. Several studies have also proposed to employ weakly supervised method to leverage the limited availability of annotated dataset for uncertainty detection [18,37]. One major limitation of ML approaches however lies in their reliance on handcrafted features and predefined cues. As such, ML approaches generally struggle to capture the nuanced semantics and syntactic intricacies of uncertainty expressions. They also seem to struggle with detecting complex uncertainty assertions and long-distance dependencies, such as if-clauses constructions [26].
<table><caption>**Table 1**
Sample of annotated sentences.</caption><thead><tr><td>Sentence</td><td>Uncertainty</td><td>Authorial Ref.</td></tr></thead><tbody><tr><td>It is possible that corticosteroids prevent some acute gastrointestinal complications.</td><td>Yes</td><td>Author(s)</td></tr><tr><td>In this test, a likelihood ratio test statistic is calculated for the two tree versus one tree models, and compared to a null distribution generated by non-parametric bootstrapping (see Methods).</td><td>No</td><td>—</td></tr><tr><td>Previous meta-analyses have shown a significant benefit for NaHCO3 in comparison to normal saline (NS) infusion [6,7], although they highlighted the possibility of publication bias.</td><td>Yes</td><td>Former/Prev. study(s)</td></tr></tbody></table>

Finally, the recent progress in deep-learning and Transformer-based approaches, relying on models such as BERT, XLNet, RoBERTa, GPT, or Mistral to name a few, have allowed to perform text and token classification tasks by integrating contextual embeddings and word representations. As such, they have shown significant advancements in the task of detecting uncertainty markers [6,38]. However, the conceptual boundaries of their training datasets generally focus on simplified classification tasks. As such, the ability of these approaches to effectively capture the expression of complex scientific uncertainty remains to be evaluated.

## 3. Datasets

This study uses two datasets: the existing AURORA-MESS dataset which supports the evaluation of the annotation of uncertainty in scientific publications; and the Const-L dataset that has been created for the specific purposes of this study. In the following subsections we describe these two datasets.

### 3.1. AURORA-MESS dataset

The AURORA-MESS dataset [39]<fnref n="1" /> is a corpus of 975 sentences extracted from 312 articles across 59 journals. It covers publications in both fields of Science, Technology, and Medicine (STM) as well as Social Sciences and Humanities (SSH). Each sentence is labeled for the presence of scientific uncertainty and the type of reference to uncertainty, distinguishing between author-originated, cited prior work, or shared uncertainty. The dataset supports empirical studies on the expression of scientific uncertainty across domains. About 45% of the sentences in this dataset are labeled as containing uncertainty and the rest are sentences that do not express uncertainty. Table 1 presents examples of annotated sentences from the AURORA-MESS dataset.

### 3.2. Const-L dataset

For the purposes of this study, we require a large dataset of full text articles that are classified by discipline, cover various scientific fields, and span over a long period of time. Thus, we present here the construction of the Const-L dataset, a large scale multidisciplinary dataset of scientific articles written in English and spanning over two decades of publications.

**Data source.** The source of our dataset is the Constellate platform<fnref n="2" /> which provides access to articles from several providers (JSTOR, Portico, Chronicling America, DocSouth, South Asian Open Archive) in multiple languages and disciplines. We chose this platform to build our dataset instead of other Open Access libraries and databases such as PubMed, arXiv, SSOAR, Microsoft Academic Graph or Mendeley, as only Constellate provided access to multiple decades of publication, as well as the full text content of articles, which are classified into disciplines.

As our aim is to build a dataset that is representative of the various disciplines, we only selected the research articles that have been categorized as belonging to one single discipline. Thus, we collected 31,849 articles from 16 different disciplines published between 2000 and 2021. The collected documents are stored in the JSON format, and include several metadata (e.g. authors' names, affiliations, publication year, ...) alongside the full text content of the article.

**Dataset preparation.** The Constellate platform provides the full text of the articles divided into lines, which correspond to the physical line breaks of the original document.<fnref n="3" /> As the text is not structured otherwise, we needed to prepare the dataset for further processing by segmenting the full text articles into sections, paragraphs and sentences. To do this, we applied the following pipeline:

Sections were identified by detecting the lines in the text that introduce standard section titles (e.g. Introduction, Methods, Conclusion...), and thus determining section boundaries where possible. We use sets of regular expressions to capture the possible

<footnote n="1">¹ Annotated Uncertainty and Reference in Open Research Articles for Multidisciplinary and Empirical Social Science (AURORA-MESS): https://doi.org/10.5281/zenodo.15001249.</footnote>

<footnote n="2">² Link to the Constellate platform: https://constellate.org. The platform was discontinued in July 1, 2025. JSTOR continues to provide access to the metadata and full text of their dataset (see https://www.jstor.org/ta-support/).</footnote>

<footnote n="3">³ This format results from an Optical Character Recognition (OCR) processing performed by the platform to recover the content of document from PDF format.</footnote>
<caption>**Table 2**</caption>

Distribution of articles and sentences by discipline in the Const-L dataset. Disciplines are sorted based on the number of collected articles.

<table><thead><tr><th>Discipline</th><th>Number of articles</th><th>Number of sentences</th></tr></thead><tbody><tr><td>Applied Science</td><td>5,595</td><td>1,126,061</td></tr><tr><td>Mathematics</td><td>5,261</td><td>1,126,818</td></tr><tr><td>Physical Sciences</td><td>4,489</td><td>1,062,823</td></tr><tr><td>Biological Sciences</td><td>3,694</td><td>789,653</td></tr><tr><td>Health Sciences</td><td>2,716</td><td>412,472</td></tr><tr><td>Linguistics</td><td>1,661</td><td>389,983</td></tr><tr><td>Information Sciences</td><td>1,519</td><td>314,297</td></tr><tr><td>Law</td><td>1,390</td><td>296,101</td></tr><tr><td>Economics</td><td>1,308</td><td>300,917</td></tr><tr><td>Business</td><td>1,085</td><td>186,549</td></tr><tr><td>Religion</td><td>630</td><td>123,217</td></tr><tr><td>Social Sciences</td><td>625</td><td>154,288</td></tr><tr><td>Political Sciences</td><td>518</td><td>91,705</td></tr><tr><td>Arts</td><td>493</td><td>85,028</td></tr><tr><td>History</td><td>462</td><td>63,049</td></tr><tr><td>Philosophy</td><td>403</td><td>75,653</td></tr><tr><td>Total</td><td>31,849</td><td>6,598,614</td></tr></tbody></table>

variations in the forms of section titles. This approach is inspired by the works of Bertin et al. [40] who tackle a similar problem of identifying the IMRaD structure in articles.

Paragraph breaks were identified based on the hypothesis that a line can be the end of a paragraph if it is much shorter than the lines surrounding it. We introduce the following threshold:

$$T(len_1, len_2) = C * \max(len_1, len_2)$$

where *len*₁ and *len*₂ are the lengths measured as numbers of characters of two lines of text *l*₁ and *l*₂. *C* = 1 if the line *l*₁ ends with a period character (.), and *C* = 0.65 otherwise. This translates the fact that sentence ends that are followed by a line break are most certainly paragraph breaks if the line is shorter than its surrounding lines. For a given line *l*, its preceding line *pl* and its following line *fl*, we consider that *l* is a paragraph break if *l* ≤ *T(l, pl)* or *l* ≤ *T(l, fl)*.

Several additional rules account for possible errors in the full text dataset. Any line other than a section title that ends by a lower case letter and is followed by a lower case letter cannot be a paragraph break. Similarly, blocks shorter than 10 characters are not paragraphs, as they might correspond to errors or to non-textual elements in the document such as tables or equations.

The paragraph breaks defined above allow to divide the documents into blocks of textual content. These blocks are then categorized in order to distinguish between the body of the articles and other textual elements such as the article's title, the affiliations of the authors or the references. Finally, we segment all paragraphs into sentences.

**Dataset description.** The Const-L dataset consists of 31,849 articles, for a total of 6,598,614 sentences. The distribution of disciplines by number of articles and sentences in the Const-L dataset is presented in Table 2. We note that the dataset is not balanced across disciplines: Applied Science and Mathematics reach 5595 and 5261 articles, while History and Philosophy are represented by 462 and 403 articles respectively.

Fig. 1 presents the distribution of articles per publication year, ranging from 2000 to 2021. On average, 91 articles were published per year and per discipline. The amount of articles published per year starts from 171 articles in 2000, and slowly increases over the years until reaching a top value of 3436 articles in 2021, with a drop in 2015. Major differences in the coverage of the various disciplines can be observed. For example, the disciplines of Applied Science and Mathematics reach top values of 778 and 754 articles in 2020 and 2014 respectively, whereas Political Sciences and Business have only 2 and 1 articles in 2001 and 2000 respectively.

Fig. 2 presents the distribution of article lengths per discipline, measured as the number of sentences per article. The article length is relatively stable across disciplines, with an average of 250 sentences per article. The shortest articles are found in History and Health Science, with an average of 168 and 183 sentences per article respectively, whereas the longest articles are found in Social Sciences and Linguistics, with an average of 312 and 300 sentences respectively.

# 4. Method

## 4.1. Annotation system

We identify expressions of uncertainty at the sentence level using the UnScientify model [41].<fnref n="4" /> This automatic annotation system, which is built upon the spaCy framework, integrates four core components:

<footnote n="4">⁴ A web-based demo is available at https://bit.ly/unscientific-demo.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 1. Number of articles per discipline and publication year.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2. Distribution of article lengths per discipline.</figcaption></figure>

1. A Text Pre-Processing Unit, designed to ensure data compatibility prior to analysis.

2. A Scientific Uncertainty Span Finder, which employs a pattern-matching process using a comprehensive lexicon and a list of 89 patterns derived from 12 scientific uncertainty (SU) pattern groups.

3. A Contextual Shift Handler, which performs complex sentence checks to identify the presence of rebuttal elements, such as negation, confirmation, or neutral statements, that may alter the intended meaning.

4. An Authorial Reference Identifier, which determines the authorship of uncertainty expressions by distinguishing whether they originate from the current authors, previous studies, or a combination of both. This level of annotation is not exploited in the present study.

Previous studies [5,42,43] have evaluated the performance of UnScientify in identifying scientific uncertainty expressions, comparing it against a range of alternative approaches under two key conditions: (1) scenarios with limited labeled training data, and (2) the need for interpretable outputs. The alternative approaches included supervised fine-tuning (SFT) of BERT-based models, ranging from generic pretrained BERT including DistILBERT, and RoBERTa, to domain-specific variants such as SciBERT, and SciBERT. Few-shot learning was also tested using models like TARS [44] and SetFit [45], which are designed for efficient text classification with limited annotations. In addition, various in-context learning (ICL) strategies were explored, including Natural Language Inference-Based Zero-shot Text Classification (NLI-BZSTC) and prompting techniques that leverage zero-shot, few-shot, and Chain-of-Thought combinations. The most recent evaluations conducted by Ningrum [43] and Ningrum [42], which advance the work conducted by Ningrum et al. [5], used the AURORA-MESS dataset.

Fig. 3 summarizes the evaluation results reported in previous studies [42,43]. As shown in Fig. 3, UnScientify consistently outperformed LLMs across all tested approaches, achieving a Precision score of 0.720, a Recall score of 0.956, an F1-score of 0.822, and an Accuracy score of 0.816. Notably, while LLMs were evaluated under comparable conditions using supervised fine-tuning (SFT) and in-context learning (ICL) strategies such as zero-shot, few-shot, and Chain-of-Thought combinations, UnScientify demonstrated superior efficiency and robustness with limited labeled data.
An external validation was also conducted by Ningrum [43] with the MUSIC-OpRA dataset<fnref n="5" /> [46], comprising 1548 sentences from 283 articles across 22 journals, further demonstrated the model's robustness. On this dataset, UnScientify achieved improved overall Accuracy (0.841), and F1-score (0.845), confirming its generalizability across disciplinary domains.

Furthermore, UnScientify showed markedly higher prediction speeds when evaluated on the AURORA-MESS dataset [43]. It processed the entire dataset in 158.97 s, with an average of 0.16 s per sentence. By comparison, top-performing LLMs such as SetFit required 210.94 s on the same hardware, while RoBERTa exceeded 2000 s. These results demonstrate the significant computational overhead of Transformer-based inference and confirm UnScientify's suitability for scenarios requiring rapid, large-scale processing. The deterministic rule-based architecture of UnScientify uses predefined linguistic patterns instead of resource-intensive neural computations. By avoiding iterative forward passes through deep networks, the system achieves rapid and consistent performance, thus making it particularly well suited for real-time applications and large-scale processing.

Based on the above considerations, UnScientify was selected as the core model for identifying the expressions of scientific uncertainty in the present study. It consistently outperformed alternative approaches, demonstrating higher accuracy. Furthermore, it is interpretable, and requires less computational resources than the alternatives we have considered.

## 4.2. Distributional analyses of the Const-L dataset

After performing the full annotation of the Const-L dataset in order to identify sentences expressing uncertainty using UnScientify, we exploit these annotations as a source to study uncertainty across disciplines and time. In particular, we perform the following analyses.

Firstly, we study and compare the distributions of sentences expressing uncertainty across disciplines as well as with respect to publication years. To do this, we consider the proportion of uncertainty in articles, which is measured as the proportion of sentences expressing uncertainty relative to the total number of sentences in each article.

Secondly, we study the presence of uncertainty based on the length of articles as well as its evolution along the text progression of article. We measure the article length as the number of sentences in the article, whereas we represent text progression as a rounded percentage of this number, from 0 to 100. We then measure the proportion of uncertainty at each position of the text progression for each article. In this process, we note that the structure of the papers in terms of sections may differ between disciplines and journals.

Finally, we study the context and objects of uncertainty in each discipline by examining the linguistic content of the sentences that were annotated as expressing uncertainty. We extract the noun groups from sentences expressing uncertainty using the spaCy tool. In general, noun groups are expressions that are composed of nouns which may be accompanied by determiners and adjectives. We convert all noun groups to lower case and base forms. To determine the most relevant noun groups related to uncertainty in each discipline, we calculate the association scores between the nouns and the disciplines using the c-TF-IDF measure [47]. Thus, after identifying and ranking the nouns, we are able to better understand the primary subjects or concepts that are related to expressions of uncertainty in each discipline in our dataset. This method of analysis has been applied for similar tasks to other datasets by Atanassova et al. [48].

# 5. Results

The application of UnScientify on the Const-L dataset resulted in a total of 801,322 sentences expressing uncertainty. The distribution per discipline of sentences expressing uncertainty is presented in Table 3. The percentage of uncertainty in the different disciplines varies between 10% for Applied Science and 18.27% for Law.

## 5.1. Distribution of uncertainty across disciplines and publication years

Fig. 4 presents the box plot of the proportions of sentences expressing uncertainty in articles per discipline. We notice that the field of Social Sciences and Humanities (SSH) express the more uncertainty, as the disciplines of Law (14.483%), Social Sciences (14.084%), Philosophy (13.735%), History (13.241%) and Economics (13.092%) achieve the highest proportion of uncertainty on average. Conversely, Health Sciences (9.525%), Biological Sciences (9.317%), Physical Sciences (8.631%), Applied Science (8.139%) and Arts (8.135%) achieve the lowest mean proportion of uncertainty. These later disciplines, except for Arts, belong to the field of Science, Technology and Medicine (STM).

To analyze this phenomenon over time, Fig. 5 presents the box plot of proportions of sentences expressing uncertainty in articles per publication year. Uncertainty in articles appears to slightly decrease over the twenty-year span covered by our dataset, as the years with the highest proportion on average are 2005, (12.130%), 2002 (11.664%), 2003 (11.650%), 2008 (10.843%) and 2010 (10.814%) whereas the years with the smallest proportion on average are 2016 (9.721%), 2019 (9.657%), 2020 (9.361%), 2018 (9.304%), 2021 (8.744%). This decrease remains slow, as the covered period achieve a mean proportion of uncertainty of 10.271%.

To characterize the evolution of the expression of uncertainty in the different disciplines, we have plotted the mean proportion of uncertainty per discipline and per publication year. Fig. 6 presents these results. Two trends are noticeable on this figure, which are independent of the scientific field. On the one hand, for the disciplines of Applied Sciences, Arts, Biological Sciences, Information

<footnote n="5">⁵ Multidimensional Uncertainty in Scientific Interdisciplinary Corpora for Open Research Article (MUSIC-OpRA): https://zenodo.org/records/15173356.</footnote>
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Performance comparison of UnScientific and alternative approaches for the annotation of uncertainty [42,43]. The best scores for each metric is circled.</figcaption></figure>

Sciences, Linguistics, Mathematics, Physical Sciences and Religion, the proportion of uncertainty increases from 2000 until 2009, before slowly decreasing until 2021, as shown on top of Fig. 6. On the other hand, for the disciplines of Business, Economics, Health Sciences, History, Law, Philosophy, Political Sciences and Social Sciences, the proportion of uncertainty keeps decreasing over time, as shown on the bottom of Fig. 6. Regardless of these trends, the evolution of uncertainty in each discipline remains generally stable, except for the fields of Religion, Political Sciences, Arts, History and Philosophy which show the most variations. This might however be caused by the smaller number of documents that represent these disciplines in the dataset.

Overall, the above-mentioned results suggest that disciplines belonging to the SSH field tend to express more uncertainty than disciplines in other fields such as STM. Moreover, the presence of uncertainty in research articles tends to slowly decrease over the twenty-year span between 2000 and 2021, although the variations are not significant. Besides this, the evolution of the use of uncertainty across time does not seem to be correlated with the field to which the disciplines belong.
<figure><img src="image_6.png" /><figcaption>Fig. 4. Proportion of uncertainty in articles per discipline.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. Proportion of uncertainty in articles per publication year.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6. Proportion of uncertainty in article per discipline and publication year.</figcaption></figure>
<caption>**Table 3**</caption>

Sentences expressing uncertainty by discipline in the Const-L dataset, as absolute value and percentage of the total sentences per discipline.

<table><thead><tr><td>Discipline</td><td>Uncertainty</td><td>Percentage</td><td>Total sentences</td></tr></thead><tbody><tr><td>Mathematics</td><td>131,563</td><td>11.68%</td><td>1,126,818</td></tr><tr><td>Applied Science</td><td>112,653</td><td>10.00%</td><td>1,126,061</td></tr><tr><td>Physical Sciences</td><td>116,578</td><td>10.97%</td><td>1,062,823</td></tr><tr><td>Biological Sciences</td><td>86,952</td><td>11.01%</td><td>789,653</td></tr><tr><td>Health Sciences</td><td>46,173</td><td>11.19%</td><td>412,472</td></tr><tr><td>Linguistics</td><td>49,569</td><td>12.72%</td><td>389,983</td></tr><tr><td>Information Sciences</td><td>38,881</td><td>12.37%</td><td>314,297</td></tr><tr><td>Economics</td><td>51,026</td><td>16.95%</td><td>300,917</td></tr><tr><td>Law</td><td>54,102</td><td>18.27%</td><td>296,101</td></tr><tr><td>Business</td><td>25,896</td><td>13.88%</td><td>186,549</td></tr><tr><td>Religion</td><td>17,549</td><td>14.24%</td><td>123,217</td></tr><tr><td>Social Sciences</td><td>26,972</td><td>17.48%</td><td>154,288</td></tr><tr><td>Political Sciences</td><td>12,894</td><td>14.06%</td><td>91,705</td></tr><tr><td>Arts</td><td>9,233</td><td>10.86%</td><td>85,028</td></tr><tr><td>Philosophy</td><td>12,332</td><td>16.30%</td><td>75,653</td></tr><tr><td>History</td><td>8,949</td><td>14.19%</td><td>63,049</td></tr><tr><td>Total</td><td>801,322</td><td>12.14%</td><td>6,598,614</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>**Fig. 7.** Proportion of uncertainty as a function of article length.</figcaption></figure>

## 5.2. Distribution of uncertainty with respect to article length and text progression

Fig. 7 presents a box plot of the proportion of uncertainty as a function of article length. This allows us to observe whether uncertainty is equally distributed across short and long articles. The proportion of uncertainty appears to slowly increase with the length of articles, and reaches a top mean value of 11.303% for articles that are 500 to 999 sentences long. This suggests that the proportion of uncertainty expressed in an article is correlated with its length. However, articles that have 1000 sentences or longer contain relatively less uncertainty, with a mean value of 5.711%, which could be explained by the small number of such articles in the dataset.

Fig. 8 presents the proportion of uncertainty along the text progression in the articles per discipline. For most disciplines, the peak of uncertainty is located between 40% and 70% of the text progression, whereas all disciplines show an important decrease of uncertainty after 70% of the text progression, which roughly corresponds to the conclusion part. However, notable differences exist in the evolution of uncertainty along the first 70% of the text progression between the disciplines. We note four different trends:

1. Law, Philosophy, History, Religion show a continued increase of uncertainty.

2. Economics, Business, Information Sciences, Applied Sciences also show an increase, but to a lower extent.

3. Social Sciences, Biological Sciences, Physical Sciences, Political Sciences, Linguistics and Health Science appear to reach two peaks, one in the 0%-20% portion and another in the 40%-70% portion of the text progression. This could be a trace of the IMRaD structure of articles which is widely used in experimental sciences, where the first 20% of the article correspond roughly to the introductory part, and the 40%-70% correspond to the Results and Discussion sections.

4. Mathematics shows a continued decrease along the text progression whereas Arts remains relatively stable.
<figure><img src="image_10.png" /><figcaption>Fig. 8. Proportion of uncertainty along the text progression of articles per discipline. Disciplines are grouped according to the trend.</figcaption></figure>

## 5.3. Contexts and objects of uncertainty

For each discipline in the Const-L dataset, Table 4 presents the top ten nouns appearing in sentences that express uncertainty and having the highest association scores with the discipline. The nouns that are specific to only one discipline are presented in bold. This type of analysis provides insight into some important disciplinary differences, by pointing out what are the objects of study that are most commonly expressed near hypotheses, interpretations and research questions. This can provide an indication of the most important fields of innovation in a given discipline, especially when considered in contrast with the rest of the disciplines.

For example, in the Social Sciences, uncertainty is often associated with nouns such as *adolescent*, *parent* and *individual*, which are at the heart of important research inquiries and are specific to this discipline. In Biological Sciences we have the nouns *effect*, *change* and *soil*. Some of the nouns that we see in the table are very common, such as *people*, *study* and *woman*, that appear in seven, six and five disciplines respectively. This means that the research questions related to these objects are studied by several disciplines. Some disciplines display as much as six exclusive nouns which shows their specificity: e.g. *retailer*, *consumer*, *customer*, *company*, and *service* for Business; *firm*, *income*, *country*, *cost*, *household*, and *price* for Economics; *reader*, *music*, *audience*, *story*, *film*, and *way* for the Arts. Conversely, Information Science only holds two exclusive nouns, namely *research* and *author*.

Fig. 9 presents the heatmap of the distribution of nouns that are shared between each two disciplines, based on Table 4. It allows to reveal strong association between disciplines in terms of object of study. The disciplines of Religion and History are the ones that have the most nouns in common, i.e. *book*, *church*, *god*, *jesus*, *people*, which means that there exists a strong relation between these two disciplines. Other pairs of disciplines have as many as four nouns in common: Political Sciences and Law share the nouns *article*, *law*, *right*, and *state*, Linguistics and Social Sciences share the nouns *child*, *participant*, *student* and *study*, Information Sciences and Applied Science share the nouns *datum*, *number*, *time* and *user*, and finally the disciplines of Health Sciences and Biological Sciences share the nouns *development*, *patient*, *study* and *treatment*.

In terms of diversity, the disciplines of Social Sciences and Economics share noun groups with respectively 11 and 10 other disciplines, making them the ones sharing nouns with most other disciplines. This indicates that the research questions of these disciplines are often related to other fields of study, and thus making possible a wide range of interdisciplinary studies. Conversely, the disciplines of Physical Sciences and Applied Sciences stand out as sharing the smallest number of nouns with other disciplines, i.e. four other disciplines. Thus, the Physical Sciences have common nouns only with the disciplines of Mathematics, Biological Sciences, Information Sciences and Applied Sciences, whereas Applied Sciences share nouns with the disciplines of Economics,
<caption>**Table 4**</caption>

Top 10 nouns and their association scores for each discipline. Nouns are presented in lower case and base form. Nouns appearing in a single discipline are in bold.

<table><thead><tr><th>Law</th><th>Social Sciences</th><th>Philosophy</th><th>History</th></tr></thead><tbody><tr><td><strong>court</strong></td><td><strong>adolescent</strong></td><td><strong>china</strong></td><td>god</td></tr><tr><td><strong>section</strong></td><td><strong>parent</strong></td><td>people</td><td>jesus</td></tr><tr><td>law</td><td>people</td><td>god</td><td>church</td></tr><tr><td>right</td><td><strong>individual</strong></td><td><strong>united states</strong></td><td><strong>paul</strong></td></tr><tr><td><strong>term</strong></td><td>study</td><td>woman</td><td><strong>christ</strong></td></tr><tr><td>case</td><td>child</td><td><strong>north korea</strong></td><td><strong>israel</strong></td></tr><tr><td>article</td><td>participant</td><td>state</td><td>people</td></tr><tr><td><strong>fact</strong></td><td>woman</td><td>notion</td><td><strong>history</strong></td></tr><tr><td>state</td><td>student</td><td><strong>russia</strong></td><td>book</td></tr><tr><td><strong>question</strong></td><td><strong>relationship</strong></td><td><strong>idea</strong></td><td>woman</td></tr><tr><td><strong>Economics</strong></td><td><strong>Political Sciences</strong></td><td><strong>Religion</strong></td><td><strong>Business</strong></td></tr><tr><td><strong>firm</strong></td><td>woman</td><td>god</td><td>patient</td></tr><tr><td>people</td><td><strong>city</strong></td><td>church</td><td><strong>retailer</strong></td></tr><tr><td>probability</td><td>government</td><td>jesus</td><td><strong>consumer</strong></td></tr><tr><td><strong>income</strong></td><td>right</td><td>people</td><td><strong>customer</strong></td></tr><tr><td><strong>country</strong></td><td>law</td><td>book</td><td>study</td></tr><tr><td><strong>cost</strong></td><td>article</td><td>text</td><td><strong>company</strong></td></tr><tr><td>risk</td><td>state</td><td><strong>psalm</strong></td><td><strong>service</strong></td></tr><tr><td><strong>household</strong></td><td><strong>president</strong></td><td>word</td><td>product</td></tr><tr><td>government</td><td>people</td><td><strong>scripture</strong></td><td>risk</td></tr><tr><td><strong>price</strong></td><td><strong>parliament</strong></td><td><strong>world</strong></td><td>information</td></tr><tr><td><strong>Linguistics</strong></td><td><strong>Information Sciences</strong></td><td><strong>Mathematics</strong></td><td><strong>Health Sciences</strong></td></tr><tr><td>student</td><td>number</td><td>number</td><td>patient</td></tr><tr><td><strong>teacher</strong></td><td>study</td><td>probability</td><td>treatment</td></tr><tr><td><strong>language</strong></td><td>datum</td><td><strong>set</strong></td><td>study</td></tr><tr><td><strong>english</strong></td><td>user</td><td><strong>theorem</strong></td><td>risk</td></tr><tr><td>child</td><td>information</td><td><strong>assumption</strong></td><td><strong>diagnosis</strong></td></tr><tr><td>word</td><td><strong>research</strong></td><td><strong>respect</strong></td><td>development</td></tr><tr><td><strong>learner</strong></td><td>result</td><td><strong>solution</strong></td><td>case</td></tr><tr><td>participant</td><td><strong>author</strong></td><td>value</td><td><strong>surgery</strong></td></tr><tr><td>study</td><td>student</td><td>case</td><td><strong>disease</strong></td></tr><tr><td><strong>use</strong></td><td>time</td><td>uncertainty</td><td>child</td></tr><tr><td><strong>Biological Sciences</strong></td><td><strong>Physical Sciences</strong></td><td><strong>Applied Science</strong></td><td><strong>Arts</strong></td></tr><tr><td>study</td><td>uncertainty</td><td>number</td><td>woman</td></tr><tr><td><strong>effect</strong></td><td>model</td><td><strong>node</strong></td><td>text</td></tr><tr><td><strong>change</strong></td><td>result</td><td>probability</td><td><strong>reader</strong></td></tr><tr><td><strong>soil</strong></td><td><strong>surface</strong></td><td>datum</td><td><strong>music</strong></td></tr><tr><td>difference</td><td>difference</td><td>time</td><td><strong>audience</strong></td></tr><tr><td>presence</td><td><strong>temperature</strong></td><td>user</td><td><strong>story</strong></td></tr><tr><td>patient</td><td>presence</td><td><strong>network</strong></td><td><strong>film</strong></td></tr><tr><td>treatment</td><td><strong>comparison</strong></td><td>value</td><td><strong>way</strong></td></tr><tr><td>result</td><td>value</td><td>model</td><td>people</td></tr><tr><td>development</td><td>time</td><td><strong>order</strong></td><td>notion</td></tr></tbody></table>

Physical Sciences, Mathematics and Information Sciences. This result suggests that the research questions in these disciplines tend to remain within the boundaries of their field and are shared only with closely-related disciplines.

## 6. Discussion

In this paper, we have presented an analysis of the Const-L dataset with the aim to reveal patterns in the expression of uncertainty across disciplines and time. However, the above results should be interpreted in light of several limitations discussed below.

Firstly, the Const-L dataset is not balanced across disciplines and publication years. This is due to the fact that the distribution of articles in the dataset depends on the number of articles proposed by the Constellate platform. This could introduce bias in the results for the less represented disciplines and time periods. Moreover, in our analysis we rely on the existing classification of articles into disciplines that is provided by the platform. However, we have not evaluated the quality of this classification.

Secondly, the initial cleaning and preparation of the full text dataset, including the segmentation into sections and paragraphs, was performed using standard tools. This process might introduce errors or struggle to adapt to the various layouts found in scientific literature over time. Alternative approaches, even simple machine-learning approaches such as linear-CRF [49], could be considered to perform such tasks and improve the quality of the dataset. However, as mentioned in Section 3, annotated datasets dedicated to such tasks remain rare, which prevented us from applying ML approaches in this study.

Thirdly, to identify common objects or concepts related to uncertainty in each discipline, we examined noun groups that occur in sentences expressing uncertainty. However, this approach can only produce an approximate image of the concepts that appear in
<figure><img src="image_11.png" /><figcaption>Fig. 9. Number of nouns shared between each two disciplines in sentences expressing uncertainty.</figcaption></figure>

the discourse of each discipline and that relate to uncertainty. A finer analysis of the structure of the sentences should be undertaken with the objective to identify the entities and relations on which the uncertainty holds, and thus make possible a precise landscape of the research questions that appear in the various disciplines.

While the UnScientify annotation system achieves the best results in terms of overall Accuracy (0.816) during our tests, some annotation errors remain and might influence the final results reported above. The distribution of errors across the disciplines has not been evaluated, as this would require much effort in terms of manual checking and annotation. However, we consider that the margin of error is small enough to allow us to observe the general trends in this large-scale dataset. More precise analyses would be necessary to provide a finer-grained representation of the use of uncertainty in articles.

## 7. Conclusion

In this paper we presented a comprehensive large-scale analysis of scientific uncertainty as expressed in full text research articles across 16 disciplines over more than two decades. We used the newly introduced Const-L dataset of 31,849 articles and the rule-based UnScientify annotation system. Our work provided novel insights into the distribution, evolution, and linguistic contexts of uncertainty in scholarly publications.

Our findings reveal clear disciplinary patterns: Social Sciences and Humanities (SSH) consistently show higher proportions of uncertainty expressions compared to Science, Technology, and Medicine (STM) fields. This variation stems from the diverse epistemic cultures and modes of scientific communication. Additionally, we observed a slight overall decline in expressed uncertainty from 2000 to 2021, suggesting that the ways in which uncertainty is communicated evolve over time. Furthermore, we show that the distribution of uncertainty within articles follows distinct trajectories depending on the discipline, with some disciplines concentrating uncertainties mainly in introductions and discussions, and others distributing it more evenly over text progression.

Our approach benefits from the interpretability and efficiency of the rule-based UnScientify system validated against alternative models. However, limitations such as dataset imbalances, potential preprocessing errors, and the approximate nature of noun-based analyses invite further refinement. Future work should aim to enhance text segmentation quality, achieve finer-grained semantic analyses of uncertainty contexts, and explore interdisciplinary relationships that are revealed by shared objects of uncertainty.

Overall, this study provides new insights into the understanding of scientific uncertainty in scholarly discourse. By mapping how uncertainty is expressed and varies across disciplines and time, it contributes to the study of scientific communication and the epistemology of research publications. Our findings aim to provide a new perspective on how scientific knowledge is constructed and communicated in publications, and how disciplinary practices influence the expression of uncertainty in scientific discourse.
CRediT authorship contribution statement

**Nicolas Gutehrlé:** Writing – original draft, Visualization, Methodology, Investigation, Formal analysis, Data curation, Writing – review & editing. **Panggih Kusuma Ningrum:** Writing – original draft, Visualization, Methodology, Investigation, Data curation, Conceptualization. **Iana Atanassova:** Writing – review & editing, Validation, Supervision, Project administration, Methodology, Funding acquisition, Conceptualization.

Funding

This work is supported by the InSciM project Modeling Uncertainty in Science (2021–2025), funded by the French ANR, grant number ANR-21-CE38-0003-01.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Iana Atanassova reports financial support was provided by French National Research Agency. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

The AURORA-MESS dataset is available on the Zenodo platform [39]. The Const-L dataset is available through JSTOR Text analysis support. The source code of UnScientify is available on Github<fnref n="6" />.

References

[1] K. Popper, Karl Popper: The Logic of Scientific Discovery, first ed., Routledge, London and New York, 2002.

[2] G. Ruhrmann, L. Guenther, S.H. Kessler, J. Milde, Frames of scientific evidence: How journalists represent the (un)certainty of molecular medicine in science television programs, Public Underst. Sci. 24 (6) (2013) 681–696, http://dx.doi.org/10.1177/0963662513510643, Publisher: SAGE Publications Ltd.

[3] T.A. Schwandt, Credible evidence of effectiveness: Necessary but not sufficient, in: Credible and Actionable Evidence, second ed., SAGE Publications, Inc., United States of America, 2015.

[4] G. Klir, T.A. Folger, Fuzzy Sets, Uncertainty and Information, Prentice Hall, United States of America, Englewood Cliffs, N.J., 1988.

[5] P.K. Ningrum, N. Gutehrlé, I. Atanassova, Etudier l'incertitude dans les articles scientifiques : mise en perspective d'une méthode linguistique, Rev. Des Nouv. Technol. de L'Information (2025) 111-122, Extraction et Gestion des Connaissances, RNTI-E-41.

[6] P.K. Ningrum, P. Mayr, N. Smirnova, I. Atanassova, Annotating scientific uncertainty: A comprehensive model using linguistic patterns and comparison with existing approaches, J. Inf. 19 (2) (2025) 101661, http://dx.doi.org/10.1016/j.joi.2025.101661.

[7] W. Walker, P. Harremoës, J. Rotmans, J. van der Sluijs, M. van Asselt, P. Janssen, M. Krayer von Krauss, Defining uncertainty: A conceptual basis for uncertainty management in model-based decision support, Integr. Assess. 4 (1) (2003) 5–17, http://dx.doi.org/10.1076/iaij.4.1.5.16466, URL: http://www.tandfonline.com/doi/abs/10.1076/iaij.4.1.5.16466.

[8] J.C. Refsgaard, J.P. van der Sluijs, A.L. Hojberg, P.A. Vanrolleghem, Uncertainty in the environmental modelling process - a framework and guidance, Env. Model. Softw. 22 (11) (2007) 1543–1556, http://dx.doi.org/10.1016/j.envsoft.2007.02.004, URL: https://doi.org/10.1016/j.envsoft.2007.02.004.

[9] J. Ascough, H. Maier, J. Ravalico, M. Strudley, Future research challenges for incorporation of uncertainty in environmental and ecological decision-making, Ecol. Model. 219 (3) (2008) 383–399, http://dx.doi.org/10.1016/j.ecolmodel.2008.07.015, URL: https://www.sciencedirect.com/science/article/pii/S0304380008003554.

[10] M.A.K. Halliday, C.M.I.M. Matthiessen, Halliday's Introduction to Functional Grammar, fourth edition, Routledge, Abingdon, United Kingdom, 2014, p. 790.

[11] G. Moncecchi, J.-L. Minel, D. Wonsever, Improving speculative language detection using linguistic knowledge, in: Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics, Association for Computational Linguistics, 2012, pp. 37–46, URL: https://www.aclweb.org/anthology/W12-3805.pdf.

[12] N.P.C. Diaz, M.J.M. López, Negation and Speculation Detection, Vol. 13, John Benjamins Publishing Company, 2019, URL: https://doi.org/10.1075/nlp.13.

[13] C. Chen, M. Song, G.E. Heo, A scalable and adaptive method for finding semantically equivalent cue words of uncertainty, J. Inf. 12 (1) (2018) 158–180, http://dx.doi.org/10.1016/j.joi.2017.12.004, URL: https://linkinghub.elsevier.com/retrieve/pii/S1751157717301712.

[14] I. Atanassova, F.-C. Rey, M. Bertin, Studying uncertainty in science: a distributional analysis through the imrad structure, in: 7th International Workshop on Mining Scientific Publications (WOSP 2018) At 11th Edition of the Language Resources and Evaluation Conference (LREC 2018), Miyazaki, Japan, 2018, URL: https://hal.archives-ouvertes.fr/hal-01940294.

[15] F.-C. Rey, M. Bertin, I. Atanassova, Une étude de l'incertitude dans les textes scientifiques : vers la construction d'une ontologie, in: Terminology & Ontology: Theories and Applications (TOTh 2018), Chambéry, France, 2018, pp. 229–242, URL: http://ontologia.fr/TOTh/Conference/TOTh2018/TOTh_2018.pdf.

[16] V. Vincze, G. Szarvas, R. Farkas, G. Móra, J. Csirik, The BioScope corpus: Biomedical texts annotated for uncertainty, negation and their scopes, in: BMC Bioinformatics, 9, 2008, http://dx.doi.org/10.1186/1471-2105-9-S11-S9, Issue: SUPPL. 11 ISSN: 14712105.

[17] J.H.A. Guillaume, C. Helgeson, S. Elsawah, A.J. Jakeman, M. Kummu, Toward best practice framing of uncertainty in scientific publications: A review of water resources research abstracts, AGU Publ. Water Resour. Res. (2017) http://dx.doi.org/10.1002/2017WR020609.

[18] B. Medlock, T. Briscoe, Weakly supervised learning for hedge classification in scientific literature, in: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Association for Computational Linguistics, Prague, Czech Republic, 2007, URL: https://aclanthology.org/P07-1125.

<footnote n="6">6 https://github.com/Project-InSciM/Scientific-Uncertainty-Extractor</footnote>
[19] I. Riccioni, R. Bongelli, A. Zuczkowski, Self-mention and uncertain communication in the British medical journal (1840-2007): The decrease of subjectivity uncertainty markers, Open Linguist. 7 (1) (2021) 739-759, https://dx.doi.org/10.1515/OPLI-2020-0179/MACHINEREADABLECITATION/RIS, URL: https://www.degruyter.com/document/doi/10.1515/opli-2020-0179/html?lang=en Publisher: Walter de Gruyter GmbH.

[20] P. Mur-Dueñas, There may be differences: Analysing the use of hedges in english and spanish research articles, Lingua 260 (2021) 103131, http://dx.doi.org/10.1016/j.lingua.2021.103131, URL: https://www.sciencedirect.com/science/article/pii/S0024384121001030.

[21] M. Yao, Y. Wei, H. Wang, Promoting research by reducing uncertainty in academic writing: a large-scale diachronic case study on hedging in science research articles across 25 years, Scientometrics 128 (8) (2023) 4541-4558, http://dx.doi.org/10.1007/s11192-023-04759-6.

[22] P.K. Ningrum, I. Atanassova, Scientific uncertainty: an annotation framework and corpus study in different disciplines, in: 19th International Conference of the International Society for Scientometrics and Informetrics, ISSI 2023, Bloomington, Indiana, US, 2023.

[23] W. Chapman, J. Dowling, D. Chu, ConText: An algorithm for identifying contextual features from clinical text, in: K.B. Cohen, D. Demner-Fushman, C. Friedman, L. Hirschman, J. Pestian (Eds.), Biological, Translational, and Clinical Language Processing, Association for Computational Linguistics, Prague, Czech Republic, 2007, pp. 81–88, URL: https://aclanthology.org/W07-1011.

[24] I. Atanassova, F.-C. Rey, M. Bertin, Studying uncertainty in science: a distributional analysis through the IMRAD structure, 2018, URL: http://rgai.inf.u-szeged.hu/conll2010st/tasks.html.

[25] F.-C. Rey, Ontologie De L'incertitude Et Annotation Sémantique D'un Corpus Autour Du Changement Climatique (Ph.D. thesis), Université Bourgogne Franche-Comté, 2022, URL: https://hal.science/tel-03780719.

[26] P. Omero, M. Valotto, R. Bellana, R. Bongelli, I. Riccioni, A. Zuczkowski, C. Tasso, Writer's uncertainty identification in scientific biomedical articles: a tool for automatic if-clause tagging, Lang. Resour. Eval. 54 (2020) 1-21, http://dx.doi.org/10.1007/s10579-020-09491-8.

[27] M. Müller, S. Bartsch, J.O. Zinn, Communicating the unknown: An interdisciplinary annotation study of uncertainty in the coronavirus pandemic, Int. J. Corpus Linguist. 26 (4) (2021) 498-531, http://dx.doi.org/10.1075/ijcl.21096.mul, URL: https://www.jbe-platform.com/content/journals/10.1075/ijcl.21096.mulPublisher: John Benjamins.

[28] M. Müller, Corpus approaches to analysing uncertainty and ignorance in academic discourse, Fachsprache 45 (1-2) (2023) 28-47, http://dx.doi.org/10.24989/fs.v45i1-2.2232, URL: https://ejournals.facultas.at/index.php/fachsprache/article/view/2232.

[29] M. Rei, T. Briscoe, Combining manual rules and supervised learning for hedge cue and scope detection, in: R. Farkas, V. Vincze, G. Szarvas, G. Móra, J. Csirik (Eds.), Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, Association for Computational Linguistics, Uppsala, Sweden, 2010, pp. 56–63, URL: https://aclanthology.org/W10-3008.

[30] L. Mamani Sánchez, B. Li, C. Vogel, Exploiting CCG structures with tree kernels for speculation detection, in: R. Farkas, V. Vincze, G. Szarvas, G. Móra, J. Csirik (Eds.), Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, Association for Computational Linguistics, Uppsala, Sweden, 2010, pp. 126–131, URL: https://aclanthology.org/W10-3018.

[31] S. Zhang, H. Zhao, G. Zhou, B.-L. Lu, Hedge detection and scope finding by sequence labeling with procedural feature selection, in: R. Farkas, V. Vincze, G. Szarvas, G. Móra, J. Csirik (Eds.), Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, Association for Computational Linguistics, Uppsala, Sweden, 2010, pp. 92–99, URL: https://aclanthology.org/W10-3013.

[32] R. Morante, W. Daelemans, Learning the scope of hedge cues in biomedical texts, in: K.B. Cohen, D. Demner-Fushman, S. Ananiadou, J. Pestian, J. Tsujii, B. Webber (Eds.), Proceedings of the BioNLP 2009 Workshop, Association for Computational Linguistics, Boulder, Colorado, 2009, pp. 28–36, URL: https://aclanthology.org/W09-1304.

[33] D. Clausen, Hedgehunter: A system for hedge detection and uncertainty classification, in: R. Farkas, V. Vincze, G. Szarvas, G. Móra, J. Csirik (Eds.), Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, Association for Computational Linguistics, Uppsala, Sweden, 2010, pp. 120–125, URL: https://aclanthology.org/W10-3017.

[34] E.R. Fernandes, C.E.M. Crestana, R.L. Milidiú, Hedge detection using the RelHunter approach, in: Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, in: CoNLL '10: Shared Task, Association for Computational Linguistics, USA, 2010, pp. 64–69.

[35] X. Li, J. Shen, X. Gao, X. Wang, Exploiting rich features for detecting hedges and their scope, in: R. Farkas, V. Vincze, G. Szarvas, G. Móra, J. Csirik (Eds.), Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, Association for Computational Linguistics, Uppsala, Sweden, 2010, pp. 78–83, URL: https://aclanthology.org/W10-3011.

[36] B. Tang, X. Wang, X. Wang, B. Yuan, S. Fan, A cascade method for detecting hedges and their scope in natural language text, in: R. Farkas, V. Vincze, G. Szarvas, G. Móra, J. Csirik (Eds.), Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, Association for Computational Linguistics, Uppsala, Sweden, 2010, pp. 13–17, URL: https://aclanthology.org/W10-3002.

[37] G. Szarvas, V. Vincze, R. Farkas, J. Csirik, The BioScope corpus: annotation for negation, uncertainty and their scope in biomedical texts, BioNLP (2008) 38–45, URL: www.inf.u-szeged.hu/rgai/bioscope.

[38] A. Khandelwal, B.K. Britto, Multitask learning of negation and speculation using transformers, in: E. Holderness, A. Jimeno Yepes, A. Lavelli, A.-L. Minard, J. Pustejovsky, F. Rinaldi (Eds.), Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis, Association for Computational Linguistics, Online, 2020, pp. 79–87, http://dx.doi.org/10.18653/v1/2020.louhi-1.9, URL: https://aclanthology.org/2020.louhi-1.9.

[39] P.K. Ningrum, I. Atanassova, AURORA-MESS: Annotated uncertainty and reference in open research articles for multidisciplinary and empirical social science, 2025.

[40] M. Bertin, I. Atanassova, C.R. Sugimoto, V. Lariviere, The linguistic patterns and rhetorical structure of citation context: an approach using n-grams, Scientometrics 3 (109) (2016) 1417-1434, http://dx.doi.org/10.1007/s11192-016-2134-8.

[41] P.K. Ningrum, P. Mayr, I. Atanassova, Unscientific: Detecting scientific uncertainty in scholarly full text, in: Joint Workshop of the 4th Extraction and Evaluation of Knowledge Entities from Scientific Documents and the 3rd AI + Informetrics (EEKE- AII2023), Santa Fe, New Mexico, USA and Online, 2023, URL: https://ceur-ws.org/Vol-3451/paper9.pdf.

[42] P.K. Ningrum, Identifier Et Annoter L'incertitude Scientifique Dans Les Textes Académiques : Méthodes, Cadres Etapplications (Ph.D. thesis), 2025, URL: Thèse de doctorat dirigée par Atanassova, Iana Sciences du Langage, mention Traitement Automatique des Langues Besançon, Université Marie et Louis Pasteur 2025.

[43] P.K. Ningrum, Identifying and Annotating Scientific Uncertainty in Scholarly Texts: Methods, Frameworks, and Applications (Ph.D. thesis), Université Marie et Louis Pasteur, 2025, URL: https://theses.fr/s306743.

[44] K. Halder, A. Akbik, J. Krapac, R. Vollgraf, Task-aware representation of sentences for generic text classification, in: D. Scott, N. Bel, C. Zong (Eds.), Proceedings of the 28th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Barcelona, Spain (Online), 2020, pp. 3202-3213, http://dx.doi.org/10.18653/v1/2020.coling-main.285, URL: https://aclanthology.org/2020.coling-main.285/.

[45] L. Tunstall, N. Reimers, U.E.S. Jo, L. Bates, D. Korat, M. Wasserblat, O. Pereg, Efficient few-shot learning without prompts, 2022, http://dx.doi.org/10.48550/ARXIV.2209.11055, URL: https://arxiv.org/abs/2209.11055.

[46] P.K. Ningrum, I. Atanassova, N. Gutehrlé, MUSIC-OpRA: Multidimensional uncertainty in scientific interdisciplinary corpora for open research article, 2025, http://dx.doi.org/10.5281/zenodo.15173356.

[47] M. Grootendorst, BERTopic: Neural topic modeling with a class-based TF-IDF procedure, 2022, arXiv preprint arXiv:2203.05794.

[48] I. Atanassova, P. Kusuma Ningrum, N. Gutehrlé, F. Larreau, C. Malaterre, Is there life on mars ? Studying the context of uncertainty in astrobiology, in: 20th International Society of Scientometrics and Informetrics Conference 2025, Yerevan, Armenia, 2025, p. 13.

[49] J. Lafferty, A. McCallum, F.C. Pereira, Conditional random fields: Probabilistic models for segmenting and labeling sequence data, 2001.
<figure><img src="image_12.png" /></figure>

**Dr. Nicolas Gutehrlé** is a post-doctoral researcher in the ANR InSciM "Modelling Uncertainty in Science" project at the CRIT laboratory of the Université Marie et Louis Pasteur (Besançon, France). He received his Ph.D. degree in language science, specializing in Natural Language Processing, from the Université Bourgogne Franche-Comté (France) in 2024. His research focus on the application and the conception of innovative information extraction methods to assist in the study of massive datasets in the fields of Humanities and Social Sciences. Since September 2025, he is Associate Professor at ERIC laboratory (Université Lumière Lyon 2, France).

<figure><img src="image_13.png" /></figure>

**Dr. Panggih Kusuma Ningrum** is a post-doctoral researcher in the ANR InSciM "Modelling Uncertainty in Science" project at the CRIT laboratory of the Université Marie et Louis Pasteur (Besançon, France). She received her Ph.D. degree in language science, specializing in Natural Language Processing, from the Université Bourgogne Franche-Comté (France) in 2025. Her research specializes in linguistic and computational discourse analysis. She develops NLP models and prototypes applied for large-scale scientific full-text processing, empirical social science and legal studies.

<figure><img src="image_14.png" /></figure>

**Pr. Iana Atanassova** is Full Professor and head of the CRIT laboratory at the Université Marie et Louis Pasteur (Besançon, France). She is also a junior member of the Institut Universitaire de France (IUF). Her research focuses on the processing of the full text of research papers with applications in bibliometrics and epistemology. She develops both linguistically-motivated approaches and deep-learning methods.