ITEXT-BIO: Intelligent Term EXTraction for BIOmedical Analysis

Rodrique Kafando, Rémy Decoupes, Sarah Valentin, Lucile Sautot, Maguelonne Teisseire, Mathieu Roche

<figure><img src="image_1.png" /></figure>

To cite this version:

Rodrique Kafando, Rémy Decoupes, Sarah Valentin, Lucile Sautot, Maguelonne Teisseire, et al. ITEXT-BIO: Intelligent Term EXTraction for BIOmedical Analysis. Health Information Science and Systems, 2021, 9 (1), pp.29. 10.1007/s13755-021-00156-6. hal-03283040v2

HAL Id: hal-03283040

https://hal.inrae.fr/hal-03283040v2

Submitted on 25 Aug 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# ITEXT-BIO: Intelligent Term Extraction for BIOmedical analysis

Rodrique Kafando¹,<fnref n="2" />, Rémy Decoupes¹,<fnref n="2" />, Sarah Valentin<fnref n="2" />,³,⁴, Lucile Sautot<fnref n="2" />,⁵, Maguelonne Teisseire¹,<fnref n="2" />* and Mathieu Roche<fnref n="2" />,³

## Abstract

Here, we introduce ITEXT-BIO, an intelligent process for biomedical domain terminology extraction from textual documents and subsequent analysis. The proposed methodology consists of two complementary approaches, including free and driven term extraction. The first is based on term extraction with statistical measures, while the second considers morphosyntactic variation rules to extract term variants from the corpus. The combination of two term extraction and analysis strategies is the keystone of ITEXT-BIO. These include combined intra-corpus strategies that enable term extraction and analysis either from a single corpus (intra), or from corpora (inter). We assessed the two approaches, the corpus or corpora to be analysed and the type of statistical measures used. Our experimental findings revealed that the proposed methodology could be used: (1) to efficiently extract representative, discriminant and new terms from a given corpus or corpora, and (2) to provide quantitative and qualitative analyses on these terms regarding the study domain.

**Keywords:** Biomedical terminology, Terminology extraction, Intelligent analysis

## Introduction

The usefulness of terminology extraction from corpora is clearly acknowledged as it has generated a great deal of research and discussion. This well-established process is used in natural language processing and has led to the development of several tailored tools such as TBXTools [31], TermSuite [9], BioTex [22], etc.

Based on [22], our proposal deals with domain-based terminology extraction from heterogeneous corpora, and how to efficiently generate a quantitative and qualitative analysis. To this end, we propose a generic methodology hinged on a combination of extraction and analysis strategies. Term extraction strategies are based on combinations of linguistic, statistical measures, and corpus segmentation approaches, while analysis strategies are based on combinations of extracted terms.

Based on the combined strategies, ITEXT-BIO aims to extract: (1) representative terms, (2) discriminant or

relevant terms, and (3) new relevant terms from a corpus or corpora. These strategies are specifically useful for dedicated tasks, such as corpus analysis, specific domain monitoring (e.g. epidemiology) or scientific research monitoring.

This paper is organized as follows. In Section Related work, we briefly present the state-of-the-art related to terminology extraction. Section Dataset description details the dataset dedicated to scientific papers. Sections Methodology and Experiments respectively provide an overview of our proposal and the experiments. In Section Case study: epidemic intelligence, we illustrate the genericity of the proposal by presenting a case study of an implementation of the combined strategies for epidemiological intelligence analysis. We conclude in Section Conclusion by presenting some perspectives for future studies.

## Related work

Domain terminology extraction is a major focus of interest and discussion in natural language processing (NLP) research. It has prompted several proposals of

<footnote>*Correspondence: Maguelonne.Teisseire@inrae.fr</footnote>

<footnote n="2">² TETIS, Univ Montpellier, AgroParisTech, CIRAD, CNRS, INRAE, Montpellier, France</footnote>

<footnote>Full list of author information is available at the end of the article</footnote>
methodologies [20, 32, 34, 36] geared towards effective extraction of terms within a given corpus. Also known as automatic term extraction (ATE), this task is considered in various NLP applications, such as in information retrieval [2, 4, 11, 37], topic modeling [15, 42], domain-based monitoring [1, 19, 27], keyword extraction [7] and summarization [2], ontology acquisition, thesaurus construction, etc.

According to [23], term extraction techniques can be categorized under four approaches: linguistic, statistical, machine learning and hybrid.

Overall, linguistic approaches take morphosyntactic part-of-speech (POS) rules into account to describe terms with common structures [5]. Statistical approaches use statistical measures such as term frequency [35, 43], or term co-occurrence between words and phrases like Chi-square [26]. Machine learning approaches use statistical measures and are mainly jointly focused on term extraction [7, 8, 12], classification [41] and summarization [2]. They combine linguistic and statistic approaches to extract terms from textual data in order to build machine learning models. In [7], the authors highlighted that most of these tasks are tackled with unsupervised learning algorithms. Hybrid approaches include, for instance, C_Value [34], C/NC_Value [13] methods, which combine statistical measures and linguistic based rules to extract multi-word and nested terms. In [6, 30], the authors combine rule-based methods and dictionaries to extract terms from Spanish biomedical texts and specialised Arabic texts respectively.

Studies such as [18, 21] related to these latter approaches have revealed the effectiveness and high performance of hybrid term extraction approaches.

The proposed methodologies apply to several domains. In [22], the authors proposed BioTex, a linguistic and statistical measure-based tool to extract terms related to the biomedical domain. The same approach was used in [1] to detect terms or signals for infectious disease monitoring on the web. In [28], a hybrid methodology was proposed to extract terminology for electronic health records. This hybrid approach was also adapted by [44] to extract concepts related to Chinese culture.

The overall related studies have focused on techniques and methods for term extraction mainly from corpora. Based on existing methodologies, we oriented our study to develop an efficient approach for term extraction from heterogeneous corpora, along with a set of combined strategies to analyze these terms in the biomedical domain. Our methodology combines and tailors linguistic and statistic criteria associated with structural information in texts in order to highlight relevant terms therein. The presented strategies also aim to overcome the time-consuming issues related to machine learning

<table><caption>**Table 1 Statistics related to the COVID19-MOOD-data dataset**</caption><thead><tr><th></th><th>NB<sub>d</sub>(C)</th><th>NB<sub>M</sub>(d)</th><th>std(c)</th></tr></thead><tbody><tr><td colspan="4"><em>Papers1</em></td></tr><tr><td>Papers1-title</td><td>9315</td><td>15</td><td>± 8</td></tr><tr><td>Papers1-abstract</td><td>9315</td><td>180</td><td>± 94</td></tr><tr><td>Papers1-content</td><td>9315</td><td>4639</td><td>± 359</td></tr><tr><td colspan="4"><em>Papers2</em></td></tr><tr><td>Papers2-title</td><td>32322</td><td>13</td><td>± 10</td></tr><tr><td>Papers2-abstract</td><td>32322</td><td>168</td><td>± 88</td></tr><tr><td>Papers2-content</td><td>32322</td><td>4913</td><td>± 720</td></tr></tbody></table>

methods which require manually annotated or partially annotated data.

## Dataset description

Our study focused on the COVID-19 Open Research Dataset<fnref n="1" /> [40] which contains scientific papers on COVID-19 and related historical coronavirus research. Throughout this study, we refer to the dataset as COVID19-MOOD-data.

The COVID19-MOOD-data dataset is divided into two main corpora, respectively named Papers1 and Papers2. Papers1 contains the commercial use subset (includes PubMed Central content), while Papers2 contains the commercial use subset (includes PubMed Central content), the non-commercial use subset (includes PubMed Central content) and the custom license subset.

Three data pre-processing operations are performed per corpus (Papers1, Papers2) in order to create three corpora according to the title, abstract and content:

- Title represents the corpus that contains only paper titles;

- Abstract represents the corpus that contains only paper abstracts;

- Content represents the corpus that contains only paper contents.

We named them PapersX-title, PapersX-abstract and PapersX-content, respectively. See Table 1 for further details and Table 2 for the acronym definitions.

## Methodology

Here we outline two complementary term extraction and analysis approaches: the free term extraction approach and the driven term extraction approach. The first one is based on a combination of the type of corpus and the

<footnote n="1">¹ https://www.semanticscholar.org/cord19/download</footnote>
<table><caption>**Table 2 Table legend**</caption><thead><tr><td>Abbreviations</td><td>Description</td></tr></thead><tbody><tr><td>NB<sub>d</sub>(C)</td><td>Number of documents in the corpus</td></tr><tr><td>NB<sub>M</sub>(d)</td><td>Average number of words of a document in the corpus</td></tr><tr><td>std(c)</td><td>Corpus standard deviation</td></tr><tr><td>NN</td><td>Noun</td></tr><tr><td>NNNN</td><td>Matches singular and plural noun terms</td></tr><tr><td>JJ</td><td>Adjective</td></tr><tr><td>NP</td><td>Proper noun</td></tr></tbody></table>

statistical measures, while the second is based on a combination of the type of corpus and the morphosyntactic variation rules.

## The free term extraction approach

The free term extraction approach seeks to ensure that users will be able to extract significant terms related to a specific domain from a given corpus. As we mentioned in Section Related work, existing tools have been proposed for term and concept extraction. We opted for the BioTex tool to support the free term extraction mode for several reasons:

- BioTex was initially built for medical domain term extraction.

- BioTex uses hybrid measures (linguistic and several statistical measures) for the term extraction process.

- Most existing tools (e.g. *Maui-indexer*<fnref n="2" />, *Topia Termextract*<fnref n="3" />, *KEA*<fnref n="4" />, etc.) are designed for keyword extraction within single documents, and they only function for English language documents, while BioTex is tailored for terminology extraction and supports sets of documents (corpora) and multi-language use.

Three essential parameters related to the BioTex tool are defined below:

- a corpus: this is the data source from which terms are extracted;

- a statistical measure: as mentioned above, the BioTex processing approach is based on linguistic and statistical measures. The linguistic parameter is defined by default, but the user must define the statistical parameter, as several exist, in order to run the term extraction process;

<figure><img src="image_4.png" /><figcaption>**Fig. 1** The Free and Driven process for term extraction using BioTex and FASTR</figcaption></figure>

- the number of words to be extracted per concept: so called n-grams, this concerns the length of the extracted terms and ranges from 1 to 4_g for BioTex.

In addition to these parameters, there is the number of linguistic patterns (like NN, JJ, NP, NP, NN, NP, NP, etc.) that can be associated, but this is preset at 20 by default in BioTex. BioTex also includes patterns for verb terms, such as: NN VBD NN, NN, NP NN VBD NN, NP, etc. Figure 1 outlines the overall three-step process for free term extraction.

At the end of the BioTex process, extracted terms are classified in two sets: TermSet, which only contains single word terms (SWT), and MultiTermSet, which contains multi-word terms (MWT). By using the Driven Extraction process (with FASTR), we can capture the entire term for a given incomplete one obtained during the first step (Free Extraction). The Driven Extraction process step uses incomplete terms to capture the entire terms in the document. For example, if “higher risk acute” or “higher risk area” terms are extracted in the Free Extraction process step, an entire term which could be “higher risk acute care area” will be obtained during the Driven Extraction process.

## The driven term extraction approach

This extraction approach seeks to ensure that the terms extracted using BioTex could be used to improve the domain terminology. From a given term, the process aims to extract some variations of this term that exist in the corpus.

The overall processing under this approach is handled with FASTR [17]. FASTR is a rule-based linguistic tool that generates morphosyntactic variants of terms.

<footnote n="2">² https://code.google.com/p/maui-indexer/.</footnote>

<footnote n="3">³ https://pypi.python.org/pypi/topia.termextract/1.1.0.</footnote>

<footnote n="4">⁴ http://www.nzdl.org/Kea/.</footnote>
<figure><img src="image_5.png" /></figure>

We respectively note NN, NNS, NNP, NNPS for noun patterns, VB, VBD, VBG, VBN, VBP, VBZ for verbs, RB, RBR, RBS for adverbs and finally JJ, JJR, JJS for adjectives. It enables extraction of variants of a given term in full-text documents. For a given term, FASTR helps extract nearby or long terms that contain the initial term. Figure 1 illustrates the two steps (4 and 5) of the driven term extraction approach. For a given term, FASTR helps extract nearby or long terms that contain the initial one.

The driven process has the advantage of extracting relevant new terms that BioTex cannot extract from the corpus.

## Proposed combination for term extraction

Based on the elements given in Sects. The free term extraction approach and The driven term extraction approach, we propose a workflow in Fig. 2 for term extraction and analysis dedicated to scientific papers. We outline this workflow according to the type of corpus, measure, and approach:

- *The type of corpus:* as described in the data section, for a given paper, we considered three parts to build the corresponding corpora, i.e. the Title (T), Abstract (A) and Content (C);

- *The measures:* BioTex integrates several statistical measures, each of which uses a specific strategy to compute the term score. In this case, we selected the two measures C_Value and F-TFIDF-C_M. C_Value indicates the importance of terms that appear most frequently in a document, based on the idea that the frequency of appearance of a term in the document reflects its importance in the document. Moreover, based on frequency criteria, C_Value favors multi-word term extraction by taking into account nested

terms (e.g. virus) in multi-word terms (e.g. influenza virus) [13]. F-TFIDF-C_M represents the harmonic mean of the two C_Value and TF-IDF values, which ranks terms by weight according to their relevance in the document while taking the whole corpus into account [24]. C_Value and F-TFIDF-C_M are complementary, as the first favors relevant MWT extraction while the second gives weight to discriminant terms. For each measure, the aim is to organise the extracted terms in to five sets. (1) Terms corresponding to the Title corpus Set(T), (2) terms corresponding to the Abstract corpus Set(A), (3) terms corresponding to the Content corpus Set(C), (4) terms that intersect within the Title and the Abstract corpus Set(TA), and (5) terms that intersect within the Title and the Content corpus Set(TC).

- *The approach:* terms could be extracted using both a given corpus and a specific statistical measure in a free extraction approach. Moreover, for the driven process, term variations are extracted by using both a given corpus and specific set of terms. The set of terms could be defined from the output of the previous approach.

## Experiments

To set the parameters, throughout our study we used C_Value and F-TFIDF-C_M as statistical measures, 50 different patterns or term extraction rules, and a number of words ranging from 1 to 4-g (n = 1, 2, 3, 4). These parameters are applied for corpora described in section 3. The choice of C_Value and F-TFIDF-C_M is based on the findings of previous studies [13, 24] which showed that both allow efficient SWT and MWT extraction.

Before applying BioTex, some specific pre-processes were applied for the Papers1-content and Papers2-content corpora due to their size. Papers1-content was divided into 09 sub-corpora (8 corpora of 1000 documents each and 1 corpus of 1315 documents) and Papers2-content into 32 sub-corpora (31 corpora of 1000 documents each, and 1 corpus of 1332 documents). Each corpus was partitioned into smaller units to enhance scalability. The results obtained from the smaller units were then composed by computing the average ranked values. The final rank for a given term was thus equal to the average of its ranked values in all sub-corpora in which it was present. The final result gave a set of terms, listed in ascending order according to the ranking values.

Table 3 shows an example of the MWT set obtained using BioTex. The *Terms* column contains the extracted terms, the *in_uml* column indicates if the corresponding term is available in the Unified Medical Language System (UMLS) Metathesaurus [3] or not, and *rank* shows the significance of the term based on statistical measures in
<table><caption>**Table 3 Example of BioTex output**</caption><thead><tr><td>Terms</td><td>in_umls</td><td>Rank</td></tr></thead><tbody><tr><td>Public health</td><td>1</td><td>1602.3971</td></tr><tr><td>Respiratory syndrome</td><td>0</td><td>1481.9399</td></tr><tr><td>Infectious disease</td><td>1</td><td>1198.2317</td></tr><tr><td>Virus infection</td><td>1</td><td>1126.9083</td></tr><tr><td>Influenza virus</td><td>1</td><td>1023.8858</td></tr><tr><td>Immune response</td><td>1</td><td>1008.0362</td></tr></tbody></table>

the whole list of terms for a given corpus. In our study, we used the UMLS Metathesaurus as reference for the extracted terms as our study is linked to a biomedical terminology analysis. This comparison aimed to separate new terminologies or terminologies that were not yet listed in the Metathesaurus.

## The free term extraction approach

We used BioTex, as outlined in Section *The free term extraction approach*, to extract terms from corpora in free mode. Several analyses are performed below on the obtained results. To this end, we conducted the experiments to address three main questions: (1) for each corpus, what are the most representative terms or domain concepts (terms that summarize the main content of the corpus) per statistical measure? (2) for each corpus, what are the most representative concepts for both measures? and (3) what are the discriminant and common concepts of the overall corpus?

For each case, we determined if the extracted terms exist or not in the UMLS Metathesaurus.

## Corpus representative terms

In this section, we illustrate how representative terms can be extracted from different datasets. Based on the BioTex ranking measures, a term is more important than another one in a given corpus if it has a higher ranking than the other term.

Figure 3 shows representative terms for the Title, Abstract and Content corpora with the corresponding statistical measures (see Tables 7 and 8 for more details).

This figure highlights which terms are important in each part of the Papers. Note that the extracted terms are different for each measure and sub-corpus, but some of them are similar for both. For example, terms like *public health*, *immune responses* are extracted using both measures from the Abstract corpus.

In order to quantitatively display the number of representative intersecting terms from different corpora, we show common terms between Title vs Abstract, and Title vs Content corpora for the Papers2 corpus in Fig. 4. For both measures, Title terms are more representative in the Abstract than in the Content of Papers, i.e. 57% and 27% compared to 28% and 5%, respectively, for Title vs Abstract and Title vs Content. However, we noted that terms extracted with C_Value generated more common terms than those extracted with F-TFIDF-C_M. The common terms represent terms extracted at once in the Title, Abstract and Content corpus for each measure.

<figure><img src="image_6.png" /><figcaption>**Fig. 3 Representative terms from Papers1**</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Table 4 TOP@20 terms extracted from Paper1-content using C_Value and F-TFIDF-C_M - SWTs vs MWTs</figcaption></figure>

<table><thead><tr><th colspan="6">C_Value Measure</th></tr></thead><tbody><tr><td colspan="6">SWTs</td></tr><tr><td rowspan="4">TOP 20</td><td>Cells</td><td>Virus</td><td>Infection</td><td>Protein</td><td>Study</td></tr><tr><td>Data</td><td>figure</td><td>al</td><td>Patients</td><td>Expression</td></tr><tr><td>rna</td><td>Analysis</td><td>Result</td><td>Disease</td><td>p</td></tr><tr><td>Mice</td><td>c</td><td>samples</td><td>Influenza</td><td>Number</td></tr><tr><td rowspan="4">TOP 20</td><td>MWTs</td><td></td><td></td><td></td><td></td></tr><tr><td>t cells</td><td>Public health</td><td>Amino acid</td><td>Immune response</td><td>Gene expression</td></tr><tr><td>Viral replication</td><td>infected cells</td><td>Cell lines</td><td>Viral infection</td><td>Virus infection</td></tr><tr><td>mg ml</td><td>Infectious diseases</td><td>Present study</td><td>Respiratory tract</td><td>Epithelial cells</td></tr><tr><td colspan="2">Previous studies</td><td>Room temperature</td><td>Cell culture</td><td>Additional file</td><td>Viral infection</td></tr><tr><td colspan="6">F-TFIDF-C_M Measure</td></tr><tr><td colspan="6">SWTs</td></tr><tr><td rowspan="5">TOP 20</td><td>Mice</td><td>Patients</td><td>Influenza</td><td>Proteins</td><td>Health</td></tr><tr><td>dna</td><td>Vaccine</td><td>Transmission</td><td>Research</td><td>Model</td></tr><tr><td>Children</td><td>Outbreak</td><td>Vaccination</td><td>e</td><td>China</td></tr><tr><td>Peptide</td><td>Fusion</td><td>Network</td><td>Percent</td><td>mers-cov</td></tr><tr><td>MWTs</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="4">TOP 20</td><td>Additional file</td><td>Infectious disease</td><td>nk cells</td><td>Health care</td><td>Endothelial cells</td></tr><tr><td>Frequency domain</td><td>Ebola virus</td><td>Influenza infection</td><td>Real-time rt-pcr</td><td>Incubation period</td></tr><tr><td>Health emergency</td><td>Index patient</td><td>Membrane rafts</td><td>pcr products</td><td>2c atpase</td></tr><tr><td>b cell</td><td>Close contact</td><td>Final dataset</td><td>3d8 scfv</td><td>pol ii</td></tr></tbody></table>

In bold terms not in the UMLS thesaurus

As indicated, extracted terms were compared with the UMLS Metathesaurus. Table 4 shows the TOP@20 terms extracted for the Papers1-content corpus using C_Value

and F-TFIDF-C_M measures. Bold terms are not in the UMLS Metathesaurus.

According to these TOP@20 terms, we can see that:
<figure><img src="image_8.png" /><figcaption>**Fig. 5** C_Value vs F-TFIDF-C_M SWTs</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Fig. 6** C_Value vs F-TFIDF-C_M MWTs</figcaption></figure>

- the majority of the SWTs are in the UMLS Metathesaurus for both statistical measures (C_Value or F-TFIDF-C_M);

- for MWTs, several terms are not in the UMLS Metathesaurus. These terms can be categorized as:

- *UMLS sub-terms* these are terms that do not exactly match to those present in the UMLS Metathesaurus but could be part of them. For example, *health emergency* is part of terms like *Emergency Health Services* in the UMLS Metathesaurus;

- *New terms* these terms are not in the UMLS Metathesaurus, but are meaningful (or not) in the COVID-19 context. For example, terms like *close contact* relate to the COVID-19 contagion mode.

Figures 5 and 6 illustrate the number of terms out of the TOP@100 terms (in percentage) for each measure (C_Value, F-TFIDF-C_M) and dataset (Papers1-title, Papers2-title):
- In_UMLS: the number of terms in the UMLS Metathesaurus;

- Not_In_UMLS_V: the number of terms that do not exactly match the UMLS terms, but have some variants or are part of the UMLS terms;

- Not_In_UMLS: the number of terms that do not match the UMLS terms at all. We indicate these as new terms. Terms which are not in the UMLS Metathesaurus but which could have greater meaning in the study context or which could be added to the UMLS Metathesaurus.

According to these statistics, we first note that the C_ Value and F-TFIDF-C_M measures enable extraction of more conventional terms or terms in the UMLS Metathesaurus regardless of the corpus. Secondly, we note that F-TFIDF-C_M generates more new terms (Not In UMLS) than C_Value regardless of the corpus. Finally, the num- ber of new terms is more substantial with MWTs (Fig. 6) than SWTs (Fig. 5) regardless of the measure.

## Relevant term extraction from corpora for both measures

This involves quantitative and qualitative analysis of the terms extracted within each corpus, while taking both measures (C_Value and F-TFIDF-C_M) into account. In other words, it consists of analysing terms obtained for both measures, i.e. terms detected at the same time, and also terms specific to each of them.

The quantitative analysis aims to highlight, for each dataset, the number of terms obtained by each measure, the number of terms obtained for both measures, and which are available or not in the UMLS Metathesaurus. While the qualitative measure aims to highlight, in each case, how the terms obtained are important or not regarding the study domain.

For the data representation, we take advantages of Venn Diagram [16], see in Appendix Fig. 10 the distribution of the Papers2-title corpus terms. Terms are organised in different sections. For example, *gene expression, human coronavirus, case report, public health, respiratory syncytial virus, etc.* are available in UMLS Metathesaurus and are recognized by both measures (C_Value and F-TFIDF-C_M). According to the study domain, these terms will tend to be more representative and important in the whole corpus. Moreover, for each measure there are new terms which are not in the UMLS Metathesaurus.

## Discriminant and common term extraction from corpora

In this case, term analysis is performed per dataset or by jointly considering multiple corpora, i.e. between Title, Abstract and Content corpora. Appendix Fig. 11

corresponds to discriminant and common term extraction from Papers1-title, Papers1-abstract and Papers1-content.

There are common terms in the overall corpus such as *gene expression, virus replication, influenza virus, etc.*. These terms tend to be relevant in the Title, Content and Abstract corpora.

Moreover, [*respiratory infection, acute respiratory infection, etc.*], [*innate immune response, endoplasmic reticulum, etc.*], and [*nucleotide sequences, room temperature, etc.*] are discriminant terms in the Title, Abstract and Content corpora.

## The driven term extraction process

We performed a driven term extraction strategy using FASTR. Our proposal addresses two main questions: (1) For a given set of terms, how can new and relevant terms variants be extracted from a corpus based on the terms? (2) Do some of the new terms exist in the UMLS Metathesaurus? In our experiment, we used the common terms extracted in section 5.1.3 based on the fact that they were more representative and relevant throughout the corpora.

Figure 7 shows an example of variant terms extracted with the term *infectious disease*. Among these variants, we only show those which are not in the UMLS Metathesaurus since they are new and might be more informative.

Table 5 contains a list of TOP@10 variants extracted with six initial terms. Among them, we highlighted (in bold) terms matching terms in the UMLS Metathesaurus. Like free mode extraction, term variant mode may be used to extract useful terms.

<figure><img src="image_10.png" /></figure>
<table><caption>**Table 5** Term extraction variations using FASTR</caption><thead><tr><th>Terms</th><th>Infectious disease</th><th>Virus replication</th><th>Laboratory tests</th><th>Respiratory syndrome</th><th>Preventive measures</th><th>Syndrome coronavirus</th></tr></thead><tbody><tr><td rowspan="4">Variations</td><td>Diseases including infectious</td><td>Replication competent viruses</td><td>Laboratory confirmation tests</td><td>Respiratory distress syndrome</td><td>Preventive measures</td><td>Syndrome coronavirus-related coronavirus</td></tr><tr><td>Infectious pulmonary diseases</td><td>replication of N1347A virus</td><td>Laboratory testing</td><td>Respiratory acute syndrome</td><td>Preventive hygienic measures</td><td>Syndrome human coronavirus</td></tr><tr><td>Infectious bursal disease</td><td>Virus optimal replication</td><td>Testing presents isolation laboratories</td><td>Syndrome coronavirus and respiratory</td><td>Prevention community-engaged measures</td><td>Syndromic Surveillance Coronavirus</td></tr><tr><td>Infectious lung diseases</td><td>Replicating influenza viruses</td><td>Laboratory diagnostic testing</td><td>Respiratory tract syndromic</td><td>Preventive health measures</td><td>Syndrome virus coronavirus</td></tr><tr><td></td><td>Infectious acute disease</td><td>Replication of human viruses</td><td>Laboratory genomic testing</td><td>Respiratory insufficiency syndrome</td><td>Preventive behavioral measures</td><td>Coronavirus Associated Syndromes</td></tr></tbody></table>

Terms in the UMLS Metathesaurus in bold

<figure><img src="image_11.png" /><figcaption>**Fig. 8** Combined intra-corpus term extraction strategies</figcaption></figure>

## Combined strategies for term analysis

Combined strategies for term analysis concern two levels: (1) Intra-corpus term extraction, and (2) Inter-corpus term extraction.

- Combined intra-corpus term extraction strategies: these are geared towards extracting common or discriminant terms from a given corpus. To this end, extracted terms from both measures are compared. We show the process in Fig. 8, where the set of terms Set(Cp) extracted from the corpus Cp (Title, Abstract or Content) using each measure (C_Value, F-TFIDF-C_M) are jointly compared with the UMLS Metathesaurus terms. Set A represents corpus terms specifically extracted with C_Value, set B represents terms that are specific to F-TFIDF-C_M, while set C represents common terms from both measures and UMLS Metathesaurus elements. We consider that sets A and B are discriminant terms of the corpus according to the measures, and otherwise set C is considered as containing common terms or the most representative terms of the corpus. The new term extraction process

with FASTR is run with one of the combined sets (discriminant or common) and the corpus.

- Combined inter-corpus term extraction strategies: these are geared towards extracting common and discriminant terms, while taking several corpora into account for a given measure. As illustrated in Fig. 9, for each measure (C_Value or F-TFIDF-C_M), the sets of terms Set(Cp1), Set(Cp2), Set(Cp3) are extracted respectively from corpus Cp1 (Title), Cp2 (Abstract), and Cp3 (Content). These sets are compared in order to compute the common term set D for both corpora, and discriminant term sets A, B, C, respectively, for corpora Cp2, Cp1 and Cp3. In this context, new terms are extracted using one of the combined sets with one corpus (Cpx).

## Case study: epidemic intelligence

Epidemic intelligence (EI) aims to detect, investigate and monitor potential health threats in a timely manner [33]. In addition to conventional surveillance system
<figure><img src="image_12.png" /></figure>

monitoring, such as outbreak notifications from the World Organisation for Animal Health (OIE), the EI process increasingly mainstreams unstructured data from informal sources such as online news. Several web-based surveillance systems have been developed and used to support public health and animal health surveillance (ProMED [25], HealthMap [14], GPHIN [29], PADI-web [38], etc.). In this case study, we focused on the choice keywords with the PADI-Web system for COVID-19 surveillance (i.e. driven surveillance) and for monitoring unknown diseases (i.e. syndromic surveillance).

The Platform for Automated extraction of Disease Information from the web (PADI-web<fnref n="5" />) is an automated surveillance system for monitoring the emergence of animal infectious diseases, including zoonoses [1, 38]. PADI-web monitors Google News through specific really simple syndication (RSS) feeds, targeting diseases of interest (e.g. African swine fever, avian influenza, etc.). PADI-web also uses unspecific RSS feeds, consisting of combinations of symptoms and hosts (i.e. species), thus allowing syndromic surveillance and detection of unusual disease events. RSS feeds consists of combinations of different categories of terms (i.e. keywords) including symptoms, disease names and species.

PADI-Web has been used for monitoring COVID-19 disease [39]. In this context, the choice of COVID-19 surveillance terms is crucial.

In the following subsections, we discuss the choice of terms given by ITEXT-BIO to use in the PADI-Web

system [38] and other web-based surveillance systems [14, 25, 29] for COVID-19 and syndromic surveillance. This enables evaluation of the relevance of terms generated by our approach for a dedicated task, i.e. web-based health surveillance.

## Relevant term extraction

We compared the relevance of the top 10 terms extracted from Papers2 corpora with either C_Value or F-TFIDF-C (Table 6). Table 9 gives more details on these terms. The relevance was assessed by classifying the terms in one or more of the following categories:

- COVID-19 surveillance: epidemiological terms specific to COVID-19 (e.g. *coronavirus spike*).

- Syndromic surveillance: epidemiological terms not specific to a particular disease (e.g. *infectious bronchiitis*).

- Domain relevant: terms related to health, i.e. either to specific diseases (e.g. *porcine epidemic diarrhoea*) or unspecific (e.g. *virus infections*). The Domain relevant category thus includes the two previous categories, plus diseases other than COVID-19.

- Part of disease multiword expression (MWE): part of a multiword expression corresponding to a disease name (e.g. *East respiratory syndrome* for *Middle East syndrome coronavirus*).

Among the terms extracted with C_Value from Titles, Abstracts or Titles and Abstracts, six to seven were parts of disease MWE. Only one term extracted with

<footnote n="5">⁵ https://padi-web.cirad.fr/en/.</footnote>
<table><caption>Table 6 Relevance of terms extracted from Papers2 depending on the metrics (C_Value or F-TFIDF-C_M)</caption><thead><tr><th>Corpus (Papers2)</th><th>Measure</th><th>n</th><th>Domain relevant</th><th>COVID-19 surveillance</th><th>Syndromic surveillance</th><th>Part of disease MWE</th></tr></thead><tbody><tr><td>Title</td><td>C_Value</td><td>10</td><td>3</td><td>0</td><td>2</td><td>6</td></tr><tr><td>Title</td><td>F-TFIDF-C_M</td><td>9</td><td>4</td><td>1</td><td>1</td><td>1</td></tr><tr><td>Abstract</td><td>C_Value</td><td>10</td><td>1</td><td>0</td><td>0</td><td>6</td></tr><tr><td>Abstract</td><td>F-TFIDF-C_M</td><td>10</td><td>5</td><td>1</td><td>2</td><td>1</td></tr><tr><td>Content</td><td>C_Value</td><td>10</td><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><td>Content</td><td>F-TFIDF-C_M</td><td>10</td><td>2</td><td>0</td><td>2</td><td>0</td></tr><tr><td>Title + abstract</td><td>C_Value</td><td>10</td><td>3</td><td>0</td><td>0</td><td>7</td></tr><tr><td>Title + abstract</td><td>F-TFIDF-C_M</td><td>0</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>Title + content</td><td>C_Value</td><td>3</td><td>1</td><td>0</td><td>0</td><td>2</td></tr><tr><td>Title + content</td><td>F-TFIDF-C_M</td><td>0</td><td>-</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

F-TFIDF-C_M was a part of disease MWE. C_Value could thus be of particular interest for extracting disease name variants, even if they are incomplete. For domain relevant COVID-19 surveillance and syndromic surveillance terms, F-TFIDF-C_M obtained better results than C_Value, even when the frequency of relevant terms was low (from one to five out of ten terms). No common terms were extracted from (Title + Abstract) or from (Title + Content) using F-TFIDF-C_M. Using C_Value, only three common terms were extracted from Title + Content. Among the top 10 terms extracted from Title + Abstract with these metrics, seven were parts of disease MWE. Regardless of the term category, we extracted more relevant terms from Titles and Abstracts than from Contents. This is in line with the fact that Title and Abstracts are more rich in key information and relevant terms due to their length limitation.

## Driven term extraction

We selected terms extracted in Section 6.1: respiratory tract, viral infections, SARS coronavirus, incubation period, influenza virus, respiratory infections and infectious diseases. We randomly extracted the variants with FASTR (Section The driven term extraction approach). An epidemiologist manually evaluated the relevance of 10 randomly selected variants per term. Among the 60 evaluated terms (see Table 10), 72% (43/60) were relevant and 7% (4/60) were irrelevant. For 13 variants (22%), the relevance could not be assessed because the expression was truncated and ambiguous, such as “disease has an infectious” for the term “infectious diseases”. FASTR thus seems to be an effective tool for generating term variants efficiently. However, we noted that FASTR generated up to 774 variants for a single term. Thus, to avoid random selection of terms, it would be interesting to compute a

relevance index that could be used to rank the proposed variants. Besides, several extracted variants were fragments of expressions that could not be evaluated. This issue could be overcome by displaying the variant context (i.e. the sentence in which the variants appeared).

## Conclusion

In this paper we describe ITEXT-BIO, a generic methodology for biomedical term extraction. We show how it allows users to extract terms (or concepts) from different types of textual data using several combined strategies. The free term extraction approach extracts terms from corpora, while the driven term extraction approach extracts, from a corpus and a set of terms, a set of variations of these terms.

We illustrate that the proposed combined strategies based on statistical measures and textual segments help efficiently extract and categorize terms (representative, discriminant and new terms) from a corpus or corpora. We also quantitatively and qualitatively analysed the extracted terms to determine those related to the study domain and those that could be considered as emerging terminology for disease monitoring.

Our future studies will focus on term extraction and analysis by: (i) taking different sections of papers into account and applying the methodology to different types of corpora derived from newspapers or social media such as Twitter, (ii) considering combinations of tools other than BioTex, and (iii) introducing word embedding strategies like BERT [10] to capture semantic aspects of the extracted terms in order to reduce context ambiguity.

## Appendix A

See Tables 7, 8, 9, and 10.
<table><caption>Table 7 Best ranked terms extracted from Paper1 using F-TFIDF-C_M</caption><thead><tr><th colspan="2">Title Corpus</th><th colspan="2">Abstract Corpus</th><th colspan="2">Content Corpus</th></tr><tr><th>terms</th><th>rank</th><th>terms</th><th>rank</th><th>terms</th><th>rank</th></tr></thead><tbody><tr><td>Respiratory syncytial virus</td><td>1.9880</td><td>Public health</td><td>1.9986</td><td>Additional file</td><td>1.9976</td></tr><tr><td>Middle east respiratory syndrome coronavirus</td><td>1.9846</td><td>Infectious diseases</td><td>1.9979</td><td>Infectious disease</td><td>1.997</td></tr><tr><td>Systematic review</td><td>1.9842</td><td>Immune responses</td><td>1.9976</td><td>nk cells</td><td>1.997</td></tr><tr><td>Open access</td><td>1.9819</td><td>Influenza virus</td><td>1.9976</td><td>Health care</td><td>1.996</td></tr><tr><td>Zika virus</td><td>1.9819</td><td>t cells</td><td>1.9975</td><td>Endothelial cells</td><td>1.9957</td></tr><tr><td>Gene expression</td><td>1.9795</td><td>Virus infection</td><td>1.9974</td><td>Frequency domain</td><td>1.9957</td></tr><tr><td>Virology journal</td><td>1.9788</td><td>Respiratory tract</td><td>1.9973</td><td>Ebola virus</td><td>1.9948</td></tr><tr><td>Human coronavirus</td><td>1.976</td><td>Viral infections</td><td>1.9969</td><td>Influenza infection</td><td>1.9943</td></tr><tr><td>Case report</td><td>1.9756</td><td>RNA viruses</td><td>1.9967</td><td>Real-time rt-pcr</td><td>1.9933</td></tr><tr><td>Syncytial virus</td><td>1.9752</td><td>Acute respiratory syndrome</td><td>1.9961</td><td>Incubation period</td><td>1.99325</td></tr><tr><td>t cell</td><td>1.9746</td><td>95percent ci</td><td>1.996</td><td>Health emergency</td><td>1.9932</td></tr><tr><td>Infectious bronchitis</td><td>1.9726</td><td>Ebola virus</td><td>1.9945</td><td>Index patient</td><td>1.9932</td></tr><tr><td>Sars coronavirus</td><td>1.9723</td><td>Influenza viruses</td><td>1.9943</td><td>Membrane rafts</td><td>1.9931</td></tr><tr><td>BMC public health</td><td>1.9701</td><td>Avian influenza</td><td>1.9939</td><td>pcr products</td><td>1.9929</td></tr><tr><td>t cells</td><td>1.9689</td><td>Respiratory tract infections</td><td>1.9938</td><td>2c atpase</td><td>1.9926</td></tr><tr><td>Acute respiratory infection</td><td>1.9672</td><td>Health care</td><td>1.9925</td><td>b cell</td><td>1.9924</td></tr><tr><td>Mini review</td><td>1.9636</td><td>Hepatitis c</td><td>1.9922</td><td>Close contact</td><td>1.9924</td></tr><tr><td>Respiratory viral infections</td><td>1.9636</td><td>Type I</td><td>1.9918</td><td>Final dataset</td><td>1.9922</td></tr><tr><td>BMC public</td><td>1.9625</td><td>Cell line</td><td>1.9914</td><td>3d8 scfv</td><td>1.9921</td></tr><tr><td>Ebola virus disease</td><td>1.9592</td><td>Spike protein</td><td>1.9909</td><td>Pol ii</td><td>1.992</td></tr><tr><td>Supplementary information</td><td>1.9574</td><td>Codon usage</td><td>1.9908</td><td>3c pro</td><td>1.992</td></tr><tr><td>Community-acquired pneumonia</td><td>1.9543</td><td>Pandemic influenza</td><td>1.9907</td><td>Influenza pandemic</td><td>1.9919</td></tr><tr><td>Global health</td><td>1.9543</td><td>Endoplasmic reticulum</td><td>1.9904</td><td>Phylogenetic tree</td><td>1.9918</td></tr><tr><td>Peer review</td><td>1.9543</td><td>Saudi Arabia</td><td>1.9904</td><td>Protein vi</td><td>1.9917</td></tr><tr><td>Japanese encephalitis virus</td><td>1.9512</td><td>Innate immunity</td><td>1.9903</td><td>ag nps</td><td>1.9916</td></tr><tr><td>Innate immunity</td><td>1.9488</td><td>Porcine epidemic</td><td>1.9903</td><td>Influenza b</td><td>1.99125</td></tr><tr><td>Multiple sclerosis</td><td>1.9488</td><td>Global health</td><td>1.9902</td><td>ifn β – 1α</td><td>1.991</td></tr><tr><td>Human rhinovirus</td><td>1.9466</td><td>Vaccine development</td><td>1.9901</td><td>ill patients</td><td>1.9908</td></tr><tr><td>Supplementary material</td><td>1.9442</td><td>Cell death</td><td>1.9898</td><td>Poly tail</td><td>1.9908</td></tr><tr><td>Cell entry</td><td>1.9417</td><td>Infectious disease</td><td>1.9896</td><td>Host range</td><td>1.9906</td></tr><tr><td>Coronavirus spike</td><td>1.9417</td><td>Peripheral blood</td><td>1.9895</td><td>Cyclin d3</td><td>1.9903</td></tr><tr><td>Human adenovirus</td><td>1.9417</td><td>Hong Kong</td><td>1.9894</td><td>Sequence accession</td><td>1.9903</td></tr><tr><td>East respiratory syndrome coronavirus</td><td>1.9414</td><td>Immune cells</td><td>1.9888</td><td>Antiviral drugs</td><td>1.9897</td></tr><tr><td>Mers coronavirus</td><td>1.9388</td><td>Cell cycle</td><td>1.9886</td><td>Subunit vaccines</td><td>1.9897</td></tr><tr><td>West Africa</td><td>1.9388</td><td>Clinical trials</td><td>1.9885</td><td>Protein sequences</td><td>1.9895</td></tr><tr><td>Molecular epidemiology</td><td>1.9323</td><td>Infection control</td><td>1.9884</td><td>Oil spill</td><td>1.9895</td></tr><tr><td>National natural science</td><td>1.931</td><td>Mass spectrometry</td><td>1.9883</td><td>Swine flu</td><td>1.9894</td></tr><tr><td>Natural science foundation</td><td>1.931</td><td>Genome sequence</td><td>1.9881</td><td>Membrane proteins</td><td>1.9893</td></tr><tr><td>Rift valley fever</td><td>1.931</td><td>Clinical samples</td><td>1.9877</td><td>Contact tracing</td><td>1.9891</td></tr><tr><td>National natural science foundation</td><td>1.9307</td><td>Acute respiratory infections</td><td>1.9874</td><td>sars 3a</td><td>1.9889</td></tr><tr><td>Influenza infection</td><td>1.9284</td><td>Severe disease</td><td>1.9868</td><td>Critical care</td><td>1.9888</td></tr><tr><td>Protein response</td><td>1.9284</td><td>Hepatitis b</td><td>1.9864</td><td>hk-2 cells</td><td>1.9888</td></tr><tr><td>Science foundation</td><td>1.9284</td><td>Host response</td><td>1.9864</td><td>ap2 group</td><td>1.9887</td></tr><tr><td>Supplementary materials</td><td>1.9284</td><td>Type II</td><td>1.9864</td><td>prp sc</td><td>1.9887</td></tr><tr><td>Natural science</td><td>1.9241</td><td>Nucleic acids</td><td>1.9862</td><td>t-cell responses</td><td>1.9887</td></tr><tr><td>Respiratory syndrome coronavirus infection</td><td>1.9241</td><td>Surveillance systems</td><td>1.9859</td><td>DNA vaccines</td><td>1.9886</td></tr><tr><td>Influenza virus</td><td>1.9212</td><td>Influenza virus infection</td><td>1.9852</td><td>Reverse genetics</td><td>1.9886</td></tr></tbody></table>
<table><caption>Table 7 (continued)</caption><caption>F-TFIDF-C_M</caption><thead><tr><th colspan="2">Title Corpus</th><th colspan="2">Abstract Corpus</th><th colspan="2">Content Corpus</th></tr><tr><th>terms</th><th>rank</th><th>terms</th><th>rank</th><th>terms</th><th>rank</th></tr></thead><tbody><tr><td>Obstructive pulmonary disease</td><td>1.92</td><td>Antiviral drugs</td><td>1.9851</td><td>Health system</td><td>1.9884</td></tr><tr><td>Emerging microbes</td><td>1.9193</td><td>DNA vaccine</td><td>1.9847</td><td>b7 – h1</td><td>1.9884</td></tr><tr><td>Original research</td><td>1.9193</td><td>Influenza infection</td><td>1.9845</td><td>hcv infection</td><td>1.9883</td></tr><tr><td>Retrospective study</td><td>1.9193</td><td>Reference genes</td><td>1.9842</td><td>Lung cancer</td><td>1.9879</td></tr><tr><td>Phylogenetic analysis</td><td>1.9153</td><td>Cell types</td><td>1.984</td><td>Nucleocapsid protein</td><td>1.9879</td></tr><tr><td>Respiratory syndrome coronavirus</td><td>1.9151</td><td>b cell</td><td>1.9835</td><td>3c protease</td><td>1.9878</td></tr><tr><td>Clinical characteristics</td><td>1.9138</td><td>Vaccine candidates</td><td>1.9835</td><td>tgev infection</td><td>1.9878</td></tr><tr><td>Mass spectrometry</td><td>1.9138</td><td>Host species</td><td>1.9833</td><td>cs dna</td><td>1.9878</td></tr><tr><td>National natural</td><td>1.9138</td><td>Respiratory viral infections</td><td>1.9832</td><td>Risk perception</td><td>1.9875</td></tr><tr><td>Rift valley</td><td>1.9138</td><td>Endothelial cells</td><td>1.9829</td><td>s1 protein</td><td>1.9875</td></tr><tr><td>Science china</td><td>1.9138</td><td>Sequence data</td><td>1.9829</td><td>Ring vaccination</td><td>1.9875</td></tr><tr><td>Valley fever</td><td>1.9138</td><td>DNA viruses</td><td>1.9826</td><td>Syrian hamster</td><td>1.9873</td></tr><tr><td>Respiratory virus infections</td><td>1.913</td><td>Host innate</td><td>1.9826</td><td>Wild mice</td><td>1.9873</td></tr><tr><td>Syndrome coronavirus</td><td>1.9096</td><td>Parainfluenza virus</td><td>1.9824</td><td>Yellow fever</td><td>1.9873</td></tr><tr><td>Classical swine fever virus</td><td>1.9087</td><td>Tract infections</td><td>1.9822</td><td>Climate change</td><td>1.9873</td></tr><tr><td>b cells</td><td>1.9074</td><td>South Korea</td><td>1.9821</td><td>Public health services</td><td>1.9873</td></tr><tr><td>Host response</td><td>1.9074</td><td>Acute respiratory infection</td><td>1.9817</td><td>Index patients</td><td>1.9872</td></tr><tr><td>Science foundation of china</td><td>1.9074</td><td>Reproduction number</td><td>1.9816</td><td>Small rna</td><td>1.9872</td></tr><tr><td>Viral proteins</td><td>1.9074</td><td>Surveillance system</td><td>1.9816</td><td>IC activity</td><td>1.9871</td></tr><tr><td>Virus disease</td><td>1.9065</td><td>Causative agent</td><td>1.9813</td><td>Ebola virus disease</td><td>1.9868</td></tr><tr><td>Clinical infectious diseases</td><td>1.9048</td><td>Multiple sclerosis</td><td>1.9811</td><td>RNA chaperone</td><td>1.9867</td></tr><tr><td>World health organization</td><td>1.9048</td><td>rsv infection</td><td>1.9809</td><td>Caco-2 cells</td><td>1.9867</td></tr><tr><td>Antiviral agents</td><td>1.9001</td><td>Cellular proteins</td><td>1.9808</td><td>m2 channel</td><td>1.9865</td></tr><tr><td>Cell culture</td><td>1.9001</td><td>West nile virus</td><td>1.9806</td><td>Overlapping genes</td><td>1.9865</td></tr><tr><td>Pulmonary disease</td><td>1.9001</td><td>Respiratory diseases</td><td>1.9805</td><td>Nasal mucosa</td><td>1.9865</td></tr><tr><td>Study protocol</td><td>1.9001</td><td>tgev infection</td><td>1.9805</td><td>Hepatitis e</td><td>1.9865</td></tr><tr><td>Dengue virus</td><td>1.8946</td><td>e protein</td><td>1.9802</td><td>Genetic drift</td><td>1.9865</td></tr><tr><td>Public health</td><td>1.893</td><td>Gene expression</td><td>1.9801</td><td>a7 gfp</td><td>1.9865</td></tr><tr><td>RNA replication</td><td>1.8915</td><td>Structural proteins</td><td>1.9799</td><td>Tumor cells</td><td>1.9864</td></tr><tr><td>Japanese encephalitis</td><td>1.8902</td><td>Acute respiratory tract</td><td>1.9792</td><td>Tanguticum nanoparticles</td><td>1.9864</td></tr><tr><td>Syndrome coronavirus infection</td><td>1.8864</td><td>Hand hygiene</td><td>1.9792</td><td>cfu ml</td><td>1.9864</td></tr><tr><td>Human respiratory syncytial virus</td><td>1.8841</td><td>Disease transmission</td><td>1.9788</td><td>Ward closure</td><td>1.9861</td></tr><tr><td>Synonymous codon usage</td><td>1.8824</td><td>Human rhinovirus</td><td>1.9785</td><td>Case definitions</td><td>1.9861</td></tr><tr><td>Clinical infectious</td><td>1.8813</td><td>Bacterial infections</td><td>1.9781</td><td>Richards model</td><td>1.9861</td></tr><tr><td>Health organization</td><td>1.8813</td><td>Cancer cells</td><td>1.9781</td><td>Epimedium koreanum</td><td>1.9861</td></tr><tr><td>Severe pneumonia</td><td>1.8813</td><td>DNA vaccines</td><td>1.9777</td><td>ms2 plp</td><td>1.986</td></tr><tr><td>Dengue virus infection</td><td>1.8772</td><td>Type III</td><td>1.9777</td><td>Gene therapy</td><td>1.9859</td></tr><tr><td>Clinical samples</td><td>1.8768</td><td>Viral pathogenesis</td><td>1.9773</td><td>Integrin b3</td><td>1.9859</td></tr><tr><td>Classical swine fever</td><td>1.8744</td><td>Zoonotic diseases</td><td>1.9773</td><td>Cardiovascular diseases</td><td>1.9859</td></tr><tr><td>Human antibody</td><td>1.869</td><td>Early detection</td><td>1.9765</td><td>Fourth site</td><td>1.9859</td></tr><tr><td>Lassa virus</td><td>1.869</td><td>Lung cancer</td><td>1.9756</td><td>Serial interval</td><td>1.9858</td></tr><tr><td>Pilot study</td><td>1.869</td><td>Nile virus</td><td>1.9756</td><td>trm cells</td><td>1.9858</td></tr><tr><td>Avian influenza viruses</td><td>1.8667</td><td>Human disease</td><td>1.9751</td><td>Electronic supplementary material</td><td>1.9857</td></tr><tr><td>Human respiratory syncytial</td><td>1.8667</td><td>rnase I</td><td>1.9751</td><td>Emergency nurses</td><td>1.9856</td></tr><tr><td>International health regulations</td><td>1.8667</td><td>Health systems</td><td>1.9746</td><td>Pet substrate</td><td>1.9856</td></tr><tr><td>Hepatitis c virus infection</td><td>1.8661</td><td>Incubation period</td><td>1.9746</td><td>fcov type</td><td>1.9856</td></tr><tr><td>Infectious bronchitis virus strain</td><td>1.8661</td><td>Rabies virus</td><td>1.9746</td><td>s1 text</td><td>1.9856</td></tr></tbody></table>
<table><caption>Table 7 (continued)</caption><caption>F-TFIDF-C_M</caption><thead><tr><th colspan="2">Title Corpus</th><th colspan="2">Abstract Corpus</th><th colspan="2">Content Corpus</th></tr><tr><th>terms</th><th>rank</th><th>terms</th><th>rank</th><th>terms</th><th>rank</th></tr></thead><tbody><tr><td>Vaccine development</td><td>1.8601</td><td>Adaptive immunity</td><td>1.9741</td><td>Global health research</td><td>1.9854</td></tr><tr><td>Protects hepatocytes from type I</td><td>1.8564</td><td>Multiplex pcr</td><td>1.9741</td><td>ace2 activity</td><td>1.9853</td></tr><tr><td>Type I interferon signaling disrupts</td><td>1.8564</td><td>nk cells</td><td>1.9741</td><td>β6 ko</td><td>1.9853</td></tr><tr><td>Adaptive immunity</td><td>1.8538</td><td>Feline coronavirus</td><td>1.9735</td><td>Global health</td><td>1.9852</td></tr><tr><td>Adenovirus type</td><td>1.8538</td><td>Human populations</td><td>1.9735</td><td>Ham tsp</td><td>1.9851</td></tr><tr><td>Nonhuman primates</td><td>1.8538</td><td>Common cold</td><td>1.9723</td><td>Blood culture</td><td>1.9849</td></tr></tbody></table>

<table><caption>Table 8 Best ranked terms extracted from Paper1 using C-Value</caption><caption>C-Value</caption><thead><tr><th colspan="2">Title Corpus</th><th colspan="2">Abstract Corpus</th><th colspan="2">Content Corpus</th></tr><tr><th>Terms</th><th>Rank</th><th>Terms</th><th>Rank</th><th>Terms</th><th>Rank</th></tr></thead><tbody><tr><td>Respiratory syndrome</td><td>386.7309</td><td>Public health</td><td>1393.182</td><td>t cells</td><td>2063.1457</td></tr><tr><td>Virus infection</td><td>366.1263</td><td>Respiratory syndrome</td><td>1095.2091</td><td>Public health</td><td>1644.7156</td></tr><tr><td>Porcine epidemic diarrhea virus</td><td>329.7138</td><td>Infectious diseases</td><td>952.5625</td><td>Amino acid</td><td>1409.82415</td></tr><tr><td>Porcine epidemic diarrhea</td><td>318.0</td><td>Immune response</td><td>908.1835</td><td>Immune response</td><td>1400.94835</td></tr><tr><td>Epidemic diarrhea virus</td><td>306.0</td><td>Immune responses</td><td>841.6151</td><td>Influenza virus</td><td>1185.8689</td></tr><tr><td>East respiratory syndrome</td><td>284.0</td><td>Influenza virus</td><td>841.6151</td><td>Immune responses</td><td>1056.536</td></tr><tr><td>Middle east</td><td>261.5188</td><td>t cells</td><td>803.576</td><td>t cell</td><td>1056.37753</td></tr><tr><td>Epidemic diarrhea</td><td>256.7639</td><td>Virus infection</td><td>760.7811</td><td>Gene expression</td><td>1050.6716</td></tr><tr><td>Diarrhea virus</td><td>245.6692</td><td>Respiratory tract</td><td>727.4978</td><td>Viral replication</td><td>1021.5083</td></tr><tr><td>Infectious diseases</td><td>245.6692</td><td>Viral infection</td><td>668.8542</td><td>Infected cells</td><td>939.72426</td></tr><tr><td>Respiratory syndrome coronavirus</td><td>240.0</td><td>Viral replication</td><td>665.6843</td><td>Cell lines</td><td>897.4057</td></tr><tr><td>Influenza a</td><td>225.0647</td><td>Viral infections</td><td>640.3249</td><td>Viral infection</td><td>888.6884</td></tr><tr><td>Public health</td><td>209.2151</td><td>East respiratory syndrome</td><td>638.0</td><td>Virus infection</td><td>872.68035</td></tr><tr><td>Syndrome coronavirus</td><td>191.7805</td><td>Respiratory syndrome coronavirus</td><td>636.0</td><td>Amino acids</td><td>866.816</td></tr><tr><td>Porcine epidemic</td><td>190.1955</td><td>Middle east</td><td>630.8151</td><td>mg ml</td><td>824.4975</td></tr><tr><td>Influenza virus</td><td>182.2707</td><td>Gene expression</td><td>627.6452</td><td>Infectious diseases</td><td>822.27855</td></tr><tr><td>Respiratory tract</td><td>180.6857</td><td>Infectious disease</td><td>613.3805</td><td>Present study</td><td>812.45177</td></tr><tr><td>Middle east respiratory syndrome</td><td>174.1446</td><td>RNA viruses</td><td>603.8707</td><td>Respiratory tract</td><td>812.13477</td></tr><tr><td>Middle east respiratory</td><td>170.0</td><td>Present study</td><td>575.3414</td><td>Epithelial cells</td><td>759.03855</td></tr><tr><td>Respiratory syncytial virus</td><td>166.0</td><td>Respiratory viruses</td><td>551.567</td><td>Previous studies</td><td>732.41119</td></tr><tr><td>Infectious bronchitis</td><td>160.0812</td><td>Acute respiratory syndrome</td><td>516.0</td><td>Room temperature</td><td>714.3426</td></tr><tr><td>Infectious disease</td><td>156.9113</td><td>t cell</td><td>513.5279</td><td>Cell culture</td><td>673.60907</td></tr><tr><td>Infectious bronchitis virus</td><td>156.0</td><td>Syndrome coronavirus</td><td>511.9429</td><td>Additional file</td><td>657.75946</td></tr><tr><td>East respiratory</td><td>136.3068</td><td>Porcine epidemic diarrhea</td><td>506.0</td><td>Viral infections</td><td>635.72848</td></tr><tr><td>Syncytial virus</td><td>134.7218</td><td>95percent ci</td><td>502.4331</td><td>Immune system</td><td>617.97689</td></tr><tr><td>Avian influenza</td><td>131.5519</td><td>Viral rna</td><td>499.2632</td><td>Respiratory syndrome</td><td>617.3429</td></tr><tr><td>Respiratory viruses</td><td>131.5519</td><td>Amino acid</td><td>489.7534</td><td>Cell line</td><td>611.16155</td></tr><tr><td>East respiratory syndrome coronavirus</td><td>130.028</td><td>Respiratory syncytial virus</td><td>472.0</td><td>Infectious disease</td><td>607.04063</td></tr><tr><td>Middle east respiratory syndrome coronavirus</td><td>129.2481</td><td>Cell lines</td><td>443.7895</td><td>μ g ml</td><td>576.13388</td></tr><tr><td>Influenza a virus</td><td>126.0</td><td>Respiratory infections</td><td>426.3549</td><td>Western blot</td><td>568.36754</td></tr><tr><td>Bronchitis virus</td><td>125.212</td><td>Epithelial cells</td><td>424.77</td><td>rnase I</td><td>565.0391</td></tr><tr><td>Respiratory infections</td><td>125.212</td><td>Virus replication</td><td>420.0151</td><td>Virus replication</td><td>560.6012</td></tr><tr><td>Systematic review</td><td>125.212</td><td>Polymerase chain reaction</td><td>408.0</td><td>Cell surface</td><td>543.9591</td></tr></tbody></table>
<table><caption>Table 8 (continued)</caption><thead><tr><th colspan="2">Title Corpus</th><th colspan="2">Abstract Corpus</th><th colspan="2">Content Corpus</th></tr><tr><th>Terms</th><th>Rank</th><th>Terms</th><th>Rank</th><th>Terms</th><th>Rank</th></tr></thead><tbody><tr><td>Ebola virus</td><td>120.4572</td><td>Epidemic diarrhea virus</td><td>406.0</td><td>xx</td><td>542.0572</td></tr><tr><td>Acute respiratory</td><td>117.2872</td><td>Epidemic diarrhea</td><td>402.5805</td><td>Host cell</td><td>539.83825</td></tr><tr><td>Viral infections</td><td>117.2872</td><td>Host cell</td><td>396.2406</td><td>Codon usage</td><td>523.03765</td></tr><tr><td>Virus replication</td><td>115.7023</td><td>Syncytial virus</td><td>378.806</td><td>Viral proteins</td><td>520.6601</td></tr><tr><td>Open access</td><td>109.3624</td><td>Porcine epidemic diarrhea virus</td><td>376.1524</td><td>Respiratory viruses</td><td>515.4298</td></tr><tr><td>Zika virus</td><td>109.3624</td><td>Antiviral activity</td><td>374.0512</td><td>nk cells</td><td>503.2256</td></tr><tr><td>Respiratory tract infections</td><td>102.0</td><td>Risk factors</td><td>374.0512</td><td>Time points</td><td>497.8367</td></tr><tr><td>Viral infection</td><td>101.4376</td><td>Immune system</td><td>369.2963</td><td>Influenza viruses</td><td>492.7648</td></tr><tr><td>Immune response</td><td>99.8526</td><td>Ebola virus</td><td>364.5414</td><td>Important role</td><td>491.0213</td></tr><tr><td>Hepatitis c virus</td><td>98.0</td><td>Chain reaction</td><td>355.0316</td><td>Allergic rhinitis</td><td>486.5835</td></tr><tr><td>Gene expression</td><td>96.6827</td><td>Influenza viruses</td><td>348.6918</td><td>Antiviral activity</td><td>481.3531</td></tr><tr><td>Pandemic influenza</td><td>96.6827</td><td>Infected cells</td><td>347.1068</td><td>Global health</td><td>473.9038</td></tr><tr><td>Respiratory syndrome virus</td><td>96.0</td><td>Diarrhea virus</td><td>340.7669</td><td>mg kg</td><td>470.0998</td></tr><tr><td>Epithelial cells</td><td>95.0978</td><td>Host cells</td><td>334.4271</td><td>Frequency domain</td><td>469.1489</td></tr><tr><td>Complete genome</td><td>93.5128</td><td>Important role</td><td>331.2572</td><td>Control group</td><td>466.13749</td></tr><tr><td>Syndrome virus</td><td>93.5128</td><td>Phylogenetic analysis</td><td>331.2572</td><td>Viral load</td><td>465.34499</td></tr><tr><td>Virology journal</td><td>93.5128</td><td>Polymerase chain</td><td>331.2572</td><td>Binding site</td><td>459.6391</td></tr><tr><td>Hepatitis c</td><td>91.9278</td><td>Respiratory disease</td><td>326.5023</td><td>Expression levels</td><td>453.6162</td></tr><tr><td>Immune responses</td><td>90.3429</td><td>Avian influenza</td><td>324.9173</td><td>Hong Kong</td><td>450.7237</td></tr><tr><td>Genome sequence</td><td>88.7579</td><td>Respiratory tract infections</td><td>320.0</td><td>Clinical signs</td><td>448.8613</td></tr><tr><td>Dengue virus</td><td>87.1729</td><td>Infectious bronchitis</td><td>285.2933</td><td>Protein expression</td><td>448.2274</td></tr><tr><td>Molecular sciences</td><td>84.0029</td><td>Cell culture</td><td>272.6136</td><td>Wild type</td><td>446.7833</td></tr><tr><td>Type i</td><td>84.0029</td><td>Hepatitis c virus</td><td>268.0</td><td>Endothelial cells</td><td>441.4120</td></tr><tr><td>Acute respiratory syndrome</td><td>84.0</td><td>Health care</td><td>264.6887</td><td>Table s1</td><td>438.4006</td></tr><tr><td>Complete genome sequence</td><td>84.0</td><td>Zika virus</td><td>264.6887</td><td>Flow cytometry</td><td>437.4496</td></tr><tr><td>Human coronavirus</td><td>82.4181</td><td>Infectious bronchitis virus</td><td>260.0</td><td>Saudi Arabia</td><td>433.4872</td></tr><tr><td>Respiratory infection</td><td>82.4181</td><td>Tract infections</td><td>258.3489</td><td>Viral genome</td><td>433.3992</td></tr><tr><td>Case report</td><td>80.8331</td><td>Hepatitis c</td><td>255.179</td><td>Negative control</td><td>433.2230</td></tr><tr><td>Tract infections</td><td>80.8331</td><td>Innate immune response</td><td>252.0</td><td>-</td><td>431.7890</td></tr><tr><td>Risk factors</td><td>79.2481</td><td>Monoclonal antibodies</td><td>248.8391</td><td>Cell types</td><td>431.1098</td></tr><tr><td>Spike protein</td><td>77.6632</td><td>Viral genome</td><td>247.2542</td><td>Viral entry</td><td>427.9399</td></tr><tr><td>t cell</td><td>77.6632</td><td>Type I</td><td>242.4993</td><td>Cell death</td><td>425.24544</td></tr><tr><td>Acute respiratory infections</td><td>76.0</td><td>Central nervous system</td><td>242.0</td><td>er stress</td><td>423.185</td></tr><tr><td>Coronavirus infection</td><td>74.4932</td><td>Amino acids</td><td>239.3293</td><td>Significant differences</td><td>420.6490</td></tr><tr><td>RNA viruses</td><td>74.4932</td><td>Animal models</td><td>237.7444</td><td>Health care</td><td>420.4905</td></tr><tr><td>Severe acute respiratory</td><td>72.0</td><td>Real-time pcr</td><td>236.1594</td><td>Tcid 50</td><td>417.3734</td></tr><tr><td>Sars coronavirus</td><td>71.3233</td><td>Dengue virus</td><td>232.9895</td><td>Cathepsin I</td><td>410.5053</td></tr><tr><td>Isothermal amplification</td><td>69.7384</td><td>Viral load</td><td>232.9895</td><td>Risk factors</td><td>408.9203</td></tr><tr><td>Respiratory disease</td><td>69.7384</td><td>World Health Organization</td><td>232.0</td><td>Positive selection</td><td>405.7504</td></tr><tr><td>BMC public health</td><td>66.0</td><td>Cell line</td><td>231.4045</td><td>Cell cycle</td><td>400.9955</td></tr><tr><td>Disease virus</td><td>64.9835</td><td>Viral proteins</td><td>229.8196</td><td>Nucleotide sequences</td><td>397.8256</td></tr><tr><td>t cells</td><td>63.3985</td><td>Nervous system</td><td>226.6496</td><td>Plasma membrane</td><td>393.5990</td></tr><tr><td>Influenza viruses</td><td>61.8135</td><td>Wide range</td><td>223.4797</td><td>Intensive care</td><td>392.2782</td></tr><tr><td>Acute respiratory infection</td><td>60.0</td><td>Virus infections</td><td>221.8948</td><td>Host cells</td><td>384.82889</td></tr><tr><td>Type i interferon</td><td>60.0</td><td>Middle east respiratory syndrome</td><td>220.5832</td><td>Hand hygiene</td><td>383.5609</td></tr><tr><td>Journal frontiers</td><td>58.6436</td><td>Immunodeficiency virus</td><td>218.7248</td><td>Significant difference</td><td>382.6099</td></tr><tr><td>Fever virus</td><td>57.0587</td><td>Spike protein</td><td>218.7248</td><td>Immune cells</td><td>381.02498</td></tr></tbody></table>
<table><caption>Table 8 (continued)</caption><thead><tr><th colspan="2">C-Value</th><th colspan="2">Abstract Corpus</th><th colspan="2">Content Corpus</th></tr><tr><th>Title Corpus</th><th>Terms</th><th>Rank</th><th>Terms</th><th>Rank</th><th>Terms</th><th>Rank</th></tr></thead><tbody><tr><td>Respiratory syncytial</td><td>57.0587</td><td>Life cycle</td><td>217.1399</td><td>Reference genes</td><td>380.3909</td></tr><tr><td>Severe acute</td><td>57.0587</td><td>Recent years</td><td>217.1399</td><td>HIV aids</td><td>377.2211</td></tr><tr><td>Respiratory tract infection</td><td>56.0</td><td>Codon usage</td><td>215.5549</td><td>Avian influenza</td><td>376.8688</td></tr><tr><td>Antiviral activity</td><td>55.4737</td><td>Viral pathogens</td><td>215.5549</td><td>Serum samples</td><td>375.8625</td></tr><tr><td>BMC infectious</td><td>55.4737</td><td>Pandemic influenza</td><td>213.9699</td><td>Body weight</td><td>375.0021</td></tr><tr><td>Hong Kong</td><td>55.4737</td><td>Clinical signs</td><td>212.385</td><td>Fig. 1a</td><td>374.0511</td></tr><tr><td>Viral replication</td><td>55.4737</td><td>Dendritic cells</td><td>209.2151</td><td>Membrane fusion</td><td>374.0511</td></tr><tr><td>Virus infections</td><td>55.4737</td><td>Acute respiratory syndrome coronavirus</td><td>208.9735</td><td>Clinical trials</td><td>373.8750</td></tr><tr><td>BMC infectious diseases</td><td>54.0</td><td>Bronchitis virus</td><td>207.6301</td><td>Time point</td><td>373.3719</td></tr><tr><td>Respiratory viral infections</td><td>54.0</td><td>Endoplasmic reticulum</td><td>207.6301</td><td>Protein synthesis</td><td>369.2962</td></tr><tr><td>Case study</td><td>53.8887</td><td>RNA virus</td><td>207.6301</td><td>Dengue virus</td><td>367.7113</td></tr><tr><td>Dendritic cells</td><td>53.8887</td><td>Saudi Arabia</td><td>207.6301</td><td>e protein</td><td>367.7113</td></tr><tr><td>Mini review</td><td>53.8887</td><td>Innate immunity</td><td>206.0451</td><td>High levels</td><td>365.3339</td></tr><tr><td>RNA virus</td><td>53.8887</td><td>Recent studies</td><td>206.0451</td><td>Virus particles</td><td>364.5414</td></tr><tr><td>Transmissible gastroenteritis</td><td>53.8887</td><td>Economic losses</td><td>204.4602</td><td>Target cells</td><td>362.5601</td></tr><tr><td>BMC public</td><td>52.3038</td><td>Porcine epidemic</td><td>204.4602</td><td>Viral particles</td><td>360.4204</td></tr><tr><td>Monoclonal antibodies</td><td>52.3038</td><td>World health</td><td>204.4602</td><td>Dendritic cells</td><td>357.5675</td></tr><tr><td>Creative commons cc-by 4</td><td>51.0824</td><td>Global health</td><td>202.8752</td><td>Total number</td><td>356.4580</td></tr><tr><td>Influenza pandemic</td><td>50.7188</td><td>Type 1</td><td>202.8752</td><td>Cancer cells</td><td>356.0883</td></tr><tr><td>Type 1</td><td>50.7188</td><td>Vaccine development</td><td>201.2902</td><td>Disease control</td><td>355.2957</td></tr></tbody></table>
<table><caption>**Table 9** Expanded terms from Table 6</caption><thead><tr><th>sous_corpus</th><th>Measure</th><th>Term</th><th>Domain relevant</th><th>COVID-19 surveillance</th><th>Syndromic surveillance</th><th>Incomplet disease name</th></tr></thead><tbody><tr><td rowspan="15">title</td><td rowspan="15">C-value</td><td>Respiratory syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Porcine epidemic diarrhea</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Epidemic diarrhea virus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Acute respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Public access</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Diarrhea virus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Infectious bronchitis</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td>Acute respiratory</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Bronchitis virus</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td rowspan="10">F-TFIDF-C</td><td>Journal pre-proof</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Virology journal</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Influenza pandemic</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Coronavirus spike</td><td>y</td><td>y</td><td>n</td><td>n</td></tr><tr><td>BMC public health</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Influenza virus infection</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Emerging infectious</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td>Porcine circovirus type</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Codon usage</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td rowspan="15">abstract</td><td rowspan="15">C-value</td><td>Acute respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Respiratory syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>East respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Present study</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Chain reaction</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Syncytial virus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Porcine epidemic diarrhea</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Polymerase chain</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td rowspan="10">F-TFIDF-C</td><td>Virus infections</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td>Porcine epidemic</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Clinical samples</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Codon usage</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Mers-cov infection</td><td>y</td><td>y</td><td>n</td><td>n</td></tr><tr><td>Pandemic influenza</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Viral entry</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td>95 percent confidence interval</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Immune cells</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Influenza pandemic</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Sono stati</td><td>n</td><td>n</td><td>n</td><td>n</td></tr></tbody></table>
<table><caption>**Table 9 (continued)**</caption><thead><tr><th>sous_corpus</th><th>Measure</th><th>Term</th><th>Domain relevant</th><th>COVID-19 surveillance</th><th>Syndromic surveillance</th><th>Incomplet disease name</th></tr></thead><tbody><tr><td rowspan="14">content</td><td rowspan="8">C-value</td><td>Infected cells</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Present study</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Individual components</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Essential medicines</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Previous studies</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>de los</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Functional task</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>der Schwangerschaft</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td rowspan="10">F-TFIDF-C</td><td>Health emergency</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td>Membrane rafts</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>pcr products</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>afa dr</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>COD trypsin</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>2c atpase</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Naked mole</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Intracellular delivery</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Close contact</td><td>y</td><td>n</td><td>y</td><td>n</td></tr><tr><td>Final dataset</td><td>n</td><td>n</td><td>n</td><td>n</td></tr><tr><td rowspan="11">title + abstract</td><td rowspan="10">C-value</td><td>Respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Acute respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Respiratory syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>East respiratory syndrome</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Syncytial virus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Porcine epidemic diarrhea</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Antiviral activity</td><td>y</td><td>n</td><td>n</td><td>n</td></tr><tr><td>Acute respiratory syndrome coronavirus</td><td>n</td><td>n</td><td>n</td><td>y</td></tr><tr><td>Infectious bronchitis</td><td>y</td><td>n</td><td>n</td><td>n</td></tr></tbody></table>

Each term has been evaluated by an expert according 4 criteria: domain relevant, COVID-19 surveillance, syndromic surveillance, incomplet disease name (y: yes, n: no)
<table><caption>Table 10 60 terms randomly selected from FASTR variants (Section The driven term extraction approach)</caption><thead><tr><th>Influenza virus</th><th>Evaluation</th><th>Respiratory infections</th><th>Evaluation</th><th>Infectious diseases</th><th>Evaluation</th></tr></thead><tbody><tr><td>Influenza a/wsn/33 virus</td><td>Not relevant</td><td>Respiratory virus infections</td><td>Relevant</td><td>Diseases relates to infectious</td><td>Relevant</td></tr><tr><td>Viruses and conventional influenza</td><td>Relevant</td><td>Respiratory viral infection</td><td>Relevant</td><td>Disease called feline infectious</td><td>Relevant</td></tr><tr><td>Virus remains the influenza</td><td>Lack of context</td><td>Infection by respiratory</td><td>Relevant</td><td>Infectious animal diseases</td><td>Relevant</td></tr><tr><td>Influenza by virus</td><td>Not relevant</td><td>Infections of the respiratory</td><td>Relevant</td><td>Infectious enteric diseases</td><td>Relevant</td></tr><tr><td>Influenza vaccine virus</td><td>Relevant</td><td>Infect respiratory</td><td>Relevant</td><td>Disease without being infectious</td><td>Not relevant</td></tr><tr><td>Virus and canine influenza</td><td>Relevant</td><td>Infections are respiratory</td><td>Relevant</td><td>Disease has an infectious</td><td>Lack of context</td></tr><tr><td>Virus influenza</td><td>Relevant</td><td>Infected with respiratory</td><td>Relevant</td><td>Infectious disease</td><td>Relevant</td></tr><tr><td>Viruses such as influenza</td><td>Relevant</td><td>Infection with other respiratory</td><td>Relevant</td><td>Disease named it infectious</td><td>Lack of context</td></tr><tr><td>Influenza b viruses</td><td>Relevant</td><td>Respiratory virus infection</td><td>Relevant</td><td>Infectious swine diseases</td><td>Relevant</td></tr><tr><td>Viruses and emerging influenza</td><td>Relevant</td><td>Infection transmitted via respiratory</td><td>Relevant</td><td>Disease models for infectious</td><td>Lack of context</td></tr><tr><td>Viral infections</td><td>Evaluation</td><td>Sars coronavirus</td><td>Evaluation</td><td>Incubation period</td><td>Evaluation</td></tr><tr><td>viral bronchopulmonary infection</td><td>Relevant</td><td>Coronavirus is urbani sars</td><td>Not relevant</td><td>Incubating period</td><td>Relevant</td></tr><tr><td>Virally infected</td><td>Relevant</td><td>Coronavirus of 18 sars</td><td>Not relevant</td><td>Periods of incubation</td><td>Relevant</td></tr><tr><td>Viral respiratory infections</td><td>Relevant</td><td>Coronavirus that causes sars</td><td>Relevant</td><td>Incubation periods</td><td>Relevant</td></tr><tr><td>Infection and encounter virally</td><td>Lack of context</td><td>Coronavirus named sars</td><td>Relevant</td><td>Period of incubation</td><td>Relevant</td></tr><tr><td>Infection or viral</td><td>Lack of context</td><td>Coronavirus related to sars</td><td>Relevant</td><td>Period than incubation</td><td>Lack of context</td></tr><tr><td>Viral skin infection</td><td>Relevant</td><td>Coronavirus isolated from sars</td><td>Relevant</td><td>Period and incubation</td><td>Lack of context</td></tr><tr><td>Virals infection</td><td>Relevant</td><td>Coronavirus responsable du sars</td><td>Relevant</td><td>Incubation for period</td><td>Lack of context</td></tr><tr><td>Infection with one viral</td><td>Relevant</td><td>Sars -associated coronavirus</td><td>Relevant</td><td>Period and incubating</td><td>Lack of context</td></tr><tr><td>viral opportunistic infections</td><td>Relevant</td><td>Sars human coronavirus</td><td>Relevant</td><td>Period covering an incubation</td><td>Relevant</td></tr><tr><td>infection at high viral</td><td>Lack of context</td><td>Sars and coronavirus</td><td>Relevant</td><td>Period of extrinsic incubation</td><td>Relevant</td></tr></tbody></table>
**Appendix B**

See Figs. 10 and 11.

<figure><img src="image_13.png" /><figcaption>**Fig. 10** Distribution of concepts according to the measures and their presence in the UMLS Metathesaurus: from Papers2-title corpus</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>**Fig. 11** Distribution of representative concepts when taking multiple corpora into account using C_Value: Papers1 corpora</figcaption></figure>
Acknowledgements

This study was partially funded by EU grant 874850 MOOD and is catalogued
as MOOD 003. The contents of this publication are the sole responsibility of
the authors and do not necessarily reflect the views of the European Com-
mission. This study was partially funded by the French Agricultural Research
Centre for International Development (CIRAD) the French General Directorate
for Food (DGAL), and the SONGES Project (FEDER and Occitanie Region).
The research was also supported by the French National Research Agency
(ANR) under the Investments for the Future Program, referred to as ANR-16-
CONV-0004, #DigitAg.

Author details

¹INRAE, Montpellier, France. ²TETIS, Univ Montpellier, AgroParisTech, CIRAD, CNRS, INRAE, Montpellier, France. ³CIRAD, F-34398 Montpellier, France. ⁴ASTRE, Univ Montpellier, CIRAD, INRAE, Montpellier, France. ⁵AgroParisTech, Montpellier, France.

Received: 23 December 2020 Accepted: 6 June 2021

Published online: 10 July 2021

References

1. Arsevska E, Valentin S, Rabatel J, de Goër de Hervé J, Falala S, Lancelot R, Roche M. Web monitoring of emerging animal infectious diseases integrated in the French Animal Health Epidemic Intelligence System. PLOS ONE. 2018;13(8):e0199960. https://doi.org/10.1371/journal.pone.0199960.

2. Azarafza M, Feizi-Derakhshi MR, Shendi MB. Textrank-based microblogs keyword extraction method for Persian language. Conference: 3rd International Congress on Science and Engineering, Hamburg, Germany, 2020

3. Bodenreider O. The unified medical language system (umls): integrating biomedical terminology. Nucleic Acids Res. 2004;32(suppl 1):D267-70.

4. Bracewell DB, Ren F, Kuriowa S. Multilingual single document keyword extraction for information retrieval. In: 2005 International Conference on Natural Language Processing and Knowledge Engineering, IEEE, 2005, pp 517-522

5. Brill E (1992) A simple rule-based part of speech tagger. In: Proceedings of the third conference on applied natural language processing, Association for Computational Linguistics, USA, ANLC '92, pp. 152–155. https://doi.org/10.3115/974499.974526

6. Campillos Llanos L, Sandoval AM, Guirao J. An automatic term extractor for biomedical terms in Spanish. In: Proceedings of the 5th international symposium on languages in biology and medicine (LBM 2013), 2013

7. Campos R, Mangaravite V, Pasquali A, Jorge A, Nunes C, Jatowt A. Yake! Keyword extraction from single documents using multiple local features. Inf Sci. 2020;509:257–89. https://doi.org/10.1016/j.ins.2019.09.013.

8. Conrado M, Pardo T, Rezende SO. A machine learning approach to automatic term extraction using a rich feature set. In: Proceedings of the 2013 NAACL HLT student research workshop, 2013; pp. 16–23

9. Cram D, Daille B. Terminology extraction with term variant detection. In: Proceedings of ACL-2016 system demonstrations, pp. 13–18, 2016

10. Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 181004805, 2018

11. Duari S, Bhatnagar V. Complex network based supervised keyword extractor. Expert Syst Appl. 2020;140:112876.

12. Foo J. Term extraction using machine learning. Linkoping: Linkoping University; 2009.

13. Frantzi K, Ananiadou S, Mima H. Automatic recognition of multi-word terms: the c-value/nc-value method. Int J Digit Lib. 2000;3(2):115-30.

14. Freifeld CC, Mandl KD, Reis BY, Brownstein JS. HealthMap: global infectious disease monitoring through automated classification and visualization of internet media reports. J Am Med Inf Assoc. 2008;15(2):150-7. https://doi.org/10.1197/jamia.M2544.

15. Habibi M, Popescu-Belis A. Keyword extraction and clustering for document recommendation in conversations. IEEE/ACM Trans Audio Speech Lang Process. 2015;23(4):746–59.

16. Ho SY, Tan S, Sze CC, Wong L, Goh WWB. What can Venn diagrams teach us about doing data science better? Int J Data Sci Anal. 2021;11(1):1–10.

17. Jacquemin C. Fastr: a unification-based front-end to automatic indexing. In: Intelligent multimedia information retrieval systems and management - Volume 1, Le centre de hautes études internationales d'informatique documentaire, Paris, FRA, RIAO '94, pp. 34–47, 1994

18. Ji L, Sum M, Lu Q, Li W, Chen Y. Chinese terminology extraction using window-based contextual information. International conference on intelligent text processing and computational linguistics. Berlin: Springer; 2007. p. 62–74.

19. Joung J, Kim K. Monitoring emerging technologies for technology planning using technical keyword based analysis from patent data. Technol Forecast Soc Change. 2017;114:281-92.

20. Kageura K, Umino B. Methods of automatic term recognition: a review. Terminol Int J Theoret Appl Issues Special Commun. 1996;3(2):259-89.

21. Lossio-Ventura JA, Jonquet C, Roche M, Teisseire M. Biomedical terminology extraction: a new combination of statistical and web mining approaches. In: JADT: Journées d'Analyse statistique des Données Textuelles, pp. 421–432, 2014a

22. Lossio-Ventura JA, Jonquet C, Roche M, Teisseire M. Biotex: a system for biomedical terminology extraction, ranking, and validation. In: Proceedings of the 2014 International Conference on Posters & Demonstrations Track - Volume 1272, CEUR-WS.org, Aachen, DEU, ISWC-PD'14, pp. 157–160, 2014b

23. Lossio-Ventura JA, Jonquet C, Roche M, Teisseire M. Yet another ranking function for automatic multiword term extraction. International conference on natural language processing. Berlin: Springer; 2014c. p. 52–64.

24. Lossio-Ventura JA, Jonquet C, Roche M, Teisseire M. Biomedical term extraction: overview and a new methodology. Inf Retriev J. 2016;19(1-2):59-99.

25. Madoff LC. ProMED-mail: an early warning system for emerging diseases. Clin Infect Dis. 2004;39(2):227-32.

26. Matsuo Y, Ishizuka M. Keyword extraction from a single document using word co-occurrence statistical information. Int J Artif Intell Tools. 2004;13(01):157-69.

27. Maynard D, Yankova M, Kourakis A, Kokossis A. Ontology-based information extraction for market monitoring and technology watch. In: ESWC workshop end user aspects of the semantic web, Heraklion, Crete, 2005

28. Meystre SM, Savova GK, Kipper-Schuler KC, Hurdle JF. Extracting information from textual documents in the electronic health record: a review of recent research. Yearbook Med Inf. 2008;17(01):128-44.

29. Mykhalovskiy E, Weir L. The Global Public Health Intelligence Network and early warning outbreak detection: a Canadian contribution to global public health. Can J Pub Health. 2006;97(1):42-4.

30. Neifar W, Hamon T, Zweigenbaum P, Khemakhem ME, Belguith LH. Adaptation of a term extractor to Arabic specialised texts: first experiments and limits. International conference on intelligent text processing and computational linguistics. Berlin: Springer; 2016. p. 242–53.

31. Oliver A, Vázquez M. Tbxtools: a free, fast and flexible tool for automatic terminology extraction. In: Proceedings of the international conference recent advances in natural language processing, pp. 473–479, 2015

32. Pais V, Ion R. Termeval 2020: Racai's automatic term extraction system. In: COMPUTERM, 2020

33. Paquet C, Coulombier D, Kaiser R, Ciotti M. Epidemic intelligence: a new framework for strengthening disease surveillance in Europe. Eurosurveillance. 2006;11(12):5–6. https://doi.org/10.2807/esm.11.12.00665-en.

34. Pazienza MT, Pennacchiotti M, Zanzotto FM. Terminology extraction: an analysis of linguistic and statistical approaches. Knowledge mining. Berlin: Springer; 2005. p. 255–79.

35. Ramos J, et al. Using tf-idf to determine word relevance in document queries. Proc First Instruct Conf Mach Learn. 2003;242:133-42.

36. Rigouts Terryn A, Hoste V, Drouin P, Lefeber E. Termeval 2020: Shared task on automatic term extraction using the annotated corpora for term extraction research (acter) dataset. In: 6th international workshop on computational terminology (COMPUTERM 2020), European Language Resources Association (ELRA), pp. 85–94, 2020

37. Shah H, Khan MU, Fränti P. H-rank: a keywords extraction method from web pages using POS tags. In: 2019 IEEE 17th international conference on industrial informatics (INDIN), IEEE, vol 1, pp. 264–269, 2019

38. Valentin S, Arsevska E, Falala S, de Goër J, Lancelot R, Mercier A, Rabatel J, Roche M. PADI-web: a multilingual event-based surveillance system for monitoring animal infectious diseases. Comput Electron Agric. 2020a;169:105163. https://doi.org/10.1016/j.compag.2019.105163.
39. Valentin S, Mercier A, Lancelot R, Roche M, Arsevska E. Monitoring online media reports for early detection of unknown diseases: Insight from a retrospective study of covid-19 emergence. Transboundary and emerging diseases. 2020b.

40. Wang LL, Lo K, Chandrasekhar Y, Reas R, Yang J, Eide D, Funk K, Kinney RM, Liu Z, Merrill W, Mooney P, Murdick DA, Rishi D, Sheehan J, Shen Z, Stilson B, Wade AD, Wang K, Wilhelm C, Xie B, Raymond DM, Weld DS, Etzioni O, Kohlmeier S. Cord-19: the covid-19 open research dataset. ArXiv abs/2004.10706. 2020a.

41. Wang R, Liu W, McDonald C. Featureless domain-specific term extraction with minimal labelled data. Proc Austral Lang Technol Assoc Workshop. 2016;2016:103–12.

42. Wang X, Zhang L, Klabjan D. Keyword-based topic modeling and keyword selection. arXiv preprint arXiv:200107866. 2020b.

43. Whissell JS, Clarke CL. Improving document clustering using okapi bm25 feature weighting. Inf Retriev. 2011;14(5):466–87.

44. Yao XM, GAN JH, Jian X. Concept extraction based on hybrid approach combined with semantic analysis. DEStech Transactions on Engineering and Technology Research, 2017.

**Publisher's Note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.