# Labeled entities from social media data related to avian influenza disease

Camille Schaeffer, Roberto Interdonato, Renaud Lancelot, Mathieu Roche, Maguelonne Teisseire

► To cite this version:

Camille Schaeffer, Roberto Interdonato, Renaud Lancelot, Mathieu Roche, Maguelonne Teisseire. Labeled entities from social media data related to avian influenza disease. Data in Brief, 2022, 43, pp.108317. 10.1016/j.dib.2022.108317. hal-03733909

HAL Id: hal-03733909

https://hal.inrae.fr/hal-03733909

Submitted on 21 Jul 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

Data Article

Labeled entities from social media data
related to avian influenza disease

<figure><img src="image_3.png" /></figure>

Camille Schaeffer<sup>a,c</sup>, Roberto Interdonato<sup>b,c</sup>, Renaud Lancelot<sup>b,d</sup>, Mathieu Roche<sup>b,c,*</sup>, Maguelonne Teisseire<sup>a,c</sup>

ARTICLE INFO

Article history:
Received 24 March 2022
Revised 20 May 2022
Accepted 23 May 2022
Available online 27 May 2022

Keywords:
Text mining
Named entity recognition
Social networks
Epidemiology
Avian influenza

A B S T R A C T

This dataset is composed by spatial (e.g. location) and the-
matic (e.g. diseases, symptoms, virus) entities concerning
avian influenza in social media (textual) data in English. It
was created from three corpora: the first one includes 10
transcriptions of YouTube videos and 70 tweets manually an-
notated. The second corpus is composed by the same textual
data but automatically annotated with Named Entity Recog-
nition (NER) tools. These two corpora have been built to eval-
uate NER tools and apply them to a bigger corpus. The third
corpus is composed of 100 YouTube transcriptions automat-
ically annotated with NER tools. The aim of the annotation
task is to recognize spatial information such as the names of
the cities and epidemiological information such as the names
of the diseases. An annotation guideline is provided in order
to ensure a unified annotation and to help the annotators.
This dataset can be used to train or evaluate Natural Lan-
guage Processing (NLP) approaches such as specialized entity
recognition.

© 2022 The Authors. Published by Elsevier Inc.

This is an open access article under the CC BY license
(http://creativecommons.org/licenses/by/4.0/)

<footnote><sup>a</sup> INRAE, Montpellier F-34398, France</footnote>

<footnote><sup>b</sup> CIRAD, Montpellier F-34398, France</footnote>

<footnote><sup>c</sup> TETIS, Univ. Montpellier, AgroParisTech, CIRAD, CNRS, INRAE, Montpellier 34090, France</footnote>

<footnote><sup>d</sup> ASTRE, Univ. Montpellier, CIRAD, INRAE, Montpellier 34398, France</footnote>

<footnote>* Corresponding author at: TETIS, Univ. Montpellier, AgroParisTech, CIRAD, CNRS, INRAE, Montpellier 34090, France.
E-mail addresses: camille.schaeffer@teledetection.fr (C. Schaeffer), roberto.interdonato@cirad.fr (R. Interdonato), renaud.lancelot@cirad.fr (R. Lancelot), mathieu.roche@cirad.fr (M. Roche), maguelonne.teisseire@inrae.fr (M. Teisseire).</footnote>

<footnote>* Corresponding author at: TETIS, Univ. Montpellier, AgroParisTech, CIRAD, CNRS, INRAE, Montpellier 34090, France.
E-mail addresses: camille.schaeffer@teledetection.fr (C. Schaeffer), roberto.interdonato@cirad.fr (R. Interdonato), renaud.lancelot@cirad.fr (R. Lancelot), mathieu.roche@cirad.fr (M. Roche), maguelonne.teisseire@inrae.fr (M. Teisseire).</footnote>
<table><caption>**Specification Table**</caption><tr><td>Subject</td><td>Data Science: Data Mining and Statistical Analysis</td></tr><tr><td>Specific subject area</td><td>Identify epidemiological events on social networks</td></tr><tr><td>Type of data</td><td>Table</td></tr><tr><td>How data were acquired</td><td>Data are the result of a manual and automatic annotation of spatial and thematic entities. The corpora 1 and 2, including 80 transcriptions and tweets, were automatically extracted from the platforms YouTube (December 2020-January 2021) and Twitter (March-April 2021). The corpus 3, including 100 transcriptions, was automatically extracted from the YouTube platform (February 2019 to October 2021). These three corpora were preprocessed after the extraction. Preprocessing programs are included in the dataset. Manual annotations were performed in accordance to the annotation guideline available within this dataset<fnref n="1" />. For right privacy, the content of tweets is not available in the dataset, instead there is the id of the tweet. On the other side, transcription texts are included.</td></tr><tr><td>Data format</td><td>Raw and Standardized</td></tr><tr><td>Parameters for data collection</td><td>The collection of texts respects two conditions: the texts have to be written in English and the term 'avian influenza' must be present in the text body for the tweets and in their title for the YouTube videos (i.e. primary data sources). The textual transcriptions collected from YouTube have been normalized by adding punctuation and capitalization, and by correcting transcription and lexical errors for specific terms (e.g. disease names). For Twitter data, non-Unicode characters (emoticons, etc.) have been removed, as well as correcting lexical errors of specific terms (i.e. secondary data sources).</td></tr><tr><td>Description of data collection</td><td>The dataset is constituted of 5 table files (i.e. 3 table files for YouTube transcriptions data and 2 table files for Twitter data). The table files describe each data through a set of features.</td></tr><tr><td>Data source location</td><td>The data are hosted on the INRAE Dataverse. The data were manually collected within the UMR TETIS, Univ. Montpellier, AgroParisTech, CIRAD, CNRS, INRAE, Montpellier, France in the context of the MOOD project<fnref n="2" />.</td></tr><tr><td>Data accessibility</td><td>Repository name: Data INRAE (Dataverse).<br>Data identification number: 10.15454/GR5EFS.<br>Direct URL to data: https://doi.org/10.15454/GR5EFS</td></tr></table>

**Value of the Data**

* This dataset contributes to the available resources for Natural Language Processing (NLP) on specialized domains and more precisely in the field of epidemiology.

* This dataset is useful for computer scientists for NLP and data mining tasks.

* This dataset can be used for evaluation or training purposes for the entity recognition task.

* The annotators have identified a variety of entities (e.g. diseases, symptoms, virus, hosts). These entities are relevant to recognize epidemiological information.

**1. Data Description**

After an extraction process from YouTube and Twitter (i.e. primary data sources), the dataset is constituted of five table data files (.tab) normalized (i.e. secondary data sources) and annotated (i.e. final data sources). One annotation guide (.pdf) details the instruction to the annotators as well as the choices that were made while designing the study.

The python codes to reproduce the transformation of the documents for the annotation step is available on github.<fnref n="3" />

<footnote n="1">¹ https://doi.org/10.15454/GR5EFS</footnote>

<footnote n="2">² https://mood-h2020.eu/</footnote>

<footnote n="3">³ DOI:10.5281/zenodo.6373107 https://github.com/interdonatos/epidemiological social media data collection</footnote>
The five data files are distributed as follows:

## 1.1. Manual annotation of spatial and thematic entities - Corpus 1 (small)

* A table data file containing the YouTube transcription data, manually annotated, from corpus 1, named corpus 1Y (datafile_1Y_manual annotation.tab);

* A table data file containing the Twitter data, manually annotated, from corpus 1, named corpus 1T (datafile_1T_manual annotation.tab).

## 1.2. Automatic annotation of spatial and thematic entities - Corpus 2 (small)

* A table data file containing the YouTube transcription data, automatically annotated, from corpus 2, named corpus 2Y (datafile_2Y_automatic annotation.tab);

* A table data file containing the Twitter data, automatically annotated, from corpus 2, named corpus 2Y (datafile_2T_automatic annotation.tab).

## 1.3. Automatic annotation of spatial and thematic entities - Corpus 3 (big)

* A table data file containing the transcriptions YouTube data annotated automatically from corpus 3, named corpus 3Y (datafile_3Y_automatic annotation.tab).

The files from the three corpora include data organized in tables. An example of the YouTube transcription data row is given in Table 1 and an example of the Twitter data row is reported in Table 2.

The data are described through a set of features from YouTube transcription data of the three corpora:

- id: the id of the data on the social media;

- publication_date: the date of publication of the data on the social media;

- raw_text: the raw text (no transformation) of the data;

<table><caption>**Table 1**
Example of a row in the transcriptions YouTube data files.</caption><thead><tr><th>Features</th><th>Values</th></tr></thead><tbody><tr><td><strong>id</strong></td><td>P5uhnL0Fu0l</td></tr><tr><td><strong>publication_date</strong></td><td>2021-02-09T14:21:34Z</td></tr><tr><td><strong>raw_text</strong></td><td>breaking news coming in bird flu hits delhi zoo [...]</td></tr><tr><td><strong>normalized_text</strong></td><td>Breaking news coming in Bird Flu, hits Delhi, Zoo [...]</td></tr><tr><td><strong>spatial_entity</strong></td><td>('Delhi', 'GPE'), ('bhopal', 'GPE')</td></tr><tr><td><strong>thematic_entity</strong></td><td>('bird flu', 'Disease or Syndrome'), ('flu', 'Disease or Syndrome'), ('coronavirus', 'Virus')</td></tr><tr><td><strong>Example of text with<br>recognized entities<br>(in bold)</strong></td><td>[...] My colleague is joining us with more details on the reports that are just coming in seven positive cases of bird flu [...] [...] the poultry markets of Delhi were reopened.</td></tr></tbody></table>

<table><caption>**Table 2**
Example of a row in the Twitter data files.</caption><thead><tr><th>Features</th><th>Values</th></tr></thead><tbody><tr><td><strong>id</strong></td><td>1377388750834040000</td></tr><tr><td><strong>publication_date</strong></td><td>1 April 2021</td></tr><tr><td><strong>spatial_entity</strong></td><td>(Europe', 'GPE')('Russia', 'GPE')('UK', 'GPE')</td></tr><tr><td><strong>thematic_entity</strong></td><td>('Avian influenza', 'Disease or Syndrome')('bird flu', 'Disease or Syndrome')</td></tr></tbody></table>
<table><caption>**Table 3**
Distribution of the data in the dataset files Corpus 1 (i.e. manual annotation).</caption><thead><tr><th>Target</th><th>Man. Ann. YTb<sup>a</sup><br>10 transcriptions</th><th>Man. Ann. Twi<sup>b</sup><br>70 tweets</th><th>all Man. Ann.<sup>c</sup><br>80 texts</th></tr></thead><tbody><tr><td><strong>SPATIAL</strong></td><td><strong>97</strong></td><td><strong>37</strong></td><td><strong>134</strong></td></tr><tr><td>- GPE</td><td>86</td><td>35</td><td>121</td></tr><tr><td>- LOC</td><td>11</td><td>0</td><td>11</td></tr><tr><td>- FAC</td><td>0</td><td>2</td><td>2</td></tr><tr><td><strong>THEMATIC</strong></td><td><strong>120</strong></td><td><strong>118</strong></td><td><strong>238</strong></td></tr><tr><td>- Disease or Syndrome</td><td>35</td><td>87</td><td>122</td></tr><tr><td>- Virus</td><td>62</td><td>30</td><td>92</td></tr><tr><td>- Sign or Symptom</td><td>23</td><td>1</td><td>24</td></tr><tr><td><strong>TOTAL</strong></td><td><strong>217</strong></td><td><strong>155</strong></td><td><strong>372</strong></td></tr></tbody></table>

- normalized_text: the normalized text (after applying transformations) of the data;

- spatial_entity: A list of spatial entities annotated, with their labels (GPE, LOC, FAC);

- thematic_entity: A list of annotated thematic entities, with their labels (Disease or Syndrome, Virus, Sign or Symptom).

The data are described through a set of features from Twitter data of both corpora:

- id: the id of the data on the social media;

- publication_date: the date of publication of the data on the social media;

- spatial_entity: A list of annotated spatial entities, with its label (GPE, LOC, FAC);

- thematic_entity: A list of annotated thematic entities, with its label (Disease or Syndrome, Virus, Sign or Symptom).

The annotation guideline (annotation_guide_spatial_thematic_entities.pdf) presents the instructions to the annotators. These instructions and the choices made are summarized in the next section. The annotation framework defines several tags to annotate the texts. Spatial concepts are annotated with three tags:

- GPE (Geopolitical entity) is used to annotate entities representing countries, cities, states etc;

- LOC (Non-GPE locations) is used to annotate entities representing mountain ranges, bodies of water, etc;

- FAC (Faculty) is used to annotate entities representing buildings, airports, highways, bridges, etc.

Thematic concepts are annotated with three tags:

- Disease or Syndrome is used to annotate entities representing a disease or a syndrome;

- Virus is used to annotate entities representing a virus;

- Sign or Symptom is used to annotate entities representing a sign or symptom.

Number and distribution of annotated information in the corpora are given in Tables 4 and 5.

## 2. Experimental Design, Materials and Methods

### 2.1. Acquisition and pre-processing

The corpora were obtained automatically from the platforms YouTube and Twitter, thanks to their dedicated APIs. The texts from the web were stored in.txt files, with the aim to obtain distinct files to annotate.

<footnote><sup>a</sup> Manual Annotation - Youtube.</footnote>

<footnote><sup>b</sup> Manual Annotation - Twitter.</footnote>

<footnote><sup>c</sup> Manual Annotation - Total.</footnote>
<table><caption>**Table 4**
Distribution of the data in the dataset files Corpus 2 (i.e. automatic annotation).</caption><thead><tr><th>Target</th><th>Auto. Ann. YTB<sup>a</sup><br>10 transcriptions</th><th>Auto. Ann. Twi<sup>b</sup><br>70 tweets</th><th>all Auto. Ann<sup>c</sup><br>80 texts</th></tr></thead><tbody><tr><td><strong>SPATIAL</strong></td><td><strong>78</strong></td><td><strong>53</strong></td><td><strong>131</strong></td></tr><tr><td>- GPE</td><td>70</td><td>49</td><td>119</td></tr><tr><td>- LOC</td><td>5</td><td>3</td><td>8</td></tr><tr><td>- FAC</td><td>3</td><td>1</td><td>4</td></tr><tr><td><strong>THEMATIC</strong></td><td><strong>73</strong></td><td><strong>54</strong></td><td><strong>127</strong></td></tr><tr><td>- Disease or Syndrome</td><td>43</td><td>49</td><td>92</td></tr><tr><td>- Virus</td><td>20</td><td>5</td><td>25</td></tr><tr><td>- Sign or Symptom</td><td>10</td><td>0</td><td>10</td></tr><tr><td><strong>TOTAL</strong></td><td><strong>151</strong></td><td><strong>107</strong></td><td><strong>258</strong></td></tr></tbody></table>

<table><caption>**Table 5**
Distribution of the data in the dataset files Corpus 3 - larger corpus and only automatic annotation.</caption><thead><tr><th>Target</th><th>Automatic Annotation - YouTube - 100 transcriptions - 605 567 words</th></tr></thead><tbody><tr><td><strong>SPATIAL</strong></td><td><strong>1926</strong></td></tr><tr><td>- GPE</td><td>1535</td></tr><tr><td>- LOC</td><td>369</td></tr><tr><td>- FAC</td><td>22</td></tr><tr><td><strong>THEMATIC</strong></td><td><strong>3028</strong></td></tr><tr><td>- Disease or Syndrome</td><td>1407</td></tr><tr><td>- Virus</td><td>1429</td></tr><tr><td>- Sign or Symptom</td><td>192</td></tr><tr><td><strong>TOTAL</strong></td><td><strong>4954</strong></td></tr></tbody></table>

The pre-processing consists of four main steps applied on the raw corpora, detailed below. First, in order to optimize the recognition of entities, the raw text of these files has been normalized with the automatic addition of punctuation by using the python library `punctuator`<fnref n="4" /> and with the automatic addition of capital letters, by using the POS tagging provided by the python library `Spacy`<fnref n="5" />.

Then, the text has been cleaned by deleting non Unicode symbols and eventual noise originated by the transcription process. As a final normalization step of the raw text, a correction of specific terms (e.g. disease names) is applied by using regular expressions [1].

After the normalization task, the manual and automatic annotation of the spatial and thematic entities can be performed on the text.

## 2.2. Manual data labeling

In order to have a unified annotation of the spatial and thematic entities, a guideline was created. This annotation guide was written by a specialist in NLP. Three other persons (specialists in epidemiology) validated and adjusted these choices. By following this guide, one person (NLP specialist) annotated manually the spatial entities on the raw text data of 70 tweets and 10 transcriptions. In the same way, one person (data scientist specialized in epidemiology) manually annotated the thematic entities on the same data. This process results in the corpus 1 (1Y and 1T) and represents the ground truth data.

<footnote>a Automatic Annotation - Youtube.</footnote>

<footnote>b Automatic Annotation - Twitter.</footnote>

<footnote>c Automatic Annotation - Total.</footnote>

<footnote n="4">⁴ https://pypi.org/project/punctuator/</footnote>

<footnote n="5">⁵ https://spacy.io</footnote>
<table><caption>**Table 6**
Evaluation of the automatic recognition of spatial and thematic entities on corpus 1</caption><thead><tr><th></th><th>Precision</th><th>Recall</th><th>F-Measure</th></tr></thead><tbody><tr><td><strong>SPATIAL</strong></td><td><strong>0.7</strong></td><td><strong>0.6</strong></td><td><strong>0.6</strong></td></tr><tr><td>- GPE</td><td>0.7</td><td>0.6</td><td>0.6</td></tr><tr><td>- LOC</td><td>0.6</td><td>0.3</td><td>0.4</td></tr><tr><td>- FAC</td><td>0</td><td>0</td><td>0</td></tr><tr><td><strong>THEMATIC</strong></td><td><strong>0.6</strong></td><td><strong>0.2</strong></td><td><strong>0.3</strong></td></tr><tr><td>- Disease or Syndrome</td><td>0.5</td><td>0.3</td><td>0.4</td></tr><tr><td>- Virus</td><td>0.6</td><td>0.1</td><td>0.2</td></tr><tr><td>- Sign or Symptom</td><td>0.5</td><td>0.3</td><td>0.4</td></tr></tbody></table>

## 2.3. Automatic data labeling

In parallel, the spatial and thematic entities were automatically annotated on the same raw text data (70 tweets and 10 transcriptions) with NER tools. We applied SpaCy [2] on the data to recognize the spatial entities and SciSpacy [3] combined with UMLS [4] in order to recognize the thematic entities. The result of this process is the corpus 2 (2T and 2Y).

Once these two annotations (manual and automatic) were performed, we evaluated the automatic recognition of the spatial and thematic entities on corpus 2, by comparing the annotation results of the corpus 1 and the corpus 2, and measured the evaluation with three metrics: Precision, Recall and F-Measure.

The evaluation is presented in Table 6. The Precision value of spatial and thematic entity recognition is quite good, with a result of 0.7 and 0.6 respectively. The Recall for automatic recognition of thematic entities is low (0.2), the tool does not recognize many thematic entities that are not included in the thesaurus associated with SciSpacy.

We then apply the processes of Acquisition and pre-processing and Automatic data labeling on a larger corpus (corpus 3) called 3Y (100 transcriptions). This enables to check the scalability of the proposed process.

All text related information such as copyright, texts, authors list or references were discarded.

The dataset is composed of the extracted annotations. The extraction program, available in the dataset, preserves the text and adds the textual entities and their annotated categories at the end of the text.

## Ethics Statement

For Twitter data, we keep the message identifiers and the content of message is not stored in accordance to Twitter's terms of use. The content of YouTube data (i.e. transcriptions) is anonymized by removing the user names and the names of person in transcripts (using SpaCy for name recognition).

No conflict of interest exists in this submission. The authors declare that the work described in this paper is original and not under consideration for publication elsewhere, in whole or in part. Its publication is approved by all the authors listed.

## Declaration of Competing Interest

The authors declare that they have no financial or personal interests that could influence the work reported in this paper.
# Data Availability

Labeled Entities from Social Media Data Related to Avian Influenza Disease (Original data) (Dataverse).

# CRediT Author Statement

**Camille Schaeffer:** Methodology, Software, Investigation, Resources, Data curation, Writing – original draft; **Roberto Interdonato:** Methodology, Writing – review & editing; **Renaud Lancelot:** Methodology, Writing – review & editing; **Mathieu Roche:** Methodology, Writing – review & editing; **Maguelonne Teisseire:** Methodology, Writing – review & editing.

# Acknowledgments

We thank Augustin Borderon for his help in annotating the thematic entities of tweets and transcripts, and to Bahdja Boudoua and Elena Arsevska for assistance and adjustment of the definitions and choice for thematic entity annotation.

This study was partially funded by EU grant 874850 MOOD and is catalogued as MOOD033. The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of the European Commission.

# References

[1] A.K. Nazir, U. Qamar, I. Zafar, A. Shaheen, A. Fatima, B. Maqbool, Automated misspelling detection and correction in clinical free-text records, IEEE, 2018, doi:10.1109/icaibd.2018.8396209.

[2] M. Honnibal, I. Montani, spaCy 2: natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing, 2017. To appear.

[3] M. Neumann, D. King, I. Beltagy, W. Ammar, ScispaCy: fast and robust models for biomedical natural language processing, in: Proceedings of the 18th BioNLP Workshop and Shared Task, Association for Computational Linguistics, Florence, Italy, 2019, pp. 319–327, doi:10.18653/v1/W19-5034. https://aclanthology.org/W19-5034.

[4] O. Bodenreider, The unified medical language system (UMLS): integrating biomedical terminology, Nucleic Acids Res. 32 (90001) (2004) 267D–270, doi:10.1093/nar/gkh061.