# Negation Detection on Mexican Spanish Tweets:
## 4 Experiments on Automatic Negation Identification  
We evaluated two methodologies for automatic negation identification in the T-MexNeg corpus. The first methodology consists of a baseline system that only uses a dictionary to identify negation cues. The second methodology consists of a system based on a Conditional Random Field (CRF) [49] algorithm. This algorithm is used for the structured prediction of labels by considering the labels of the previous and following words in a sentence. To compare the results of the automatic negation identification, we performed the same experiments on the SFU Review\({}_{\text{SP}}\)-NEG [18] corpus. In this way, we can compare the performance of methodologies in different variations of Spanish as well as in different domains. To handle the information in the T-MexNeg corpus, a prior treatment on the data was executed. We used the spaCy's ([[https://spacy.io/](https://spacy.io/)]([https://spacy.io/](https://spacy.io/)), accessed on 30 January 2021) library for Natural Language Processing in Python to tokenize each tweet into words or tokens. Then, using the same library, we labeled the Part of Speech (POS) tag of each token. Finally, the labels of the negation cue, event, and scope given in both corpora were translated into a BIO [50] tagging scheme. In the case of the SFU Review\({}_{\text{SP}}\)-NEG corpus, the text is already tokenized and labeled with POS tags, so such process was not necessary. We only arranged the data in the same way as in the T-MexNeg corpus. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Annotator Pair** & **1 \& 2** & **1 \& 3** & **2 \& 3** \\ \hline Percent of agreement & 96.477 & 96.065 & 97.569 \\ [PERSON]’s \(\kappa\) score & 0.890 & 0.875 & 0.926 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Agreement scores by each pair of annotators of the binary classification of negation. The dictionary-based system consists of a classifier that uses a list of the most commonly used negations in the Spanish language as the only search argument. This system validates each word in a sentence; if a given word appears in the dictionary, then this word is labeled as a negation cue. We considered five of the most common negative adverbs in Spanish language: _no_ (no), _nada_ (nothing), _nunca_ (never), _tampoco_ (neither), and _jamds_ (ever). The dictionary-based system obtained an F1-score of 0.91 on the T-MexNeg corpus, which indicates that, including only some of the most frequent labels, a high percentage of negation expressions is covered. In the next section, we cover in detail the obtained results. To improve the results achieved by the dictionary-based system, we developed a system based on a Conditional Random Field algorithm [49]; this is a framework modeled as a probabilistic graph to segment and label sequence data. CRF offers several advantages over other models, including the ability to relax independence assumptions and better relate training variables, in other words, it takes context into account by making connections between neighboring nodes (words). [PERSON] et al. [34] implemented a CRF-based system and presented results of the state-of-the-art for the SFU Review\({}_{\text{SP}}\)-NEG corpus where for cue identification the system outperforms state-of-the-art results, while for scope detection they provide the first experimental results. [PERSON] and [PERSON] [31] showed that CRF achieves high F1 scores and is accurate when identifying negation cues and its scope in clinical notes. The feature set used in our study is inspired by the work done by [PERSON] et al. [51] for the shared task of the Workshop NEGES 2018 [44]. They implemented a supervised approach combining CRF and several features for negation cue detection in Spanish for training the model. This approach ranked first in the official testing results of the Workshop. Our feature set used to train the model includes information of the word under observation, as well as the prior and posterior word; the word itself in upper- and lowercase is added to the feature list.