SlowGuess/ACL_2020_OCR huggingface dataset:
Nils Reimers and Iryna Gurevych. 2017. Optimal hyperparameters for deep LSTM-networks for sequence labeling tasks. arXiv preprint arXiv:1707.06799.
Sabine Rosenberg and Sabine Bergler. 2012. UConcordia: CLaC negation focus detection at *Sem 2012. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pages 294–300, Montréal, Canada. Association for Computational Linguistics.
Zahra Sarabi and Eduardo Blanco. 2016. Understanding negation in positive terms using syntactic dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1108-1118, Austin, Texas. Association for Computational Linguistics.
Zahra Sarabi and Eduardo Blanco. 2017. If no media were allowed inside the venue, was anybody allowed? In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 860-869, Valencia, Spain. Association for Computational Linguistics.
Longxiang Shen, Bowei Zou, Yu Hong, Guodong Zhou, Qiaoming Zhu, and AiTi Aw. 2019. Negative focus detection via contextual attention mechanism. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2251-2261, Hong Kong, China. Association for Computational Linguistics.
György Szarvas, Veronika Vincze, Richard Farkas, and János Csrik. 2008. The bioscope corpus: Annotation for negation, uncertainty and their scope in biomedical texts. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing, BioNLP '08, pages 38-45, Stroudsburg, PA, USA. Association for Computational Linguistics.
Ann Taylor, Mitchell Marcus, and Beatrice Santorini. 2003. The Penn Treebank: An Overview, pages 5-22. Springer Netherlands, Dordrecht.
Veronika Vincze, György Szarvas, Richard Farkas, György Móra, and János Csririk. 2008. The bioscope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9:S9 - S9.
Xin Wang, Yuanchao Liu, Chengjie Sun, Baoxun Wang, and Xiaolong Wang. 2015. Predicting polarities of tweets by composing word embeddings with long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1343-1353, Beijing, China. Association for Computational Linguistics.
8399
Michael Wiegand, Alexandra Balahur, Benjamin Roth, Dietrich Klakow, and Andres Montoyo. 2010. A survey on the role of negation in sentiment analysis. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, pages 60-68, Uppsala, Sweden. University of Antwerp.
Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 347-354, Vancouver, British Columbia, Canada. Association for Computational Linguistics.
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480-1489, San Diego, California. Association for Computational Linguistics.
Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2014. Negation focus identification with contextual discourse information. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 522-530, Baltimore, Maryland. Association for Computational Linguistics.
Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2015. Unsupervised negation focus identification with word-topic graph model. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1632-1636, Lisbon, Portugal. Association for Computational Linguistics.
In this section, we provide additional details on the neural models discussed in this paper.
A.1 Details on the Neural Architecture
The neural model shown in Figure 1 is our full model. It consists of a base network and additional components indicated with dotted shapes. The additional components incorporate information about the scope of the negation and context (previous and next sentence). In this section, we provide additional information about the input representation and the additional components.
Input Representation. As discussed in Section 4, we map each word token to its 1,024-dimensional pre-trained ELMo embedding (Peters et al., 2018). We do not update the ELMo embeddings during the training of the network.
Our baseline model leverages two additional embeddings for encoding positional information of
the negated verb as well as the semantic role labels of the input tokens. We extract semantic roles from the training, development and test sets in the CoNLL-2005 Shared Task (Carreras and Márquez, 2005). The embeddings indicating the negated verb and semantic role labels are trained from scratch along with all the other weights in the full network.
We employ an additional embedding to incorporate scope information into the network (Section 4). Like the two additional embeddings described above, the embeddings to indicate scope information are trained from scratch.
Figure 2 shows the construction of the input representation. The input sentence is "The carrier has not yet turned a profit."  $\mathrm{E}_{\mathrm{token}}$  (top) denotes the 1,024-dimensional ELMo embeddings of token. The other embedding vectors shown in Figure 2 are to indicate the position of the negated verb, semantic roles and the scope of the negation. We have two tags to indicate the negated verb ("Y" when the token is the negated verb and "N" otherwise), two tags to indicate the scope ("I_S" when the token is inside the scope and "O_S" otherwise), and 15 tags to indicate semantic roles (one per role label). All the embedding weights for each token are concatenated before feeding them into the first BiLSTM layer. The final input dimension per token is 1,474:1,024 from the word token embedding, 50 from the negated verb embedding, 200 from the semantic role embedding, and another 200 from the scope embedding.
Note that in the sample sentence shown in Figure 2, all tokens are inside the scope of the negation except the negation cue (negation cues are annotated as outside of the scope in CD-SCO (Morante and Daelemans, 2012)). The scope of a negation, however, can span over all the tokens or a small part of a sentence, or even be discontinuous (Morante and Daelemans, 2012). In the example sentence below, for example, the scope of the negation only spans over the last clause:
Mr./O_S Paul/O_S says/O_S he/O_S  
had/O_S not/O_S one/O_S but/O_S  
four/O_S advisers/O_S and/O_S that/O_S  
he/I_S never/O_S bid/I_S impulsively/I_S  
./O_S
BiLSTM-Attention Network for Context. To capture contextual information, we add two attention-based recurrent networks, one for the previous sentence and another one for the next sentence. These additional components are shown in
8400
Figure 2: Input representation of our neural model. The symbol  $\oplus$  denotes concatenation, not addition.
the left and right dotted rectangles in Figure 1. The previous and next sentences in the example shown in Figure 1 are "StatesWest operates four twin-engine turboprop aircraft, connecting 10 cities in California, Arizona and Nevada" and "The former president of FirstSouth F.A., a defunct Arkansas thrift, pleaded guilty to conspiring to inflate the institution's earnings by concealing worthless loan guarantees" respectively.
Like in the baseline model, we map each word of the adjacent sentences to its 1,024-dimensional ELMo embedding vector before feeding them into the recurrent network. Each network component consists of a 2-layer Bidirectional LSTM with 50 hidden units. A dropout rate of  $30\%$  is applied to the recurrent layers. We add an attention layer on top of the final BiLSTM layer. More specifically, we adopt the word-attention technique proposed by Yang et al. (2016). The attention weights from both networks are concatenated with the final hidden representation of the base 3-layer BiLSTM network (Figure 1). Subsequently, the additional network components are trained with the original BiLSTM network.
8401