Word-level Prefix/Suffix Sense Detection: A Case Study on Negation Sense with Few-shot Learning (2023)

Abstract: Morphological analysis is an important research issue in the field of natural language processing. In this study, we propose a context-free morphological analysis task, namely word-level prefix/suffix sense detection, which deals with the ambiguity of sense expressed by prefix/suffix. To research this novel task, we first annotate a corpus with prefixes/suffixes expressing negation (e.g., il-, un-, -less) and then propose a novel few-shot learning approach that applies an input-augmentation prompt to a token-replaced detection pre-training model. Empirical studies demonstrate the effectiveness of the proposed approach to word-level prefix/suffix negation sense detection.

## introduction morphological analysis mainly refers to processing a word into a lemma (root) and a well-defined morphological tag @xcite @xcite @xcite . for instance, through morphological analysis, the word "unhappy" will be divided into a lemma "happy" and a negation sense prefix tag "un-". morphological analysis has played an important role in natural language processing (nlp) and it has been applied to many downstream tasks such as spelling checking @xcite @xcite and machine translation @xcite @xcite . one major challenge in morphological analysis is that prefixes/suffixes are sometimes ambiguous. for instance, in english, the prefix "un-" often means a meaning "not", i.e., a negation sense. however, not all words with the prefix "un-" have a negation sense, such as "unanimous" and "unpick". besides, the substring "un-" sometimes does not appear as a prefix in some words, such as "universe" and "unique". in this study, we directly address the above challenge by proposing a novel morphological analysis task, namely word-level prefix/suffix negation sense detection, which aims to detect whether a substring in a word is a prefix/suffix and meanwhile takes a specific pre-defined morphological sense. as a preliminary study, we focus on negative prefixes/suffixes. in many languages, one way to make a negative expression is to add a negative prefix/suffix to a word. for instance, in english, il-, im-, un-, and -less are some popular negative prefixes/suffixes. one straightforward approach to prefix/suffix negation sense detection is to build a dictionary that covers all words with the prefixes/suffixes expressing such a sense. however, this is unrealistic because there are always many newly-emerging words due to non-standard language usage or incorrect spelling in some informal texts like twitter. therefore, we address the task of word-level prefix/suffix negation sense detection in a computational way. specifically, to further reduce the annotation cost, we propose a few-shot learning approach by employing the token-replaced detection model as our basic prompt-learning model due to its excellent performance in few-shot learning @xcite . furthermore, we propose a novel prompt, namely input-augmentation prompt, which relies only on the input word. as illustrated in fig. 1(c ), for the input word is "unhappy", the prompt, " unhappy it is not happy", is used to predict whether the word "not" is original or replaced so as to determine whether the input word is a negation word or not, where the substring "happy" is generated by removing the potential prefix (i.e., un-) from the input word. the design of our input-augmentation prompt can avoid one major shortcoming of existing few-shot learning approaches, i.e., the selection of labels (e.g., two labels, "positive" and "negative" in fig. 1a ) or the selection of label description words (e.g., "negative positive" in fig. 1b ) has a big impact on learner performance @xcite @xcite . moreover, our empirical studies also demonstrate that our approach achieves much better performances than the existing few-shot learning approaches. ## related work morphological analysis aims to learn about the morphological structure of a given word form, and in general, there are four specific tasks: morphological tagging (i.e., assigning some pre-defined morphological tags to a word in a sentence) @xcite @xcite @xcite , lemmatization (i.e., converting a word in a sentence into the normalized form) @xcite @xcite @xcite , morphological segmentation (i.e., judging whether the substring in a word could be segmented as a prefix/suffix) @xcite @xcite , and morphological disambiguation (i.e., assigning a correct morphological segmentation to a word by leveraging the context) @xcite @xcite . compared to the above tasks, our work has at least three different aspects. first, our task is a combination of morphological tagging and morphological segmentation. second, our task is word-level, i.e., the input contains only a single word without context, which leads to the inapplicability of previous approaches based on contextual information. third, we propose a novel few-shot learning approach to our task. to the best of our knowledge, this is the first attempt of studying fewshot learning in morphological analysis. ## corpus generation we use six prefixes, i.e., un-, im-, in-, il-, irand dis-as negation prefixes and two suffixes, i.e., -less and -f ree as negation suffixes to collect words from two resources, i.e., the ninth edition of oxf ord advanced learner ′ s dictionary @xcite and 1.6 million english tweeter data collected by @xcite . in summary, we obtain 2,717 and 6,671 words with negation prefixes/suffixes from the oxford dictionary and tweeter data, respectively. then, we randomly select 3,000 words and annotates such words as our corpus. specifically, we assign two annotators to annotate each word into two categories, i.e., positive and negative. the kappa consistency check value of the human annotation is 0.87. moreover, for words with different sense annotations, we assign another annotator to make a final decision. ## methodology problem statement: the prefix/suffix negation sense detection task can be formulated as follows. let d @xmath0, y} be labeled data, where w is the input word and y is a label in {positive, negative}. our approach aims to provide a few-shot learner for such a detection task. approach overview: as shown in figure 1(c ), a prompt-based learner, which is based on a pretrained token-replaced detection model and an input-augmentation prompt, is built for the prefix/suffix negation sense detection task. the goal of a pre-trained token-replaced detection model (e.g., electra) is to predict whether a token in the input string is replaced or not. ## conclusion in this study, we propose a novel word-level morphological analysis task, namely prefix/suffix sense detection, and make a case study on negation sense. we provide an annotated corpus for the prefix/suffix negation sense detection, and then propose a novel few-shot learning approach, which uses an input-augmentation prompt and a pretrained token-replaced detection model to effectively make the negation sense detection. empirical studies show that our approach performs much better than other approaches in the few-shot scenario, such as using only 16 training samples.