YNU - HPCC at S em E val-2020 Task 10: Using a Multi-granularity Ordinal Classification of the B i LSTM Model for Emphasis Selection (2020)

Abstract: In this study, we propose a multi-granularity ordinal classification method to address the problem of emphasis selection. In detail, the word embedding is learned from Embeddings from Language Model (ELMO) to extract feature vector representation. Then, the ordinal classifica-tions are implemented on four different multi-granularities to approximate the continuous em-phasize values. Comparative experiments were conducted to compare the model with baseline in which the problem is transformed to label distribution problem.

## introduction short texts have a great impact on visual communication. they are usually designed to grab a viewer's attention and convey a message efficiently. for text, word emphasis is used to capture the intent better, removing the ambiguity that may exist in some plain texts. word emphasis can clarify or even change the meaning of a sentence by drawing attention to some specific information, and it can be done with colors, backgrounds, fonts, italics, or boldface. the purpose of this task is to design automatic methods for emphasis selection, i.e., choosing candidates for emphasis in short written text, to enable automated design assistance in authoring @xcite . in the task definition, given a sequence of words or tokens, there is a subset of words that are good candidates to emphasize, and the word probability represents the degree of emphasis, which needs to be predicted. in a previous work, a rule-based approach is used @xcite . later, text-based features such as part-of-speech (pos), information content, position in the sentence and other information was adopted @xcite . some methods have been proposed for predicting emphasized words for expressive text-to-speech (tts) based on a deep neural network (dnn) @xcite . to address the multiple annotators problem, a majority voting ensemble has been used to transform the problem into single-label learning @xcite . @xcite suggest that the task should be transformed into label distribution learning (ldl). the main difference between such a method and previous works is that the label is not a single dispersed value, but rather a continuous label distribution. therefore, the model can fit more accurately to a real label instead of having the distortion of transforming a probability into a single label. the essence of the problem is to use a sequence labeling model to predict a probability value for each word. the existing methods mainly applied either a softmax or conditional random field (crf) @xcite layer to predict labels, which cannot be used in this task. therefore, using a linear decoder to output continuous linear values or using sigmoid output nonlinear probability values as the result have been considered, but the performance was not suitable. in this paper, we propose a multi-granularity ordinal classification method for the task. this involves an ensemble of four different models with different levels of granularity, including binary classification, ternary classification, quaternary classification, and quintuple classification. specifically, it divides the probability between (0,1) into different parts as category labels. for the binary classification, the probability (0,1) is divided into two categories, i.e., (0-0.5) and (0.5-1). for the ternary classification, the probability (0,1) is divided into three categories, i.e., (0-0.33), (0.33-66), and (0.66-1). for the quaternary classification, the probability (0,1) is divided into four categories, i.e., (0-0.25), (0.25-0.5), (0.5-0.75), and (0.75-1). the quintuple classification divides (0,1) into five categories, i.e., (0-0.2), (0.2-0.4), (0.4-0.6), (0.6-0.8), and (0.8-1). the reason why we use multi-granularity ordinal classification is that binary classification ignores the differences between word emphasis probability. for example, if two words' emphasis probabilities are 0.3 and 0.4, they belong to the same class for binary classification, while they belong to different classes for ternary classification. the finer the granularity of the division is, the more the model can learn the difference between key words. however, there is still a balance between granularity and the number of classifications, that is, the performance of multi-granularity classification will decrease as the granularity increases. this approach can use different granularities of information to improve performance. the rest of the paper is organized as follows. in section 2, we describe the multi-granularity ordinal classification of the bidirectional long short-term memory (bilstm) model in detail. the comparative experimental results are presented in section 3. conclusions are drawn in section 4. ## multi-granularity ordinal classification of bilstm model we use an ensemble of multi-granularity ordinal classification of the bilstm model to learn emphasis patterns. figure 1 shows the overall architecture of the proposed model. first, we fine-grain words' emphasis probabilities, and the granularity labels (0, 1, 2, 3, 4) denote the emphasis degrees from high to low. after we load the test into the word embedding layer, a bilstm is used to obtain word annotations that summarize the information from both directions and to learn more abstract features. a fully connected softmax layer is used to output a probability distribution over all classes from the bilstm output, which can be weighted into a single-value probability. finally, we use a voting ensemble to obtain the final probability. ## experiments and evaluation experiments were conducted to evaluate the proposed model. we report the results of the official review. the details of the experiment are described as follows. ## conclusion in this paper, we describe a task system that we submitted to semeval-2020 for emphasis selection. we propose a multi-granularity ordinal classification of the bilstm model. in future work, we will attempt to generalize models with better capabilities.