Don’t Eclipse Your Arts Due to Small Discrepancies: Boundary Repositioning with a Pointer Network for Aspect Extraction (2020)

Abstract: The current aspect extraction methods suffer from boundary errors. In general, these errors lead to a relatively minor difference between the extracted aspects and the ground-truth. However, they hurt the performance severely. In this paper, we propose to utilize a pointer network for repositioning the boundaries. Recycling mechanism is used, which enables the training data to be collected without manual intervention. We conduct the experiments on the benchmark datasets SE14 of laptop and SE14-16 of restaurant. Experimental results show that our method achieves substantial improvements over the baseline, and outperforms state-of-the-art methods.

## introduction aspect extraction @xcite ) is a crucial task in the field of real-world aspect-oriented sentiment analysis, where an aspect stands for a sequence of tokens which adhere to a specific sentiment word, in general, serving as the target on which people express their views. for example, the tokens "twist on pizza" is the aspect of the opinion "healthy" in 1). in this paper, we concentrate on the study of aspect extraction conditioned on the unawareness of sentiment words. ## pointer network based boundary repositioning we train a pointer network to predict the start and end positions of the correct aspect. what we feed into the network include a candidate aspect and the sentence which contains the candidate (herein called source sentence). the candidate may be a boundary-misspecified aspect, truly-correct aspect or other text span. the network outputs two words w s and w e , one of which is predicted to be the start position, the other the end: it is noteworthy that, during testing, the status (boundary-misspecified, truly-correct or other) of the candidate aspect is assumed to be unknown. this is derived from the consideration of the practical situation in which the status of the pre-extracted aspect is unforeseeable. encoding assume @xmath0, ..., w n } represents the candidate aspect, where w c @xmath1, position and segment embeddings of the i-th token in c. the source sentence is represented in the same way and denoted by @xmath2,..., w m }. we concatenate c and u to construct the input representation: decoding due to the use of the multi-head selfattention mechanism @xcite , bert is able to perceive and more heavily weight the attentive words in the source sentence u , according to the information in the candidate aspect c, and vice versa. this property allows the attentionworthy words out of c to be salvaged and meanwhile enables the attention-unworthy words in c to be laid aside. on the other hand, a trainable decoder tends to learn the consistency between the ground-truth aspect and the attentive words. therefore, we suppose that the decoder is able to leave the boundaries of c unchanged if c aligns with the ground-truth aspect, otherwise redefine the boundaries in u in terms of the attentive words. following the practice in prior research @xcite , we decode the representation h i with a linear layer and the softmax function, where @xmath3: training our goal is to assign higher probabilities to the start and end positions ŵs and ŵe for all the ground-truth aspects in the training set. therefore, we measure loss by calculating the average negative log-likelihood for all pairs of ŵs and ŵe : ## bilstm-crf based pre-extraction we use the pointer network to post-process the pre-extracted aspects (which are referred to the candidate aspects in section 2). in our experiments, we employ a bilstm-crf model to obtain the candidate aspects. in this case, we solve aspect pre-extraction as a sequence labeling task. bio labeling space @xmath4, i, o} @xcite is specified as the output for each token in the source sentence, in which b, i and o respectively signal the beginning of an aspect, inside of an aspect and non-aspect word. first of all, we represent the tokens in the source sentence using glove embeddings @xcite . on the basis, we use a bidirectional recurrent neural network with long-short term memory (bilstm for short) @xcite to encode each token, so as to obtain the initial hidden state vector h lstm i . self-attention mechanism @xcite is utilized for the resolution of long-distance dependency, by which we obtain the attention-weighted hidden state h att i . we concatenate h lstm i and h att i to produce the final feature vector for the i-th token: ĥi =h lstm i ⊕ h att i . conditioned on the feature vector ĥi emitted by bilstm with attention, we estimate the emission probabilities that the i-th token may serve as b, i and o respectively. the fully-connected dense layer is used to map ĥi to the bio labeling space: p i (bio) = f den ( ĥi ). over the emission probabilities of all the tokens in the source sentence, we utilized a linear-chain conditional random field (crf) @xcite to predict the optimum label sequence of bio. eventually, the tokens labeled with b and i will be taken as the aspects. we train the extractor by maximizing the loglikelihood of sequence labeling @xcite : ## recycling mechanism the extractor can be trained on the benchmark datasets provided by the semeval tasks @xcite . however, it is impractical to separately train the positioner because there is a lack of boundary-misspecified negative examples. to solve the problem, we recycle the negative examples occurring during the training of the extractor. we define a negative example to be a text span which partially overlaps with the ground-truth aspect. the text spans which are completely inconsistent with the ground-truth are not considered. for example, "fresh ingrediants" in 3) is an eligible negative example, but "super tasty" is ineligible. ## conclusion our experimental results demonstrate that boundary repositioning can be used as a simple and robust post-processing method to improve aspect extraction. our findings reveal that illustrative aspects in scientific literature are generally long-winded. extracting these aspects suffers more severely from boundary errors. in the future, we will develop a syntax-based multi-scale graph convolutional network to deal with both short and long aspects.