A Joint Model for Aspect-Category Sentiment Analysis with Shared Sentiment Prediction Layer (2020)

Abstract: Aspect-category sentiment analysis (ACSA) aims to predict the aspect categories mentioned in texts and their corresponding sentiment polarities. Some joint models have been proposed to address this task. Given a text, these joint models detect all the aspect categories mentioned in the text and predict the sentiment polarities toward them at once. Although these joint models obtain promising performances, they train separate parameters for each aspect category and therefore suffer from data deficiency of some aspect categories. To solve this problem, we propose a novel joint model which contains a shared sentiment prediction layer. The shared sentiment prediction layer transfers sentiment knowledge between aspect categories and alleviates the problem caused by data deficiency. Experiments conducted on SemEval-2016 Datasets demonstrate the effectiveness of our model.

## introduction aspect-category sentiment analysis (acsa) is a subtask of aspect-based sentiment analysis (absa) @xcite @xcite . acsa aims to identify all the aspect categories mentioned in texts and their corresponding sentiment polarities. an aspect category (or simply aspect) is an entity e and attribute a pair, denoted by e#a. for example, in the text "the place is small and cramped but the food is fantastic.", the aspect categories mentioned in the text are ambience#general and food#quality, and their sentiment polarities are negative and negative, respectively. many methods have been proposed to address the acsa task. however, most existing methods @xcite @xcite @xcite @xcite divide the acsa task into two subtasks: aspect category detection (acd) which detects aspect categories in a text and sentiment classification (sc) which categorizes the sentiment polarities with respect to the detected aspect categories, and perform these two tasks separately. such two-stage approaches lead to error propagation, that is, errors caused by aspect category detection would affect sentiment classification. to avoid error propagation, previous studies @xcite @xcite have proposed some joint models, which jointly model the detection of aspect categories and the classification of their polarities. further more, given a text, these joint models detect all the aspect categories mentioned in the text and predict the sentiment polarities toward them at once. although these joint models obtain promising performances, they train separate parameters for each aspect category and therefore suffer from data deficiency of some aspect categories. for example, the english laptops domain dataset from semeval-2016 task 5: aspect-based sentiment analysis @xcite has a quarter of the aspect categories whose sample size are less than or equal to 2 (see to solve the problem caused by data deficiency mentioned above, we propose a novel joint model, which contains a shared sentiment prediction layer. our model is based on the observation that aspect category text polarity laptop#quality ...i was surprised at the overall quality and the price... positive laptop#price positive laptop#operation performance ...i was surprised with the performance and quality of this hp laptop... positive laptop#quality positive the sentiment expressions and their polarities of different aspect categories are transferable. for instance, in ## related work existing methods for aspect-category sentiment analysis (acsa) can be divided into two categories: two-stage methods and joint models. two-stage methods perform the acd task and the sc task separately. @xcite and @xcite perform the acd task. @xcite propose a semi-supervised word embedding algorithm to obtain word embeddings on a large set of reviews, which are then used to generate deeper and hybrid features to predict the aspect category. @xcite utilize topic attention to attend to different aspects of a given text. many methods @xcite @xcite @xcite @xcite @xcite @xcite have been proposed for the sc task. @xcite first propose aspect embedding (ae) and use an attention-based long short-term memory network (at-lstm) to generate aspect-specific text representations for sentiment classification based on aspect embedding. @xcite propose a hierarchical bidirectional lstm (h-lstm) to modeling the interdependencies of sentences in a review. @xcite propose a method named aspect fusion lstm (af-lstm) to model word-aspect relationships. @xcite propose a model, namely gated convolutional network with aspect embedding (gcae), which incorporates aspect information into the neural model by gating mechanisms. @xcite proposed new capsule networks to model the complicated relationship between aspects and contexts. all the two-stage methods have the problem of error propagation. joint models jointly model the detection of aspect categories and the classification of their polarities. only a few joint models @xcite @xcite have been proposed for acsa. schmitt et al. @xcite propose two joint models: end-to-end lstm and endto-end cnn, which produce all the aspect categories and their corresponding sentiment polarities at once. @xcite propose constrained attention networks (can), which extends at-lstm to multi-task settings and introduces orthogonal and sparse regularizations to constrain the attention weight allocation. as a result, the can achieves better sentiment classification performance. however, to train the can, we need to annotate the multi-aspect sentences with overlapping or nonoverlapping. @xcite propose the aspect-level sentiment capsules model (as-capsules), which utilizes the correlation between aspect category and sentiment through shared components including capsule proceedings of the 19th china national conference on computational linguistics, pages 1112-1121, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0 computational linguistics sh a re d p a ra m e te rs @xmath0 a o 1 m c e j k p t r y q j q y 8 t l i d z u p 4 q p 0 v + 5 t 3 p j 3 v 3 s b 0 d 3 o v k f i j e 2 l / 0 s 0 y / 6 t t t u g m c k p r 8 k m m r d o q o i 9 3 s x v x 1 m 3 n l 1 v j c k i i u 7 h p c u 7 y 0 8 p z n 0 2 t e b p 2 1 v t h x 9 9 0 p m l v 3 s t z u 7 y r w 9 k a 7 z / j n a f n o 6 p t v e 2 r 4 0 r t l b 9 1 e x v y x y h n 8 w q 1 x k c o b n m h e m q t n o 1 l q x q z m f 1 m n q q 5 z h f f l v h w a e l t k t @xmath1 a o 1 m c e j k p t r y q j q y 8 t l i d z u p 4 q p 0 v + 5 t 3 p j 3 v 3 s b 0 d 3 o v k f i j e 2 l / 0 s 0 y / 6 t t t u g m c k p r 8 k m m r d o q o i 9 3 s x v x 1 m 3 n l 1 v j c k i i u 7 h p c u 7 y 0 8 p z n 0 2 t e b p 2 1 v t h x 9 9 0 p m l v 3 s t z u 7 y r w 9 k a 7 z / j n a f n o 6 p t v e 2 r 4 0 r t l b 9 1 e x v y x y h n 8 w q 1 x k c o b n m h e m q t n o 1 l q x q z m f 1 m n q q 5 z h f f l v h w a e l t k t @xmath2 a o 1 m c e j k p t r y q j q y 8 t l i d z u p 4 q p 0 v + 5 t 3 p j 3 v 3 s b 0 d 3 o v k f i j e 2 l / 0 s 0 y / 6 t t t u g m c k p r 8 k m m r d o q o i 9 3 s x v x 1 m 3 n l 1 v j c k i i u 7 h p c u 7 y 0 8 p z n 0 2 t e b p 2 1 v t h x 9 9 0 p m l v 3 s t z u 7 y r w 9 k a 7 z / j n a f n o 6 p t v e 2 r 4 0 r t l b 9 1 e x v y x y h n 8 w q 1 x k c o b n m h e m q t n o 1 l q x q z m f 1 m n q q 5 z h f f l v h w a e l t k t @xmath3 a o 1 m c e j k p t r y q j q y 8 t l i d z u p 4 q p 0 v + 5 t 3 p j 3 v 3 s b 0 d 3 o v k f i j e 2 l / 0 s 0 y / 6 t t t u g m c k p r 8 k m m r d o q o i 9 3 s x v x 1 m 3 n l 1 v j c k i i u 7 h p c u 7 y 0 8 p z n 0 2 t e b p 2 1 v t h x 9 9 0 p m l v 3 s t z u 7 y r w 9 k a 7 z / j n a f n o 6 p t v e 2 r 4 0 r t l b 9 1 e x v y x y h n 8 w q 1 x k c o b n m h e m q t n o 1 l q x q z m f 1 m n q q 5 z h f f l v h w a e l t k t @xmath4 a q w b e 5 i e s 6 v 8 e 6 x j z m g n j k / h m 3 a a s e 3 m d m p d 5 4 c m i o p i p o o t 3 u k o v 6 4 6 6 5 y p t d c y y j 4 g b g w r y v u v k l z h h d w k c z i j a e e m r d u f b 0 t o g c w c p c r 0 m i r o e u i k z j f a i b u z z j d i 8 y i / p o 6 b d o 2 d j 2 m t p a d q b n r l s k 0 h p y 4 s 0 c e u j w v o 0 2 8 q z 4 6 z z 3 7 y h x l p f 7 z b + f u 4 v e a t w q e x f u n h m f 3 w 6 f o u + 9 k 0 n n g p k d a o r c 3 k x z h r f 3 9 z + u p u i h 5 q 4 j x s u f 4 q d o x z 3 2 t y a a w r x v f v m / m 1 k a l b v g z w 3 w 7 u + j q 3 y / t n o s d d y q b p o 1 t 3 z r r w c 5 q m u y g o b 2 k z 5 7 u e a x 6 i h t t 5 x e m q t n q 1 t 6 8 a 6 s + 4 / u 6 1 c r l n h t 2 u 9 f a c q l z s @xmath5 a q w b e 5 i e s 6 v 8 e 6 x j z m g n j k / h m 3 a a s e 3 m d m p d 5 4 c m i o p i p o o t 3 u k o v 6 4 6 6 5 y p t d c y y j 4 g b g w r y v u v k l z h h d w k c z i j a e e m r d u f b 0 t o g c w c p c r 0 m i r o e u i k z j f a i b u z z j d i 8 y i / p o 6 b d o 2 d j 2 m t p a d q b n r l s k 0 h p y 4 s 0 c e u j w v o 0 2 8 q z 4 6 z z 3 7 y h x l p f 7 z b + f u 4 v e a t w q e x f u n h m f 3 w 6 f o u + 9 k 0 n n g p k d a o r c 3 k x z h r f 3 9 z + u p u i h 5 q 4 j x s u f 4 q d o x z 3 2 t y a a w r x v f v m / m 1 k a l b v g z w 3 w 7 u + j q 3 y / t n o s d d y q b p o 1 t 3 z r r w c 5 q m u y g o b 2 k z 5 7 u e a x 6 i h t t 5 x e m q t n q 1 t 6 8 a 6 s + 4 / u 6 1 c r l n h t 2 u 9 f a c q l z s @xmath6 a q w b e 5 i e s 6 v 8 e 6 x j z m g n j k / h m 3 a a s e 3 m d m p d 5 4 c m i o p i p o o t 3 u k o v 6 4 6 6 5 y p t d c y y j 4 g b g w r y v u v k l z h h d w k c z i j a e e m r d u f b 0 t o g c w c p c r 0 m i r o e u i k z j f a i b u z z j d i 8 y i / p o 6 b d o 2 d j 2 m t p a d q b n r l s k 0 h p y 4 s 0 c e u j w v o 0 2 8 q z 4 6 z z 3 7 y h x l p f 7 z b + f u 4 v e a t w q e x f u n h m f 3 w 6 f o u + 9 k 0 n n g p k d a o r c 3 k x z h r f 3 9 z + u p u i h 5 q 4 j x s u f 4 q d o x z 3 2 t y a a w r x v f v m / m 1 k a l b v g z w 3 w 7 u + j q 3 y / t n o s d d y q b p o 1 t 3 z r r w c 5 q m u y g o b 2 k z 5 7 u e a x 6 i h t t 5 x e m q t n q 1 t 6 8 a 6 s + 4 / u 6 1 c r l n h t 2 u 9 f a c q l z s @xmath7 a q w b e 5 i e s 6 v 8 e 6 x j z m g n j k / h m 3 a a s e 3 m d m p d 5 4 c m i o p i p o o t 3 u k o v 6 4 6 6 5 y p t d c y y j 4 g b g w r y v u v k l z h h d w k c z i j a e e m r d u f b 0 t o g c w c p c r 0 m i r o e u i k z j f a i b u z z j d i 8 y i / p o 6 b d o 2 d j 2 m t p a d q b n r l s k 0 h p y 4 s 0 c e u j w v o 0 2 8 q z 4 6 z z 3 7 y h x l p f 7 z b + f u 4 v e a t w q e x f u n h m f 3 w 6 f o u + 9 k 0 n n g p k d a o r c 3 k x z h r f 3 9 z + u p u i h 5 q 4 j x s u f 4 q d o x z 3 2 t y a a w r x v f v m / m 1 k a l b v g z w 3 w 7 u + j q 3 y / t n o s d d y q b p o 1 t 3 z r r w c 5 q m u y g o b 2 k z 5 7 u e a x 6 i h t t 5 x e m q t n q 1 t 6 8 a 6 s + 4 / u 6 1 c r l n h t 2 u 9 f a c q l z s @xmath8, shared encoders, shared attentions and a shared recurrent neural network. these joint models train separate parameters for each aspect category, which results in that these models under-fit on the aspect categories with deficient samples. ## proposed model we first formulate the problem. there are n predefined aspect categories @xmath9, a 2 , • • • , a n } and m predefined sentiment polarities @xmath10, p 2 , • • • , p m } in the dataset. given a sentence or a review, denoted by @xmath11, w 2 , • • • , w n }, the task aims to predict the aspect categories and the corresponding sentiment polarities, i.e., aspect-sentiment pairs{< a j , p @xmath12, expressed in the text. the overall model architecture is illustrated in figure 1 , which contains six modules: embedding layer, bi-lstm layer, aspect attention layer, sentiment attention layer, aspect category prediction layer, and shared sentiment prediction layer. then, we display the details of each module and introduce the training objective function. ## conclusion in this work, we propose a novel joint model which contains a shared sentiment prediction layer. the shared sentiment prediction layer transfers sentiment knowledge between aspect categories and alleviates the problem caused by data deficiency. experiments conducted on four datasets from semeval-2016 task 5 demonstrate the effectiveness of our model. furture work could consider introducing extra component that prevents the shared sentiment prediction layer from transfering aspect-specific sentiment knowledge.