Aspect-Category Enhanced Learning with a Neural Coherence Model for Implicit Sentiment Analysis
Authors: Jin Cui
Fumiyo Fukumoto
Xinfeng Wang
Yoshimi Suzuki
Jiyi Li
Wanzeng Kong
Venue: EMNLP 2023 Findings (2023)
Keywords: Aspect-Based Sentiment Analysis
Implicit Sentiment
Coherence
TL;DR: We propose aspect-category enhanced learning with a neural coherence model (ELCoM) for the aspect-based sentiment analysis (ABSA) task.

Abstract: Aspect-based sentiment analysis (ABSA) has been widely studied since the explosive growth of social networking services. However, the recognition of implicit sentiments that do not contain obvious opinion words remains less explored. In this paper, we propose aspect-category enhanced learning with a neural coherence model (ELCoM). It captures document-level coherence by using contrastive learning, and sentence-level by a hypergraph to mine opinions from explicit sentences to aid implicit sentiment classification. To address the issue of sentences with different sentiment polarities in the same category, we perform cross-category enhancement to offset the impact of anomalous nodes in the hypergraph and obtain sentence representations with enhanced aspect-category. Extensive experiments on benchmark datasets show that the ELCoM achieves state-of-the-art performance. Our source codes and data are released at \url{https://github.com/cuijin-23/ELCoM}.

CopeeryCategory Enhanced Learning with a Neural Coherence Model for Implicit Sentiment Analysis
Jin Cui', Fumiyo Fukumoto', Xinfeng Wang', Yoshimi Suzuki',
Jiyi Li and Wanzeng Kong"
University of Yamanashi, Kofu, Japan
Hangzhou Dianzi University, Hangzhou, China (g22dtsa5, fukumoto, g22dtsa7, , jyl j@yamanashi. ac jp.
kongwanzengehdd. edu. cn
Abstract
The faod here rather good, but only you like to wait for it
ffood
APOOERQUALITY:
Fosttive
Explioh
NULI
SHERVIERGEEERAAA Negative
Impiit
Aspect-based sentiment analysis (ABSA) has been widely studied since the explosive growth of social networking services. However, the recognition implicit sentiments that do not contain obvious opinion words remains less explored. In this paper, we propose aspect- category enhanced learning with neural co- herence model (ELCoM). It captures document- level coherence by using contrastive learning, and sentence-level by hypergraph mine opinions from explicit sentences to aid implicit sentiment classification. To address the issue of sentences with different sentiment polarities in the same category, we perform cross-category enhancement to offset the impact anomalous nodes in the hypergraph and obtain sentence representations with enhanced aspect-category. Extensive experiments on benchmark datasets show that the ELCoM achieves state-of-the-au performance. Our source codes and data are re- leased https:// thub. com/cui cui jin-33/ ELCoM.
like <UOUDIIUAALIY>
the thases kinde disppoining
Explist
Tebooe!
<TOUDEQUALITY>
Explial
service varys from day- sometiness they're very nice, and sometimess not.
"seevise!
<SURYIUUKKINUURA
Negoiiv
Explicr
Yai The blecben, howeever, almost alwayy slew.
"kiichen! GSURVICUAGENERAD
Negaive
taplich
Aboo speeify I/ you
spicy- natker bland
if you dor
4/0GONQUALITY>
Negative
Implich
Figure I: A review from SemEval-2016- Words in an- gle brackets refer to aspect-categore The blue lines indicate that the review is coherent. The orange and green lines show that each "service" and "food" cate- gory points out its sentiment.
from dependency trees which results in insufficient representations capturing their contexts.
One feasible solution is to leverage one aspect of document quality: a document forms coher- ence if it is well-written and easy understand. Many coherence models such entity- -baeed ap- proaches (Jeon and Strube, 2022), and neural co- herence models (Nguyen and Joty, 2017; Moon et al., 2019) have been proposed and applied various NLP tasks. However, it is often the case that each sentence in review includes several dif- ferent aspect-categorics, and even the same aspect- category within a review expresses different senti- ment polarities. As illustrated in bold lines in Fig- ure 1, 81 has two different aspect-categories, "food" and "service," " with different polarities. Likewise, "food" in 81 is positive, while that of 85 shows negative polarity. Much of the existing coherence models obtained from the distribution of entities or adjacent sentences are inadequate and make it hard to predict an accurate polarity of implicit sentiment. Motivated by the issue mentioned above, we pro- pose an aspect-category enhanced learning with a neural coherence model (ELCoM) that leverages coherence information for ABSA, especially ben-
1 Introduction
Aspect-based sentiment analysis (ABSA) has been one of the major research topics in NLP since large volume of reviews has been accessible via social networking services. Much of the previous work on ABSA has focused on explicit sentiment (Yang and Zhao, 2022; Yan al., 2021; Chen et al., 2022), while implicit sentiment, in which obvi- ous sentiment polarity words do not appear in the sentence but convey sentiments, often appears in reviews. As illustrated in Figure 1, "The food here is rather good, but only if you like to wait for it." in $1, cannot be clearly identified as "negative" with respect to the aspect category "service." because it does include any opinion words related to it. Even less work on implicit sentiment mainly leverages intrasentential information to exploit ef- fective contexts, for example, syntactic information

---

eficial for implicit sentiment classification. On the one hand, observing from the SemEval-15 and 16 ABSA datasets, (i) more than 50% of the reviews had the same sentiment polarity regard- less of different aspect-categories, and (ii) more than 70% of reviews had the same sentiment po- larity if they had the same aspect-category. These observations indicate that (i) the reviewer's opin- ions are likely to be preserved throughout the re- view and (ii) the espect-categon is strong clue for capturing sentence-level coherence to aid implicit sentiment classification. We thus utilize document- level coherence to deal with (i) and exploit the re- lationships among sentences and aspect-categories by utilizing a hypergraph for (ii).
mance, while their models treat a sentence inde- pendently and ignore how sentences are connected as well as how the entire document is organized to convey information to the reader.
In the context of leveraging a whole document for sentiment analysis, Chen et al. (2020) assumed intra- and inter-aspect sentiment preferences clas- sify aspect sentiment. Their approach is similar to ours since they focused on aspect-categories to cap- ture sentiment polarity lendencies. However, their model only explored explicit sentiment classifica- tion. Cai et al. (2020) attempted implicit aspect- category detection and category-orientee sentiment classification by applying hierarchical graph con- volutional network. Their approach also sim- ilar to ours in that they consider document-level information. The difference is that we leverage contextual features by utilizing both document- and sentence-level coherence based on aspect- categories to learn more fine-grained contextual representation.
On the other hand, the rest of the 20~30% re- views include opposite polarities with each other even in the same category, while these reviews pre- serve coherence. To compensate for the dilemma of the side effect, we perform tross-categon en- hancements in the hypergraph. More specifically, we utilize a self-attention (SA) filtering to offset the impact of anomalous nodes, apply retrieval- -based attention (Rba) technique (Zhang et al., 2019) to learn the enhanced embedding of the aspect term, and finally obtain sentence representations with enhanced aspect-category.
2.2 Coherence Analysis
With the success of deep learning (DL) techniques, many authors have attempted to apply DL to learn features for coherence. One attempt is to model coherence as the relationship between adjacent sen- tences. This type includes the CNN (Nguyen and Joty, 2017), a hierarchical RNN (Nadeem and Os- tendorf, 2018), and an attention mechanism (Liao et al., 2021). Another attempt is coherence as whole document. This includes inter-sentence discourse relations (Moon et al., 2019) as well word- and document-level coherence (Farag and Yannakoudakis, 2019). Our approach lies across both attempts and provides a comprehension frame- work for sentiment coherence.
The main contributions of our work can be sum- marized as follows: (1) we propose an ELCoM that Iearns document-level coherence by using con- trastive learning and sentence-level by hypergraph for mining opinions to aid implicit sentiment clas- sification; (2) we propose cross-category enhance- ments on node embedding to offset the impact of anomalous nodes to correctly identify the sentiment the same categorics but have different polarities; and (3) extensive experiments on SemEval-2015 and 2016 show that our method achieves state-of- the-art performance.
Most of the attempts to apply neural coher- ence models to NLP tasks, such as text generation (Parveen al., 2016; Guan et al., 2021), summa- rization (Eva et al., 2019; Goyal et al., 2022), and text quality assessment (Farag et al., 2018; Mesgar and Strube, 2018), emphasize how to capture the global level of coherence in the text. Yang and Li (2021) proposed to exploit coherency sentiment representations to help implicit sentiment analy- sis. They focused on the local (word-level) aspect of sentiment coherency within the target sentence. Our EL.oM captures document-level coherence by using contrastive learning and sentence-level by hypergraph, and in this way, our model is sen-
2 Related Work
2.1 Implicit Sentiment Analysis
To date, there has been very little work on im- plicit sentiment analysis. To address this issue, Li et al. (2021a) proposed a supervised contrastive pre-training model that learns sentiment clues from large-scale noisy sentiment-annotttedd cor- pora. Wang al. (2022) established the causal representation of the implicit sentiment by identi- fying the causal effect between the sentence and sentiment. These attempts achieved better perfor-

---

sitive capturing both global and local patterns, even in a small size of the training data such as the SemEval sentiment dataset.
Specifically, we create an input sentence, 8; [SEP] d', [SEP] [CLS], for each aspect d', that appeared in the target sentence s; = {w,]ブ=1- Here, the input is padded with two special symbols, [SEP] and [CLS], which are the same as those of BERT. We apply XLNet to the input and obtain cach word embedding emy E Rdm and the aspect-based sen- tence embedding e, dm marked with [CLS], where dm is the dimension size. Likewise, given the imput document D = { ( and create an input document sequence, 81 [SEP] 81 [SEP] [CLS]. For the input sequence, we apply XLNet and obtain the document embedding ed E R. [xdm marked with [CLS] that contains both document- and sentence-level representations.
3 Task Definition
Let D {= be an input review consisting of the number of sentences, where the i-th sentence {w) consists of the number of n words. Let ag = (a also be set consisting of T aspects within 8;. Each d; consists of a pair of the aspect-category and term. Let c (positive, nega- tive, neutral] be a label set of sentiment polarities. The goal of the ABSA task is, for the given 1-th aspect in 8,, to predict the sentiment label l(a
we concatenate it
4 Approach
4.2 Coherence Modeling
The overall architecture of the ELCoM illus- trated in Figure 2. comprises three key steps: (I) representation learning with XLNet, (2) coherence modeling (CoM) to capture document-level and sentence-level coherence, and (3) cross-category enhancement to mitigate the influence of anoma- lous nodes.
Document-Level Coherence with Contrastive Learning. Following Jwalapuram et al. (2022), to learn robust coherence representations, the sentence ordering task by using contrastive learning. It enforces that the coherence score of the positive sample (original document) should be higher than that the negative sample (disorder document). Therefore, we disorder the original review to generate the number of B negative sam- ples by randomly shuffling the sentences within the document. For the results, we applied contrastive learning to align the coherent and incoherent rep- resentations. Let fe(es) be linear projection convert coherent document embedding e into co- herence scores. The margin-based contrastive loss is given by:
adopt
4.1 Representation Learning with XLNet
We utilized XLNet (Yang et al., 2019) the back- bone model obtain the sentence representation related to the target aspect and review docu- ment representation. The XLNet known to im- prove performance, especially for tasks involving a longer text sequence, e.g. text summarization, text classification, and text quality assessment. It is also utilized to model the coherence represen- tation (Jwalapuram et al., 2022), because it takes advantage of both the autoregressive model and the BERT (Devlin et al., 2019).
eo((ett)
La = - -loog(
efolest) + E E,,,)t
[olee )-+))
(2)
Formally, for a sequence of the length I, there are possible orders for autoregressive factoriza- tion. Let 2 be the set of total permutations of the length index sequence [1,2,... ... I]. 2, and Zer denote the r-th element and the first 7-1 elements of permutation z 2,. The objective of XLNet is given by:
where fo(ex*) indicates the coherence score of the positive sample, fo ... fo ( denote the scores of B negative samples, and is the margin. Sentence-Level Coherence by Hypergraph. Re- call that more than 70% reviews have the same sentiment polarity if they have the same aspect- category. This indicates that the aspect-category is beneficial for sentiment identification. We thus uti- lize hypergraphs to exploit the relationships among sentences and aspect-categories. The hypergraph is variant of the graph, where hyperedge (edge in hypergraph) connects any number of vertices. while in graph-based methods, e.g., graph convolu- tional networks (GCN), an edge connects only two vertices (Yu and Qin, 2019; Wang et al., 2023).
Σ log po (z (xace)
max Ex~Zi Zi
(1)
As such, the permutation language modeling optimization objective enables XLNett to effectively model the coherence across the document-level review.

---

Coherence Modeling
Cross-Categor Enhancement
Sentiment Analysis
Matching
9:
ABSA Loss
Contrastive
Loss
4:
Output
Query voctor
Self-Attention Filtering
Contrastive Learning
Sentence-Leve Coherence
Augmentation e
en
Cy
Retrieval-based
H
Attention
Hypergreph
(t)
dd
Aspect-based Representation
Graph
ed
Convolution
Disordered Coherence
Ordered Coherence
Syntactic Representation
Sentence Representation
Syntactical Adjaccncy
XLNet
XLNet
Matrix
conj
amod
advymod
nsubj
$1 [SEP] S [SEP] [CLS]
Nice ambience,
but highly overrated place
s [SEP] [SEP) [CLS]
JJ
NN
CC
RB
JJ
NN
Stanford Parser
Negative Samples
Disordering Sentences
Positive
Target ABSA Sentence
sample
Input
Belong to
Disorder Document
Original Document
Figure 2: The architecture of the ELCoM. It comprises representation learning with XLNet, coherence modeling, and cross-category enhancement, with the input data consisting of original reviews and the target ABSA sentences. The final output is obtained through sentiment polarity classification.
We define uspect-categories (C' in total) as hy- peredges, and sentences as nodes in the hypergraph H E R Each row in H is for sentence and each column is for the hyperedge of aspect- category. Hij 1 if vertex is connected by hyperedge j, and Hig multiplication of the document embedding O and the set of multi-hot vectors (HH. E k/x1) can be regarded as the slicing operation that selects the corresponding embeddings of the sentences in the same category. The representation of the sentences toward the same aspect, e₁, E R Xddm is obtained by:
sentence-level coherence modeling often leads to eтror propagation as it learns the information on both polarities from other sentences during training, To alleviate this issue, perform cross-cated crossccatggory enhancement on node embeddings in the hyper- graph which illustrated Figure 2. More pre- cisely, (1) we utilize a self-attention (SA) and re- duce the influences between anomalous nodes, and (2) we apply a retrieval attention technique, Rba, to obtain enhanced embedding of the aspect term.
otherwise. Note that the
eh HH T
ed-
Self-Attention (SA) Filtering. Assume that if the aspect-category of the target query sentence con- tains different sentiments from the same aspect other sentences, the SA weight should be small dilute its features. We use the SA mechanism the transformer, which given by:
4.3 Cross-Category Enhancement
We observed that 20~300% reviews contain differ- ent polaritics even the same category, while these reviews preserve coherence. The document- and

---

Imp.
Exp.
QK
Dataset
Train
Test
Train
Test
COR (%)
COC (%)
S = softmax
V
(4)
Vdk
RESTI5
108
103
1560
736
57,43
78.29
LAPI5
132
1837
936
53.11
82.89
where Q, K, and V refer to the query, key, and value matrices obtained by linear transformations of еb, respectively. The result is fed into feed- forward network, combined with layer normaliza- tion and residual connection. Each encoder layer takes the output of the previous layer as the input. This allows attention to be paid to all positions of the previous layer. The results are passed to an av- erage pooling, and we obtain the filtered sentence representation éj, E Rd Syntactic Representation Learning, We use the Stanford parser' to obtain a dependency tree of the input sentence and apply graph convolution operations learn the high-order correlations of the words. Given the syntactic adjacency matrix E R of the sentence and the embedding set nodes (words) (o)) [ew,1, ew,1, ..., e, the node representations are updated as follows:
RESTI6
211
2296
818
56.14
77.95
LAP16
136
2773
768
53.01
81.39
Table I: Statistics the ABSA dataset. COR and COC indicate the ratio of reviews in which sentiment polari- ties are consistent, and in which sentiment polarities in the same aspect-category are consistent, respectively.
4,4 Multi-Task Learning
We use the multi-task learning (MTL) framework to optimize both the ABSA task and the sentence ordering task. As for the ABSA task, sentence representations with an enhanced aspect-category are obtained by passed on to the linear transformation layer. Using the softmax function, we obtain probability score E c
[o,, e, The result is
p softmax SeitmaxxWWrt (W,e p bp),
(7)
n
e!"
Σ A j=1
1 w !-11
where W E Rl and by are the weight, and bias term, respectively. The task trained with the cross-entropy loss, denoted as follows:
(5)
= Re (1 (d: + b)),
where ej 11-1) Rdm denotes the j-th word repre- sentation obtained from the GCN layer, e, refers the i-th word representation of the current GCN layer, and d; = Aij is the degree the i-th token in the dependency tree. The weights w() and bias ((d) are trainable parameters.
Lea = -E -E(())oo(p)) log(p), IED
(8)
where 7 E TR/o] denotes the true label vector. Recall that we utilize
based contrastive
loss Lat to train the sentence ordering task. The final loss is given by:
We apply Rba to the output of the GCN. Specif- ically, we mask out the non-aspect words in the output of the GCN to obtain the masked representa- tion em The attention weights aj of word wj are given by:
L(mmitii (d(st)) Φ1, 02)
(9)
Sa Φ1) + 82L Φ2),
where ((sh) indicates the shared parameters, di and the stand for parameters estimated in the ABSA task and the sentence ordering task, respectively. 81, 62 E [0, 1] are hyperparameters used to balance the weights of the tasks.
exp(3,)
aj
E exp(33))
a+m
(6)
83 = Σ ex,) T em: Σ (exy)
Te
emp
i=p+1
Experiments
where the position of the target word ranges at Ju 1, m]. and j E [1,,+11 (n + m, n] denotes the position of non-target words. 8g caleu- lates the semantic relatedness between the aspect and words other than the aspect in the sentence. The enhanced embedding of the aspect term is for- mulated by e, = Σ Crjea
5.1 Data and Evaluation Metries
We conducted the experiments four benchmark datasets: RESTI5 and LAP15 from the SemBval- 2015 task12 (Pontiki al., 2015), and RESTI6 and LAP16 from the SemFval-2016 task5 (Pontiki et al., 2016). The dataset consists of restaurant, and laptop domains, and positive, neutral, and negative
//ttpp://stanoorddd/ github. github.io/Correl io/CoreNLP/

---

RESTIS
LAPIS
RESTI6
LAPI6
Method
ACC
F1
LAC
EAC
ACC
F1
IAC
EAC
ACC
F1
LAC
EAC
ACC
F1
LAC
EAC
ASGCN
85.44
62.39
83,50
85.89
84,93
73.92
100.0
84.82
89,98
76.76
85.36
90.31
83.63
69.32
63.64
84,42
Dual-GCN
86.47
71.03
81.58
86.87
85.53
71.62
100.0
85.53
91.48
79.52
87. 80
91.63
86.12
71.81
78.79
86.31
AAGCN
86.79
68.22
84.84
86.96
85.65
72.43
100.0
85.64
92.02
77.51
87.80
92.13
85
71.58
69.70
86.15
Sentic-GCN
85.89
70.67
84.54
86.09
85.82
72.89
100.0
85.87
91.23
79.31
85.36
91.37
85.27
71.61
69.70
85.78
BIGAT
84.70
65.20
70.40
89.10
75.00
85.70
65.10
MPGN
86.92
68.54
80.62.
87.41
85,94
72.18
100.0
85.88
91.89
80.91
92.69 29.69
91.91
86.
68.33
75.76
86.03
SSEGCN
87.08
69.07
81.51
86,93
85,58
72.11
100,0
85.54
92.13
79.08
87.23
92.32
85 35.64
70.78
78.79
85,91
BERT-SIC
85,92
66.02
78.58
86.89
85.68
74.32
100,0
85.64
90.89
78.11
90.24
91.37
84.07
68.09
75.76
84.27
T-SCAPT
85.12
66.31
86.44
85,28
80,50
65.82
100.0
80.41
88.88
72.46
75.61
89.89
78.99
56. 12
78.79
69.72
B-SCAPT
87.61
71.88
85,38
87.87
88.96
78.86
100.0
89.02
92.01
79.62
85.41
92.34
86.81
72.27
81.82
86.88
CLEAN
84.43
70.89
81.61
88.44
84.61
71.29
100.0
84.73
89.85
72.21
90.24
89.50
85.50
72:22
75.76
85.87
CoGAN
84:20
70.70
85. 10
74.50
92.00
81.60
87.20
73,20
w/o CoM
85.43
68.08
85.42
85.28
85.89
71.64
100.0
85.76
90.89
75.84
87. 80
91.11
86.33
73:42
63.64
87.16
w/o CL
88.33
61.74
91.34
87.86
87.79
77.27
100.0
87.83
92.21
77.53
87.80
92.43
87.63
66.81
90.91
87.53
w/o CCE
67.23
89.29
86.83
76.54
100.0
91.91
76.22
92.69
87.94
62.54
87.88
87.93
ELCoM
89.63
72.53
92.23
89.34
89.52
79.71
100.0
89.23
93.36
81.32
95.12
93.28
89.14
73.62
93.94
88.93
Table 2: Main results for four datasets. IAC and EAC refer to the accuracy of implicit and explicit sentences, respectively. "w/o CoM" refers to the result without any coherence information. # indicates the results from the original papers. Note that the IACs of LAP15 are 100.0. The reason for this is that there are only four implicit sentiment sentences, and all baselines are identified correctly.
sentiment polarities. SemEval-2016 labeled only with explicit sentiments. We thus manually anno- tated implicit sentiment labels in the dataset. The data statistics are shown in Table 1.
1). Graph neural networks (GNN)-based meth- ods: ASGCN (Zhang et al., 2019), Dual-GCN (Li et al., 2021a), a)), AASGCN (Liang et al., 2021), Sentic-GCN (Liang et al., 2022), BiGAT (Shan et al., 2022), MGFN (Tang et al., 2022), and SSEGCN (Zhang et al., 2022);
We used accuracy ACC (%), and macro- averaged F1 (%) scores as metrics. We evaluated our model by using implicit, and explicit sentiment accuracy, IAC (%), and EAC (%), respectively. For a fair evaluation, we conducted each experiment five times and reported the average results.
2). Knowledge-enhanced (KE)-based meth- ods: BERT-SPC (Song et al., 2019), TransEn- CAsp+SCAPT (T-SCAPT) (Li et al., 2021b), BERTASp+SCAPT (B-SCAPT) (Li et al., 2021b), and CLEAN (Wang et al., 2022);
5.2 Implementation Details
3). Global context (GC)-based methods: Co-
Following Chen al. (2020) and Cai et al. (2020), we randomly chose 10% of the training data and used it as the development data. The optimal hyper- parameters are as follows: The initial learning rate for coherence modeling was 6e-6 and others were 2e-5. The weight decay was set at 1e-3, and the dropout rate was 0.1. The number of negative sam- ples B was 5, and the margin was 0.1. The bal- ance coefficients in and 52 were at 0.9 and 0. respectively. The number of graph convolutional layers was 2. All hyperparameters were tuned us- ing Optuna². The search ranges are reported in Appendix A.3. We used Adam W (Loshchilov and Hutter, 2017) as the optimizer.
GAN (Chen et al., 2020).
6 Results and Discussion
6.1 Performance Comparison
Table 2 shows the results. Overall, the ELCoM at- tained an improvement over the second-best meth- ods by 0.63~2.31% 1% ACC and 0.57~ 1.08% F1- score, except for the Fi-score dataset. In particular, achieved remarkable results in implicit sentiment polarity classification, as the ELCoM achieved an improvement of IAC over the second-best method by 2.62~~14.81 %, while that of EAC was 0.24~2.366% This reveals that lever- aging document- and sentence-level coherence and reducing the influence of anomalous sentences sig- nificantly benefit sentiment analysis. Table 2 also provides the following observations and insights: Most basclines suffer from implicit sentiment analysis, while the ELCoM breaks the bottleneck
the RESTI6
5.3 Baselines
We compared our approach with the following base- lines:
hhupss//itthub.com///icccguuuuuu

---

Textual review in RESTI5 <Review rid= P#9
ad Raymond the bartender rocks! Pacifico is a great place to casually hang out. The drinks are great, especially when made by Raymond
88 The omlette for brunch great... the spinach is fresh, definately not frezen... quacamole pacifico yummy, as are the wings with chimmichuri, A weukness is the chicken salads, just average, just shredded, no seasoning on it. Also, personally wasn't fan of the portohello and asparagus
a9
all
al2
Overall, decent good price, with friendly people.
FOOD
QUALITY
FOOTS
QUALITY
RESTAURANTA MISCHLLANEOES
Negg
FOOD PRICE
Nege-)
Pos(+)
Pos(+))
Pos(+)
FOODY
IGENERAL
QUALITY
Neg(-)
FOODS QUALITY
FOOD? QUALITY
DRINKSR
QUALITY
FOODS PRICE
FOOD QUALITY
AGENEHAL
POODA QUALITY
BRINKS QUALITY
IIENEHAL
SERVICEC
AGENERAE
AASTAURANT
XLNet
XL with CoM
XLNet with CoM and CCE
Figure 3: Visualization of the sentence representations. Each color corresponds to each aspect-category, and bold font underlined words refer to the aspect term. dotted line separates positive negative polarities.
and maintains good performance.
Category
RESTI5
LAPIS
RESTI6
LAPI6
SCAPT, which exploits pre-training a large sentiment corpus and regards implicit and ex- plicit sentiments contrastive pairs, is compet- itive among baselines, and especially effective for the laptop domain. This indicates that prior knowledge is beneficial for sentiment analysis. GNN-based methods such as MGFN and SSEGCN achieved inspired results, suggesting that word-level syntactic representation can en- rich sentiment features while ignoring that the reviewer's opinions are likely be preserved throughout the review.
Sac
Rba
ACC
ACC
ACC
ACC
76.5
62.5
x
88.3
70.1
88.7
78.9
92.6
77.8
88.4
66.3
*
87.3
69.0
88.3
77.3
92.9
79.8
89.0
67.8
89.6
72.5
89.5
79.7
93.4
81.3
89.1
73.6
Imp.(%)
2.9
7.9
2.4
4.1
1.6
6.7
1.4
17.7
Table 3:
Performance on cross-category enhance- ment. "Saf" indicates SA filtering, and "Rba" indicates retrieval-basse attention. V and X denote with/without each module, respectively.
RESTIS
LAPIS
RESTI6
LAPI6
Methou
ACC
F1
ACC
F1
ACC
F1
ACC
F1
The ELCoM and CoGAN exploit document- level sentiment knowledge, suggesting that cap- turing the reviewer's s consistent sentiment expres- sions contributes to improving performance.
SSEGON
80.1
67.3
83.9
B-SCAPT
82.1
61.7
83.7
72.7
83.3
70.5
79.1
62.5
78.6
67.6
69.6
BLCMM
83.3
66.4
83.2
73.7
84.9
75.4
82.3
65.5
Table 4: Results on reviews that contain different polar- ities in the same category.
6.2 Ablation Study
We conducted an ablation study examine the ef- fects of each component the ELCoM. The results Table prompts the following observations:
RESTI5 by Fl, indicates that document-Level coherence learned from the sentence ordering
The worst result, especially the ICA by the EL.
task contributes to improving performance.
CoM without applying CoM (w/o CoM) sup-
The ELCoM without cross-category enhance- ment (w/o CCE) suffers from a severe perfor- mance drop. particularly on LAP16 by F1, indi- cating that the SA mechanism that we used to reduce the influences between anomalous nodes is effective for accurate sentiment analysis.
ports our hypothesis that the aspect-category is
a strong clue for capturing sentence-level coher-
ence aid implicit sentiment classification.
The ELCoM without contrastive learning (EL- CoM w/o CL), particularly the worst result on

---

Case
Review Text. Underline: the target aspect to identify.
Aspect Category
BERT-SPC
B-SCAPT
CLEAN OURS
Si Great /pes)
Explicit
Implicil
Buz boy (SERVICE)
Neg Pos
Pos
Pos
The bus bov even spatied that my table was shaking stabilizeed it for (po))
Food (POOD)
Nou X
Pos
Pos
5/ posl fine, with linl--asteer-han--oooe pposs
Sabo (POOD)
Explioit
Explicit
Sy. Mayhe wouldn't 't back
many years from when
forgotten went already Explicil
(RESTAURANT)
Pos
Pos
Pos
Neg v/
Sg: Maybe I'Il go back eade more many vears from now when I've forgettien
weni there already Implicit
51 No pomperistt)oss: Implkit
$2: I cun't say enough about this place pus)-
Explicit
(RESTAURANT)
Neg X
Nou X
Neg X
Pos v/
5yt great sushiipas and better scrvice! Explicit
Figure 4: Case study RESTI6 data with their polarities predicted by BERT-SPC, B-SCAPT, CLEAN, and our approach: V (or x) denotes that the predicted sentiment polarity is correct (or incorrect).
Type of Contexts
RESTIS
LAPIS
RESTI6
LAPI6
Over-capturing contexts by SA,
ACC
F1
ACC
F1
ACC
F1
ACC
F1
Sy: You are with bot date and ho/she has an urge for sushi...then
LSTM
80,3
64.9
80.6
66.3
83.4
70.1
80.4
62,8
SA
87.8
66.9
87.6
74.6
91.4
75.2
87.9
67.2
this might be the place.
Part))
Coherence
89.6
72.5
89.5
79.7
93.4
81.3
89.1
73.6
82: The fish was fresh, though was cut very thin.
Negis)
8; Grent service.
Table 5: Comparison against several network models.
84 Good sake selection
Pegel
Figure 5: Illustration of over-capturing contexts. Words such as "great," and "good" overly affect the negative opinion word "thin."
Recall that our cross-category enhancement utilizes two attention mechanisms, SA filtering, and Rba, to offset the impact anomalous nodes. To examine the effectiveness each mechanism, we performed experiments, which are shown in Table 3. Overall, the enhancement improved the performance, for instance, 1.36~2.93% by ACC and 111-~1.7729 by Fl in all datasets. Specifically, SA filtering yields more benefits for RESTI5 and LAP15, while Rba works well for other datasets.
that mention the same aspect are grouped together and share sentiments. (3) XLNet with CoM and cross-category enhancement shows that the dots in the same aspect-category better clustered, while different aspect-categories are dispersed.
6.3 Efficacy of Coherence-Based Contexts
It is interesting to note how cross-category en- hancement dealing with anomalous nodes affects performance. Table 4 shows the results against SSEGCN and B-SCAPT by focusing reviews containing opposite polarities in the same cate- gory. We can see that with CCE, the model im- proves 1.27~53.76% by ACC and 4.86~~8,94 9.9%% by F1. This clearly supports the effectiveness of our cross-category enhancement, To better understand the ablation study, we visualized the distribution of sentence representations of each module by t- SNE (Van der Maaten and Hinton, 2008), which is illustrated in Figure 3. We can that (1) XL- Net without CoM and cross-category enhancement roughly identifies the sentiment. a2 close to ag. 010, and 011, although they have opposite polari- ties. (2) Xet with CoM shows that the sentences
We compared the coherence-based contexts with the SA- and LSTM network-based ones to ver- ify the effectiveness of the ELCoM, these tech- niques are well known for effectively learning con- text dependencies. The results are shown in Table 5, is reasonable that SA works better than LSTM, as the latter learns long-term dependencies. However, reveals that SA may overcapture the context of irrelevant words even when the sentiment of the target aspect is explicit. As shown in Figure 5, 82 contains two aspects: "food quality" and "food style." The SA-based context incorrectly predicts the sentiment polarity toward the latter aspect, as it overly captures positive sentiment from the words, "great," "good," and "fresh" as the descriptors this aspect regardless of their dramatically effec-

---

tive transition, which is in fact not the case. In contrast, our approach captures not only the origi- nal sentiment the sentence but also the context of sentiments. From the experimental results, we can conclude that coherence-basee contexts are cur- rently the optimal alternative compared to SA and LSTM in sentiment analysis tasks.
*Explicit sentiment: liked the atmosphere very much the food not worth the price.
"unegor=="abbieeeesggne calegory-"foodiqquliy polarity-"negative Satngory==foodHpiiio polarity=
"positivv"
-Implicit sentiment: are locals, and get the feeling only way this place survives with such average because most customery probubly one-time customer lourists. "fodguuy""ood"Huulltt" polarity"negaatve" calegory= "rsserrattttrrnnngggnnnalll polarity- "ncgiivviininiv
6.4
Case Study
We highlighted the typical and difficult examples and compared the ELCoM with the baselines. We chose BERT-SPC, B-SCAPT, and CLEAN as base- lines, since BERT-SPC is often used as a bench- mark model, and others are focused on implicit sentiment analysis. Figure illustrates the results.
Figure Examples of explicit and implicit sentiments.
Underlined words indicate aspect terms.
neutral sentiments are always too ambiguous to identify, and neutral samples are not enough in training sets. (2) Many sentences with mixed (positiveeneeaativ) sentiment polarities were in- correctly identified, caused by negation words and unspecified referents within the sentences. The majority of implicit sentences do not have both positive and negative sentiment polarities, as a user often exerts an objective fact to express an implicit opinion, which means less probabil- ity of containing more than one different senti- ment. In contrast, much more explicit sentences have mixed sentiment polarities. In the example from the RESTI6 test dataset which is shown in Figure 6, we can see that only one sentiment polarity, negative appears in the implicit sen- tence, while the sentence with explicit sentiment includes both positive and negative sentiment.
Case 1. BERT-SPC and B-SCAPT failed implicit sentiment in 84, as they could not correctly iden- tify the sentiment of "service", which appears in $1- Likewise, 85 contains two aspects in the same category of "food", and because of its complex syn- tactic structure, it results BERT-SPC incorrectly classifying the "salsa (food)" aspect as positive. To correctly identify these aspects, CLEAN infers causal representations of implicit sentiments 81 and and the ELCoM learns coherent contexts. Case 2. The sentiment of 85 shows an implicit negative polarity in terms of the "restaurant". The ELCoM captures the polarity of the sentiment in 84 via sentence-level coherence by hypergraph toward the same aspect-category assist sentiment clas- sification.
Case 3. is extremely difficult to analyze the sentiment within short sentences, such as the "No comparison" in case 3. In contrast to the baselines, the ELCoM can capture the context of sentiment from $2 and 83 auxiliary information.
Conclusion
We proposed aspect-category enhanced learning with neural coherence model (ELCoM) for im- plicit sentiment analysis. To mine opinions from explicit sentences to aid implicit sentiment classifi- cation. ELCoM captures document-level coherence by using contrastive learning, and sentence-level by a hypergraph. To further offset the impact anomalous nodes in hyperedges, we proposed a cross-category enhancement on node embeddings. Extensive experiments have shown that the EL- CoM achieves competitive performance against Mate-offthe-art sentiment analysis methods. Fu- ture work includes, (i) improving the ELCoM by introducing pre-training large sentiment corpus, and (ii) extending the EL.CM to simultaneously detect catect-cs and their polarities (Cai et al., 2020).
6.5 Error Analysis on Explicit and Implicit Sentiments
We conducted error analyses on four datasets and found that the implicit sentence accuracy by EL- CoM better than that of explicit sentences in some cases. There are three possible reasons:
One reason is the effectiveness of the coherence modeling (CoM). Table 2 in Section 6.2 indicates that without CoM, the ELCoM obtains better accuracy explicit sentiments than that of the implicit sentiments in the RESTI6 and LAP16 datasets.
There are two major error cases on explicit sen- tences: (1) Many sentences containing neutral sentiments are incorrectly classified, because

---

Limitations
Sharma Eva, Huang Luyang, Zhe Hu, and Wang Lu. 2019. An entity-driven framework for abstractive summarization. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages Linguistics..
The ELCoM model adopts time- consuminn mod- ules, i.e., transformer (O(n²) and GCN (O(n²) where n refers to the number of words, therefore its computational cost heavily relies on the length of textual reviews. Although the ELCoM outper- forms SOTA baselines in ACC, it still struggles with the performance in Fl. One reason is because of neutral sentiments which are very ambiguous and difficult to predict correctly.
Association for Computational
Youmna Farag and Helen Yannakoudakis.. 2019. Multi- task learning for coherence modeling. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 629-639. Asso- ciation for Computational Linguistics (ACL).
Ethics Statement
Youmna Farag, Helen Yannakoudakis, and Ted Briscoe. 2018. Neural automated essay scoring and coherence modeling for adversarially crafted input. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guisticx: Human Language Technologies, Volume (Long Papers), pages 263-271.
This paper does not involve the presentation of a new dataset, an NLP application, and the utilization of demographic or identity characteristics informa- tion.
Acknowledgements
Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022. SNaC: Coherence error detection for narrative sum- marization. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 444 463, Abu Dhabi, United Arab Emi- rates. Association for Computational Linguistics.
We would like to thank anonymous reviewers for their thorough comments and suggestions. This work is supported by Kajima Foundation's Support Program. The first author is supported by the China Scholarship Council (N0.202208330091).
Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wen- biao Ding, and Minlie Huang. 2021. Long text gener- ation by modeling sentence-level and discourse-level coherence. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 6379-6393.
References
Hongjie Cai, Yaofeng Tu, Xiangsheng Zhou, Jianfei Yu, and Rui Xia. 2020. Aspect-category based senti- ment analysis with hicrarchical graph convolutional network. In Proceedings of the 28th international conference on computational linguistics, pages 833- 843.
Sungho Jeon and Michael Strube. 2022. Entity-based neural local coherence modeling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 7787-7805.
Chenhua Chen, Zhiyang Teng, Zhongqing Wang, and Yue Zhang. 2022. Discrete opinion tree induction for aspect-based sentiment analysis. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 2051-2064.
Prathyusha Jwalapuram, Shafiq Joty, and Xiang Lin. 2022. Rethinking self-supervision objectives for gen- eralizable coherence modeling. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 6044 6059.
Xiao Chen, Changlong Sun, Jingjing Wang, Shoushan Li, Luo Si, Min Zhang, and Guodong Zhou. 2020. Aspect sentiment classification with document-level sentiment preference modeling. In Proceedings of the 58th Annual Meeting of the Association for Com- putatoonal Linguistics, pages 3667-3677.
Ruifan Li, Hao Chen, Fangxiang Feng, Zhanyu Ma, Xi- aojie Wang, and Eduard Hovy. 2021a. Dual graph convolutional networks for aspect- based sentiment analysis. In Proceedings of the 59th Anmual Meet- ing of the Association for Computational Linguistics and the Itth International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers). pages 63199..
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4171 117661. Association for Computational Linguis- tics.
Zhengyan Li, Yicheng Zou, Chong Zhang, Qi Zhang, and Zhongyu Wei. 2021b. Learning implicit senti- ment in aspect-based sentiment analysis with super- vised contrastive pre-training. In Proceedings of the

---

2021 Conference on Empirical Methods in Natural Language Processing, pages 246-256.
Maria Pontiki, Dimitrios Galanis, Harris Papageorgiou. Suresh Manandhar, and lon Androutsopoulos. 2015. Semeval-201 200 5 task 12: Aspect based sentiment analy- sis. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pages 486- 495.
Bin Liang, Hang Su, Lin Gui, Erik Cambria, and Ruifeng Xu. 2022. Aspect-based sentiment anal- ysis via affective knowledge enhanced graph con- volutional networks. Knowledge-Based Systems, 235:107643.
Yongxue Shan, Chao Che. Xiaopeng Wei, Xiaodong Wang. Yongjun Zhu, and Bo Jin. 2022. Bi-graph attention network for aspect category sentiment clas- sification. Knowledge- Bassd Systems, 25:: 109972.
Bin Liang, Hang Su, Rongdi Yin, Lin Gui, Min Yang, Qin Zhao, Xiaoqi Yu, and Ruifeng Xu. 2021, Beta distribution guided aspect-aware graph for aspect category sentiment analysis with affective knowledge. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 208- 218.
Youwei Song, Jiahai Wang. Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Attentional encoder network for targeted sentiment classification. arXiv preprint 190xv09902.0914.
Dongliang Liao, Jin Xu, Gongfu Li, and Yiru Wang. 2021. Hierarchical coherence modeling for docu- ment quality assessment. In Proceedings of the AAAI Conference on Artificial Intellience, volume 35, pages 13353-13361.
Siyu Tang. Heyan Chai, Ziyi Yao, Ye Ding, Cuiyun Gao, Binxing Fang, and Qing Liao. 2022. Affective knowledge enhanced multiple- pultiple-grahh fusion networks for aspect- based sentiment analysis. In Proceedings of the 2022 Conference Empirical Methods in Natural Language Processing, pages 5352-5362.
Nlya Loshchilov and Frank Hutter. 2017. Decou- pled weight decay regularization. arXiv preprint arXiv:1711.05101.
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11).
Mohsen Mesgar and Michael Strube. 2018 A neural local coherence model for text quality assessment. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 3328-4339 Brussels, Belgium. Association for Com- putational Linguistics.
Siyin Wang. Jie Zhou, Changzhi Sun, Junjie Ye, Tao Gui, Qi Zhang, and Xuan-Jing Huang. 2022. Causal intervention improves implicit sentiment analysis. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6966-6977.
Han Cheol Moon, Muhammad Tasnim Mohiuddin, Shafiq Joty, and Chi Xu. 2019. A unified neural coherence model. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2262-2272.
Xinfeng Wang, Fumiyo Fukumoto, Cui, Yoshimi Suzuki. Jiyi Li, and Dongjin Yu. 2023. Eedn: Enhanced encoder-decoder network with local and global context learning for poi recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, pages 383-392.
Farah Nadeem and Mari Ostendorf. 2018 Estimat- ing linguistic complexity for science texts. In Pro- ceedings of the Thirteensh Workshop on Innovative Use of NLP for Building Educational Applications. pages 45-55, New Orleans, Louisiana. Association for Computational Linguistics.
Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, and Zheng Zhang. 2021. A unified generative framework for aspect-based sentiment analysis. arXiv preprint 10iv:2200.043000
Heng Yang and Ke Li. 2021. Improving implicit senti- ment learning via local sentiment aggregation. arXiv preprint 1rXiv:2110.08600
Dat Tien Nguyen and Shafiq Joty. 2017. A neural local coherence model. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 1320-1330
Yifei Yang and Hai Zhao. 2022. Aspect-based senti- ment analysis as machine reading comprehension. Proceedings of the 29th International Conference on Computational Linguisties, pages 2461-2471.
Daraksha Parveen, Mohsen Mesgar, and Michael Strube. 2016. Generating coherent summaries of scientific articles using coherence patterns. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 772-783.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Car- bonell, Russ R Salakhutdinov, and Quoc V L.. 2019. XLNet: Generalized autoregressive pretraining for language understanding. Advances neural infor- mation processing systems, 32.
Maria Pontiki, Dimitrios Galamis, Harris Papagcorgiou, Ion Androutsopoulos. Suresh Manandhar, AL-Smadi Mohammad, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphee De Clercq, et al. 2016. Semeval- 2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-200 20061) pages 19-30.
Wenhui Yu and Zheng Qin. 2019. Spectrum-enhannee pairwise learning to rank. In The world wide web conference, pages 2247-2257.

---

Chen Zhang, Qiuchi Li, and Dawei Song. 2019. Aspect- based sentiment classification with aspect-specific graph convolutional networks. In Proceedings of the 2019 Conference Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (FMNIP-IJCNIPP), pages 4568-4578.
Zheng Zhang, Zili Zhou, and Yanna Wang. 2022. SSEGCN: Syntactic and semantic enhanced graph convolutional network for aspect-based sentiment analysis. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pages 49 4916-4925.

---

A Appendix
Negative Positive
A.1 Consistency of sentiment polarity.
Figure 7 shows the ratio of training and test re- views having consistent sentiment. We see that 50.3%~57.44% of the reviews had the same sentiment polarity regardless of different aspect categories over the review, and 70.8%~~84.88% of the reviews have the same sentiment polarity if they have the same aspect category.
R2 Document-Leww Coherence
Sentence- by Hypergraph
(c) CategrCa
Enhuncementt
Figure & Illustration a document- and sentence- coherence, and cross-category enhancement.
Commincacy Ovez Revice
Crncarayy Oher Seriee
100.0
Oomiikag Ove Citgary
Crealtag Over Oatggry
A.3 Implementation and hyperparameter setting
80.0
80.0
60.0
40.0
40.0
We implemented ELCoM and experimented with Pytorch single GPU: NVIDIA GeForce RTX 3090 (24GB memory). The search ranges of the hy- perparameters used our experiments are shown in Table 6.
20.0
20.0
REST 18
REST 13
LAP 16
LAP 15
0.0
REST 16
BEST 15
LAP
LAP
()) Trining Ser
(b) Test Se:
Figure 7: The ratio of reviews having consistent senti- ment in different datasets. Consistency Over Review (COR): the ratio of reviews which sentiment polari- ties are consistent. Consistency Over Category (COC): sentiment polarities in the same aspect category are con-
Parameter
Range 1c-6 lc-5
LR of CoM
LR of others
lc-5 lc44
Weight decay
(1c-4, 1c-3, 1c-2)
sistent.
Dropout rate
(0.1, 0.2, 0.3)
#Negative Samples B
(5, 6, 8, 9, 10)
Margin
(0.05, 0.1, 0.15, 0.2]
(0.7, 0.8, 0.9, 1.0)
A.2 Overview of coherence modeling and cross-category enhancement
d2
(0.05, 0.1, 0.15, 0.2]
#Bookk of GCN
[1,
Figure 8 illustrates a document- and sentence-level coherence and cross-category enhancement, where each circle alongside 8j indicates a sentence in the document, and a; indicates the aspect-category. (a) of Figure 8 indicates document-level coherence, i.e., four sentences that contain negative sentiment with the same category indicate preserving coher- ence. (b) in Figure 8 shows sentence-leve coher- ence. The sentiment of $1 and s5 toward the a1 is enhanced by each other and avoids ineffective propagation from the sentiment in irrelevant aspect- categories, such as 84 or the sentiment 81 re- lated to the aspect-category 4g. In contrast, (c) of Figure 8 illustrates cross cross-category enhancement to compensate for the dilemma of side effects from the sentences that contain different polarities even in the same category. For example, in Figure (c), the sentiment of 55 related to aspect-category 02 is prone to be incorrectly classified as positive, while the impact of 52 and 83 can be offset by cross- category enhancement.
Table 6: Search range of each hyperparameter: LR refers to the learning rate. LR of CoM indicates learning rate of coherence modeling. LR of oth- shows aspect-based sentiment analysis and cross- category enhancement.
A.4 Example of input data for multi-task learning
As shown in Table 7, the input data of the multi- task learning is as follows: the input of the sentence ordering task consists of the original review and its disordered review, and the input of the ABSA task comprises "Text" and "Opinions".
Following Li et al. (2021b), we labeled "Text" as implicit it does not contain any obvious opin- ion words for certain aspect. The number of B disordered "Text" are generated from the original reviews by randomly shuffling the sentences, which is illustrated in Table

---

Original Review [Review rid="1726677")
Text
$1: Average to good Thai food, but terrible delivery.
Opinions
target= "Thai food" eutegory="fooddkuuliny"" polarit=="oositvv" implicit sentiment="Falss
target="delivery" calegory=""ervvicetteennn"" polarit=="eegatve"
Text
$2: I've waited over one hour for food.
Opinions
target="null" "ntroory""srrvicceeeeen""" polarit="negativ" implicit sentiment="False"
Text
$3: They were very abrupt with me when I called and actually claimed the food late because they were out of rice.
Opinions
target="null"" calegory= "srrceie=rrrvicceeeeeeer""" polarit=="eegativ" implicit_ sentteeenntttnaaaaeee
Text
94: A Thai restaurant out of rice during dinner?
Opinions
Target="Thai restaurant" plicit_ "egi_ssetiment="Tnss
"rreooy="resarrmttiiislllaa
polarity""negative im-
Text
85: The food arrived 20 minutes after I called, cold and soggy.
Opinions
target="food" categoy=""foodHuually"" polarity="negative seppiictssentmmet=="FFaaa
target="null" calegory="serriccsfeeennd polarit=="negativ" implicct_sentiment= sentiment="True
Disordered Review
Text
84: A Thai restaurant out of rice during dinner?
Text
$1: Average to good Thai food, but terrible delivery.
Text
85: The food arrived 20 minutes after I called, cold and soggy.
Text
8y: They were very abrupt with me when called and actually claimed the
food was late because they were out of rice.
Text
$2: I've waited over hour for food.
Table 7: The example of original and disor- dered review: "target" and "category" refer to pect terms, and aspect category, respectively. Po- larity takes a positive, neutral, or negative value. implicit_sentimen takes "False" for explicit and "True" for the implicit sentiment.


Reviews:
[{"review_id": "YfkHG1qjTx", "reviewer": "Reviewer_mTUy", "summary": "", "questions": "A. Why do you utilize XLNET as the representation model instead of BERT or RoBERTa?\n\nB. How BERT and RoBERTa models perform in the framework of ELCoM?\n\nC. What are the differences between your work and previous work that model aspect coherency? Related papers can be found in Missing References.\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "###  **Q: Why do you utilize XLNET as the representation model instead of BERT or RoBERTa?**\n\n**A:** Thank you for your feedback. There are three reasons why we utilized XLNet as a representation model:\n\n1). XLNet uses permutation language modeling as the optimization objective. The XLNet is more appropriate for contrastive learning which aligns the coherent and incoherent representations (in Section 4.2) as it learns the correct order of all the tokens in the input.  It has been shown to be effective in capturing global dependencies between input tokens [1]. On the other hand, BERT and RoBERTa as masked language models focus on the contexts of masked words.  \n\n2). The max token length of both BERT and RoBERTa is 512. Therefore, many review texts must be manually deleted to be embedded by the BERT and RoBERTa models. There were 7.75% samples of REST 15 and 11.11% samples of LAP 16 datasets, leading to a lack of information in this task. In contrast, XLNet can handle long texts (i.e., 1,024), which is enough for all datasets.  \n\n3). Several recent studies have verified that XLNet is effective in modeling coherence tasks [2, 3] as XLNet can capture document (discourse) -level coherence.  \n\nMore interestingly, the ablation results in Table 2 (Section 6.2) show that even if we use XLNet as the backbone, ELCoM without coherence modeling (CoM) cannot exceed baselines except for F1 on LAP16, showing that the coherence model contributes most to improving the performance.\n\nWe will mention the reason in our paper.\n\n###  **Q: How BERT and RoBERTa models perform in the framework of ELCoM?**\n\n**A:** The table below shows the results (ACC) of ELCoM with XLNet, BERT, and RoBERTa. We only report ACC here due to space limitations, while we found that the results of IAC, EAC, and F1 had similar tendencies. The number of input tokens is the first 512 tokens for BERT and RoBERTa.\n\n|      |  REST 15 | LAP 15 | REST 16 | LAP 16 |  \n| ----------- | ----------- | ----------- | ----------- | ----------- |\n| SOTA | 87.61 (**B-SCAPT**) | 88.96 (**B-SCAPT**)  |  92.13 (**SSEGCN**) | 87.20 (**CoGAN**)|  \n| BERT | 86.30 | 87.16 | 90.56 | 86.62 |  \n| RoBERTa | 87.02 | 89.06 | 92.32 | 87.75 |  \n| XLNet | 89.63 | 89.52 | 93.36 | 89.14 |  \n\n\nThe results provide the following observations and insights:\n\n1). We can see from the table that XLNet works better than BERT and RoBERTa. The result indicates that the XLNet is effective for modeling coherence.\n\n2). ELCoM by BERT and RoBERTa are still comparable against the baselines by average -1.5%, and +0.1% improvement in ACC, respectively. Some of the results by BERT and RoBERTa in the framework of ELCoM were worse than the second-best results (underlined in Table 2, Section 6.1). This is reasonable because input token length is restricted, and they cannot model the document-level coherence which is one of the main contributions of our ELCoM.\n\n### **Q: What are the differences between your work and previous work that model aspect coherency? Related papers can be found in Missing References**\n\n**A:** Thank you for your helpful suggestions. We will refer to Yang et al. work [4] and refine Section 2 Related Work to explain the differences between our work and [4].\n\nYang et al. focused on the local (word-level) aspect of sentiment coherency within the target sentence [4]. They attempted to utilize three aspect features, i.e., sentence pair, local context-based, and syntactical local context-based to construct sentiment aggregation windows to model aspect sentiment coherency. Their approach is similar to ours in that they consider sentence-level information. The difference is that our model leverages opinions from explicit sentences by capturing both sentence- and document-level to aid implicit sentiment learning.\n\n### **Q: In line 200, \\citet may be more proper than \\cite.**\n\n**A:** Thank you very much for your advice. We have corrected the citation in line 200, and carefully checked our paper again.\n\n\n### **Reference:**\n\n[1] Yang Z, Dai Z, Yang Y, et al. Xlnet: Generalized autoregressive pretraining for language understanding[J]. Advances in neural information processing systems, 2019, 32.  \n[2] Jwalapuram P, Joty S, Lin X. Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling[C]// ACL. 2022: 6044-6059.  \n[3] Jeon S, Strube M. Centering-based neural coherence modeling with hierarchical discourse segments[C]// EMNLP. 2020: 7458-7472.  \n[4] Yang, Heng, and Ke Li. \"Improving Implicit Sentiment Learning via Local Sentiment Aggregation.\" arXiv preprint arXiv:2110.08604 (2021).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is well-written.\n2. Utilizing coherence model to recognize implicit sentiments is novel.\n3. Detailed experiments not only show the ability of ELCoM but also give exhaustive analysis of the superiority of ELCoM.\n", "reasons_to_reject": "1. It seems that there already have work done for implicit sentiment learning via aspect coherency, while the authors do not explain the differences.\n2. Lack of explanation of utilizing XLNET as the representation model. Moreover, to provide a fair comparison, it is valuable to provide the results based on BERT or RoBERTa model.\n3. Unfair comparison between baselines. Since most of the baselines are based on BERT and RoBERTa, it is unfair to compare these models with ELCoM based on XLNET.\n", "questions_to_authors": "A. Why do you utilize XLNET as the representation model instead of BERT or RoBERTa?\n\nB. How BERT and RoBERTa models perform in the framework of ELCoM?\n\nC. What are the differences between your work and previous work that model aspect coherency? Related papers can be found in Missing References.\n", "paper_topic_and_main_contributions": "The authors propose aspect category enhanced learning with a neural coherence model (ELCoM) to tackle the implicit sentiments in ABSA. The presented model can capture document- and sentence-level coherence by contrastive learning and hypergraph, respectively. Moreover, cross-category enhancements have been utilized to tackle sentiment conflict. The experimental results show that ELCoM achieves SOTA performance and reveal the effectiveness of ELCoM. The contribution are three folds: 1) propose ELCoM to aid implicit sentiment classification; 2) propose CCE to offset the impact of anomalous nodes; 3) extensive experiments and analysis on public datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "1. Yang, Heng, and Ke Li. \"Improving Implicit Sentiment Learning via Local Sentiment Aggregation.\" arXiv preprint arXiv:2110.08604 (2021).", "Typos_Grammar_Style_and_Presentation_Improvements": "1. In line 200, \\citet may be more proper than \\cite."}, "extra_text": {}}, {"review_id": "84NJF5YVuK", "reviewer": "Reviewer_3JpN", "summary": "", "questions": "a.\tOn some test datasets, IAC (implicit sentence accuracy) of the proposed method is better than EAC (explicit sentence accuracy). Why does not the proposed method work well on the explicit cases?  More illustration should be provided.\n\nb.\tIn the evaluation，IAC and EAC refer to the accuracy of implicit and explicit sentences.  How to calculate IAC and EAC  for the sentences with multi aspects.  \n\nc.\tHow about the aspect-level accuracy of the proposed method? \n", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Q: a. On some test datasets, IAC (implicit sentence accuracy) of the proposed method is better than EAC (explicit sentence accuracy). Why does not the proposed method work well on the explicit sentiment cases? More illustration should be provided.**\n\n**A:** Thank you for your comments and suggestions. In addition to the Case study section (Section 6.4), there are three possible reasons why IAC is better than EAC:\n\n1). One reason is the effectiveness of the Coherence modeling (CoM). Table 2 in Section 6.2 shows that without CoM, the ELCoM obtains better accuracy on explicit sentiments than that of the implicit sentiments in the REST 16 and LAP 16 datasets. The accuracy of the IAC and EAC in REST15 is close. For the LAP 15 dataset, we cannot make a fair comparison because there are only four implicit sentences in this dataset.  \n\n2). We found two main error cases of explicit sentences: (1) Many sentences containing neutral sentiments are incorrectly classified, because neutral sentiments are always too ambiguous to identify, and neutral samples are not enough in training sets. (2) Many sentences with mixed (positive/negative) sentiment polarities were incorrectly identified, caused by negation words and unspecified referents within the sentences.  \n\n3). Furthermore, we examined datasets and found that the majority of implicit sentences do not have both positive and negative sentiment polarities, as a user often exerts an objective fact to express an implicit opinion, which means less probability of containing more than one different sentiment. In contrast, much more explicit sentences have mixed sentiment polarities. An example from the REST 16 test dataset is shown below. This could be one of the reasons for this result.  \n\n- Example of explicit sentiment:\n\n  **S1: I liked the atmosphere very much but the food was not worth the price.**  \n    target=\"atmosphere\" category=\"ambience#general\" polarity=\"positive\"  \n    target=\"food\" category=\"food#quality\" polarity=\"negative\"  \n    target=\"food\" category=\"food#prices\" polarity=\"negative\" \n\n- Example of implicit sentiment:\n\n    **S2: We are locals, and get the feeling the only way this place survives with such average food is because most customers are probably one-time customer tourists.**  \n    target=\"food\" category=\"food#quality\" polarity=\"negative\"  \n    target=\"place\" category=\"restaurant#general\" polarity=\"negative\"  \n\nWe will mention these in our paper.\n\n### **Q: b. In the evaluation，IAC and EAC refer to the accuracy of implicit and explicit sentences. How to calculate IAC and EAC for the sentences with multi aspects.**\n\n**A:**  An aspect denotes a pair of aspect category and term (line 166). In the experiments, we regard each aspect as one sample. For example, if there exist three aspects in a sentence, then there will be three samples for sentiment analysis. We used the same setting as [5] and [6]. \n\n### **Q: c. How about the aspect-level accuracy of the proposed method?**\n**A:** We are sorry for making you confused at this point. Our result is based on aspect-level accuracy.\n\n### **Reference:**\n\n[5] Li Z, Zou Y, Zhang C, et al. Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training[C]// EMNLP. 2021: 246-256.  \n[6] Wang S, Zhou J, Sun C, et al. Causal Intervention Improves Implicit Sentiment Analysis[C]// COLING. 2022: 6966-6977.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Implicit sentiment detection is a very challenging task in the aspect-based sentiment analysis applications. This paper proposes a neural coherence model (ELCoM) for implicit sentiment detection, which captures document-level coherence contrastive learning (CL) and sentence-level by hypergraph in the sentiment analysis. \n\nThe proposed method is novelty and effective. \n\nExtensive experiments on the popular datatsets demonstrate that the proposed method outperforms state-of-the-art sentiment analysis methods.\n\nThis paper is written and organized well.\n\n", "reasons_to_reject": " More experimental analysis should be provided. On some test datasets, IAC (implicit sentence accuracy) of the proposed method is better than EAC (explicit sentence accuracy). Why does not the proposed method work well on the explicit sentiment cases?  More illustration should be provided.\n\nIn the evaluation，IAC and EAC refer to the accuracy of implicit and explicit sentences.  How to calculate IAC and EAC  for the sentences with multiple aspects? \n\n Aspect-level evaluation should be reported since it is also very important.  \n", "questions_to_authors": "a.\tOn some test datasets, IAC (implicit sentence accuracy) of the proposed method is better than EAC (explicit sentence accuracy). Why does not the proposed method work well on the explicit cases?  More illustration should be provided.\n\nb.\tIn the evaluation，IAC and EAC refer to the accuracy of implicit and explicit sentences.  How to calculate IAC and EAC  for the sentences with multi aspects.  \n\nc.\tHow about the aspect-level accuracy of the proposed method? \n", "paper_topic_and_main_contributions": "Implicit sentiment detection is a very challenging task in the aspect-based sentiment analysis applications. This paper proposes an aspect-category enhanced learning with a neural coherence model (ELCoM), which learns document-level coherence by using contrastive learning (CL) and sentence-level by hypergraph for mining opinions to aid implicit sentiment classification. Experiments on four benchmark datasets are provided. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7Ycs2gf4eL", "reviewer": "Reviewer_3M1A", "summary": "", "questions": "Question A: Regarding the hypergraph construction process in section 4.2, if two sentences have different sentiments on the same AC, will they still be connected by the same hyperedge?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Question A: Regarding the hypergraph construction process in section 4.2, if two sentences have different sentiments on the same AC, will they still be connected by the same hyperedge?**\n\n**A:** Yes, it is, and that is exactly why we performed a cross-category enhancement (CCE) to offset the impacts of anomalous nodes in the hypergraph. Specifically, we capture more fine-grained features (i.e., word-level syntactic representations via Standford Parser) and capture the correlations between the target sentence (to be classified) and the context sentences (which may mention the aspect category) by self-attention mechanisms for enhancement.\n\n###  **Q: As the paper is focused on implicit sentiment extraction, the authors should present the number and percentage of aspects with implicit sentiment in each dataset. In this way, the research purpose becomes more justified.**\n\n**A:** Thank you for your helpful comments.   \nThe number of aspects (percentage) for implicit and explicit is 211/2296 (7.07%) in REST15, 136/2773 (4.18%) in LAP15, 252/3114 (6.64%) in RES16, and 169/3541 (1.42%) in LAP16, respectively. We will report these values in the paper. \n\n###  **Q: I notice that there are not any 2023 models for comparison. The authors should compare the proposed ELCoM to the most recent models.**\n\n**A:** Thank you for your suggestion. We have followed the ACL 2023 conference and collected several related works [7, 8], although we are still waiting for them to release their codes for comparison. Once they have released their codes, we will try to add them.\n\n###  **Q: MAMS is a novel and widely used dataset for ABSA evaluation. It will be better if ELCoM is evaluated and accessed on MAMS also.**\n\n**A:** Actually, we have investigated this dataset before our submission. Unfortunately, this dataset does not contain available document-level contexts, which cannot support our conjecture that the opinions from explicit sentences in the review can improve the performance of implicit sentiment analysis.\n\n###  **Q: The abbreviation AC is first mentioned in the body of the paper in section 1, paragraph 2, but the meaning it stands for is not introduced until 2 paragraphs later. The explanation should be moved forward.**\n\n**A:** Thank you for pointing it out. We have moved the explanation of the abbreviation AC in section 1, paragraph 2 to section 1 line 51.\n\n###  **Q: In section 5.1, paragraph 2, the evaluation metric for explicit sentiment accuracy should be ECA instead of EAC.**\n\n**A:** Thank you very much for your feedback. We have corrected the typo, and carefully checked our paper again.\n\n###  **Reference:**\n\n[7] Ma F, Hu X, Liu A, et al. AMR-based Network for Aspect-based Sentiment Analysis[C]// ACL. 2023: 322-337.  \n[8] Zhang M, Zhu Y, Liu Z, et al. Span-level Aspect-based Sentiment Analysis via Table Filling[C]// ACL. 2023: 9273-9284.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The idea of leveraging document-level and sentence-level coherence for ABSA is very novel. The paper is easy to follow and the proposed model is well explained. Extensive experiments are conducted to verify the solidness of the model, accompanied with well-illustrated graphs.", "reasons_to_reject": "1. As the paper is focused on implicit sentiment extraction, the authors should present the number and percentage of aspects with implicit sentiment in each dataset. In this way, the research purpose becomes more justified. \n2. I notice that there are not any 2023 models for comparison. The authors should compare the proposed ELCoM to the most recent models. \n3. MAMS is a novel and widely used dataset for ABSA evaluation. It will be better if ELCoM is evaluated and accessed on MAMS also.", "questions_to_authors": "Question A: Regarding the hypergraph construction process in section 4.2, if two sentences have different sentiments on the same AC, will they still be connected by the same hyperedge?", "paper_topic_and_main_contributions": "This paper tries to solve the Aspect-based Sentiment Analysis task. It proposes a novel neural coherence model named ELCoM to leverage the coherence of the article to mine the implicit aspect sentiments. The Coherence is measured at both the sentence and document levels, through hypergraphs and contrastive learning respectively. The idea is based on two observations made by the authors regarding sentiment relationships. An additional Cross-Category Enhancement module is designed to reset the nodes that do not follow the observed relationships. \nThe main contributions of the paper are:\n1. The ELCoM model adopts coherence information for ABSA utilizing contrastive learning and hypergraphs. \n2. The Cross-Category Enhancement (CCE) model to correctly offset the anomalous nodes. \n3. Extensive experiments to evaluate the proposed model.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. The abbreviation AC is first mentioned in the body of the paper in section 1, paragraph 2, but the meaning it stands for is not introduced until 2 paragraphs later. The explanation should be moved forward. \n2. In section 5.1, paragraph 2, the evaluation metric for explicit sentiment accuracy should be ECA instead of EAC. \n3. ....."}}]

Author rebuttal:

Meta-review: There is a consensus among the reviewers that the paper presents a novel and well-organized approach to tackling implicit sentiments in ABSA. The proposed ELCoM model leverages both document-level and sentence-level coherence through contrastive learning and hypergraphs, and it also incorporates cross-category enhancements to address sentiment conflicts.
Reviewers have commended the paper for its clear presentation and extensive experiments, which demonstrate that ELCoM achieves state-of-the-art performance. However, there are also some concerns and suggestions for improvement. Reviewer 1, for instance, raises questions about the choice of the XLNET representation model over BERT or RoBERTa, and suggests a fairer comparison with these models. Reviewer 2 raises the issue of why ELCoM performs better on implicit sentence accuracy (IAC) than explicit sentence accuracy (EAC) on some test datasets and calls for more explanation. Reviewer 3 suggests that the paper should justify its research purpose by presenting the number and percentage of aspects with implicit sentiment in each dataset and also recommends evaluating ELCoM on the MAMS dataset.
In summary, the paper is well-received for its innovative approach to ABSA but also needs to address some concerns and incorporate additional experimental analysis to strengthen its claims.
Decision: Accept-Findings