A Self-enhancement Multitask Framework for Unsupervised Aspect Category Detection
Authors: Thi-Nhung Nguyen
Hoang Ngo
Kiem-Hieu Nguyen
Tuan-Dung Cao
Venue: EMNLP 2023 Main (2023)
Keywords: Sentiment Analysis
Aspect Category Detection
unsupervised learning
weakly supervised learning
Aspect-based Sentiment Analysis

Abstract: Our work addresses the problem of unsupervised Aspect Category Detection using a small set of seed words. Recent works have focused on learning embedding spaces for seed words and sentences to establish similarities between sentences and aspects. However, aspect representations are limited by the quality of initial seed words, and model performances are compromised by noise. To mitigate this limitation, we propose a simple framework that automatically enhances the quality of initial seed words and selects high-quality sentences for training instead of using the entire dataset. Our main concepts are to add a number of seed words to the initial set and to treat the task of noise resolution as a task of augmenting data for a low-resource task. In addition, we jointly train Aspect Category Detection with Aspect Term Extraction and Aspect Term Polarity to further enhance performance. This approach facilitates shared representation learning, allowing Aspect Category Detection to benefit from the additional guidance offered by other tasks. Extensive experiments demonstrate that our framework surpasses strong baselines on standard datasets.

A Self-enhancement Multitask Framework for Unsupervised Aspect Category Detection
Thi-Nhung Nguyen', Hoang Ngo', Kiem-Hieu Nguyen', Tuan-Dung Cao²
VinAI Research
School of Information and Communications Technology,
Hanoi University of Science and Technology
w.. (hinutk Jesictt hust. edu. vn
(v. nhungnt8,,, wwoonggnv49eevinai..i.
Abstract
2019; Tulkens and van Cranenburgh, 2020; Shi et al., 2021). However, this approach faces ma- jor problem with the mapping process, requiring manual labeling and a strategy to resolve aspect label inconsistency within the same cluster. Re- cent works have introduced using seed words to tackle this problem (Karamanolakis et al., 2019; Nguyen al., 2021; Huang et al., 2020). These works direct their attention learning the embed- ding space for sentences and seed words estab- lish similarities between sentences and aspects, As such, aspect representations are limited by a fixed small number the initial seed words. (Li et al., 2022) is one of the few works that aims to expand the seed word set from the vocabulary of a pre- trained model. However, there is overlap among the additional seed words across different aspects, resulting in reduced discriminability between the aspects. Another challenge for the unsupervised ACD task is the presence of noise, which comes from out-of-domain sentences and incorrect pseudo labels. This occurs because data is often collected from various sources, and the pseudo labels are usually generated using greedy strategy. Current methods handle noisy sentences by either treat- ing them having GENERAL aspect (He et al., 2017; Shi al., 2021) or forcing them to have one of the desired aspects (Tulkens and van Cranen- burgh, 2020; Huang et al., 2020; Nguyen et al., 2021). To address incorrect pseudo labels, (Huang et al., 2020; Nguyen et al., 2021) attempt to assign lower weights to uncertain pseudo-labels. However, these approaches might still hinder the performance of the model models are learned from large amount of noisy data.
Our work addresses the problem of unsuper- vised Aspect Category Detection using small set of seed words. Recent works have focused on learning embedding spaces for seed words and sentences establish similarities between sentences and aspects. However, aspect rep- resentations are limited by the quality of ini- tial seed words, and model performances compromised by noise. To mitigate this lim- itation, we propose a simple framework that automatically enhances the quality of initial seed words and selects high-quality sentences for training instead of using the entire dataset. Our main concepts are to add number of seed words to the initial set and to treat the task of noise resolution as task of augmenting data for a low-resource task. In addition, we jointly train Aspect Category Detection with Aspect Term Extraction and Aspect Term Polarity to further enhance performance. This approach facilitates shared representation learning, al- lowing Aspect Category Detection to benefit from the additional guidance offered by other tasks. Extensive experiments demonstrate that our framework surpasses strong baselines on standard datasets.
1 Introduction
Aspect Category Detection (ACD), Aspect Term Extraction (ATE), and Aspect Term Polarity (ATP) are three challenging sub-tasks of Aspect Based Sentiment Analysis, which aim identify the as- pect categories discussed ie.i., FOOD), aspect terms presented (e.i., 'fish', 'rolls'), ) and determine the polarity of each aspect term (e.i.. 'positive', 'negative") in each sentence, respectively. To better understand these problems consider the example in Table 1.
In this paper, propose A Self-enhancement Multitask (ASeM) framework to address these lim- itations. Firstly, enhance the understanding of aspects and reduce reliance on the quality of initial seed words, we propose a Seedword Enhancement Component (SEC) to construct a high-quality set
Unsupervised ACD has mainly been tackled by clustering sentences and manually mapping these clusters to corresponding golden aspects using top representative words (He et al., 2017; Luo et al.,

---

(While the fish
posl is unquestionably fresh, rolls(neg) lend to be inexplicably bland. FOOD
Table 1: Aspect Category Detection, Aspect Term Extraction, and Aspect Term Polarity example.
of seed words from the initial set. The main idea is add keywords that have limited connections with the initial seed words. Connections are de- termined by the similarity between the keyword's context (the sentence containing the keywords) and the initial seed words. In this way, our task is simply to find sentences with low similarity to the initial seed words and then extract their im- portant keywords add to the seed word set. Since pseudo-label generation relies on the similarity be- tween sentences and seed words, we expect that the enhanced seed word set would provide sufficient knowledge for our framework to generate highly confident pscudo-labels for as many sentences as possible. Secondly, to reduce noise in the training data, instead of treating them as having a GEN- ERAL aspect (He et al., 2017; Shi et al., 2021) or forcing them to have one of the desired aspects (Tulkens and van Cranenburgh, 2020; Huang et al., 2020; Nguyen al., 2021), we propose leverage a retrieval-bait based data augmentation technique to automatically search for high-quality data from a large training dataset. To achieve this, we leverage a paraphrastic encoder (e.g. Arora et al. (2017); Ethayarajh (2018); Du et al. (2021)), trained to output similar representations for sentences with similar meanings, to generate sentence representa- tions that are independent of the target task (ACD). Then, we generate task embeddings by passing the prior knowledge (e.g., seed words) about the target task to the encoder. These embeddings are used as a query to retrieve similar sentences from the large training dataset (data bank). In this way, our frame- work aims to effectively extract domain-specific sentences that can be well understood based on seed words, regardless of the quality of the training dataset.
bly bland") for 'rolls'. Mcanwhile, ACD wants to detect the aspect category of the sentence, requir- ing awareness of the presence of "fish" and "rolls" (terms related the aspect of "food") within the sentence. Despite the usefulness of these relation- ships for ACD, the majority of existing works do not utilize this information. (Huang et al., 2020) is one of the few attempts to combine learning the representations of aspect and polarity at the sen- tence level before passing them through separate classifiers.
Our contributions are summarized follows:
We propose simple approach to enhance pect comprehension and reduce the reliance on the quality of initial seed words. Our frame- work achieves this by expanding the seed word set with keywords, based on their un- certain connection with the initial seed words.
A retrieval-based data augmentation technique is proposed to tackle training data noise. In this way, only data that connect well with the prior knowledge (e.g., seed words) about the target task used during training. As a result, the model automatically filters out-of-domain data and low-quality pseudo labels.
We propose to leverage a multi-task learning approach for unsupervised ACD, ATE, and ATP, aiming to improve ACD through the utilization of additional guidance from other tasks during the shared representation learn- ing.
Comprehensive experiments are conducted demonstrate that our framework outperforms previous methods on standard datascts.
Related Works
Another contribution to our work concerns the recommendation of multi-tasking learning for un- supervised ACD, ATE and ATP in a neural net- work. Intuitively, employing multi-task learning enables ACD to leverage the benefits of ATE and ATP. Referring to the example in Figure 1, ATE extracts Opinion Target Expressions (OTEs): 'fish' and 'rolls', which requires understanding the emo- tion 'positive" (expressed as 'unquestionably fresh") for 'fish' and 'negative' (expressed as 'inexplica-
Topic models were once the dominant approach (Brody and Elhadad, 2010; Mukherjee and Liu, 2012; 2; Chen et al., 2014) for unsupervised Aspect Category Detection. However, they can produce incoherent aspects. Recently, neural network-based methods have been developed address this chal- lenge.
Cluster Mapping-based resolvers: These meth- ods utilize neural networks to cluster effectively

---

and manually map (many-to-many mapping) the clusters their corresponding aspects. They em- ploy attention-based autoencoders (He al., 2017: Luo et al., 2019) or contrast learning approach (Shi et al., 2021) ) for clustering. Shi et al. (2021) further enhance performance by using knowledge distilla- tion to learn labels generated after clustering.
Output
Classification
Roberta
Seed words-based resolvers: These upproaches automate the aspect category mapping process by utilizing seed words that indicate aspect ap- pearance. Angelidis and Lapata (2018) use the weighted sum of seed word representations as as- pect representations, allowing mapping one-to-one in the auto-encoder model. Recent works focus on learning embedding spaces for sentences and seed words, generating pscudo labels for weakly supervised learning. They use Skip-gram (Mikolov et al., 2013) for embedding space learning and convolutional neural networks or linear layers for classification (Huang et al., 2020; Nguyen et al., 2021). Huang et al. (2020) jointly learn ACD with Sentence-level ATP, while Nguyen et al. (2021) consider the uncertainty of the initial embedding space. Without any human supervision, (Tulkens and van Cranenburgh, 2020; Li et al., 2022) rely solely on label names, similar to seed words. Tulkens and van Cranenburgh (2020) detect aspects using cosine similarity between pre-trained aspect and label name representations, while Li et al. (2022) train the clustering model with instance- level and concept-level constraints.
Retrieval-based Augmentation
Pseudo-label Generation
SEC
task specitic in-domain data
initaal seed words
data Dank
Unlabeled training
Input
Figure Overview our proposed Seff-enhancement Multitask (ASeM) framework.
seed words in this dataset, we utilize SEC expand the set of seed words order enhance its quality. By feeding the task-specific in-domain data and enhanced seed words to the pseudo-label genera- tion, we obtain high-quality pscudo labels for the task-specific in-domain data. Then, we leverage the retrieval-based augmentation to enhance the number of training samples from the data bank (the remaining part of the training data), based on our prior knowledge of the target task (seed words, task- specific in-domain data with high-quality pseudo labels). To this end, the high-quality pseudo labels and augmented data are passed through a multitask classifier to predict the task outputs.
3 Method
Our framework addresses three tasks for which no annotated data is available: Aspect Category De- tection (ACD), Aspect Term Extraction (ATE), and Aspect Term Polarity (ATP), ACD involves assign- ing given text to one of K pre-defined aspects of interest. ATE extracts OTEs in the text. ATP assigns sentiment each OTE. Note that, during sam-
3.1 Pseudo-label generation
training, we do not any ples, but rather rely on a small set of seed words to provide supervision signals.
The first step in our framework to generate pseudo-labels for the three subtasks ACD, ATE, and ATP, on a small unannotated in-domain dataset. In detail, the pseudo-labels for the tasks are created as follows:
Our framework called ASeM (short for A Self- enhancement Mutitask Framework), consists of three key components: (i) Pseudo-label generation, (ii) Retrieval-based data augmentation, and (iii) Classification. Figure 1 presents an overview of the framework. Initially, we extract a small sub- set of the training data to serve as the task speciipe domaimin data. Based on the quality of the initial
Aspect Category Detection: First, we map dic- tionary words into an embedding space by train- ing CBOW (Mikolov et al., 2013) on the un- labeled training data. Second, we embed sen- tences from the task-specific in-domain data as

---

8 sumn(W1, w2...,, W,), in which w; is the rep- resentation of the ith word and n is the sentence length. Similarly, the aspect category represen- tation a; sum(w (o) (a) .
Algorithm 1: Seedword Enhancement
Component (SEC)
Input: sentence set S, initial seed word set H, threshold ạy
wi2 w. in which the representation of the j"h seed word of unl the 1th aspect, and 4 is the number of seed words in the fth aspect. To this end, aspect category pseudo label of sentence is defined as follows:
Output: additional seed word set Ta
begin
2
+ Pseudo-LLbbll-Genneatiee H)
with Eq. I;
To < Boundary-Keyword-- Extraction(S, P);
y = argmax(sim(s.. 4,),, Sisk (1)
Sa < Ø with uncertain pscudo-label:
> Initialize set of sentences
where (s, a.) is the similarity between sentence
s and aspect ag Given set Ga; = To: U Ho. 1 < i < K in which Ha, is the set of given initial seed words, Ta, is the set of additional seed words. The similarity is calculated follows:
5
for S do
Calculate Connection(s): if Connection(() < then Add (s) to Sy; end
7
8
if s' n Go. Ø, v1 < < K then sim(s, a:) = 5" a,, sisk K else
9
end
P, <- Pseudo label of S,; Tu + Bicywods--Ettraciin (S,, Pa) D
Σ WEE'NG.
w if n Go, #
Extract keywords from sentences with uncertain prediction;
sim(s, (4)
otherwise
13
T <- Ty TT::
where s and a, are sentences and aspect repre- sentations, respectively. w and w are a word in a sentence and its representation. s is the set of words in the sentence s.
14
To < Auto mapping(T);
return Ta:
16 end
As discussed in the introduction, our framework proposes SEC described in Algorithm 1 to ob- tain Ta. To begin, generate temporary pseudo labels for all given sentences using the initial seed words. Based on the obtained lasud,,labels, we extract nouns and adjectives (called keywords) in the sentences for each aspect label and then extract keywords that appear in multiple aspects (called boundary keywords) and obtained Tü. At line 6, we calculate the connection between sentences and initial seed words based on the difference between the similarity of the sentence with its two most similar aspects. Note that, if Connection(s) Z 7 which y is a parammerr,meter, sentences are considered to have a certain connection with seed words, and if Connection(s) < 7,, there are un- certain connections. At line 12, we extract key- words from the sentences with uncertain connec- tions and obtain T.. Finally, the intersection of Tj and T, will be mapped to the relevant aspect. We utilize variant of clarity scoring function (Cronen- Townsend al., 2002) for the automatic mapping. Clarity measures the likelihood of observing word e in the subset of sentences related to aspect 0; as
compared to aj. A higher score indicates greater likelihood of word being related aspect dz-
claritya,aa))(() = ta, (w))oo ta, (w) lay (w)
(2)
where tu, (w) and tay (w) correspond to the 4-- normalized TF-IDF scores of w in the sentences annotated pseudo-label with aspect as and aj. re- spectively. In the training process, after obtaining pseudo labels, SEC recalculates the certainty of connec- tions similar to lines 5 to 10 of Algorithm 1, then removes uncertain connections S, out of S. Aspect Term Extraction: We extract aspect terms by considering all nouns that appear more than m times in the corpus.
Aspect Term Polarity: After generating aspect term pseudo-labels, we find polarity pseudo-labels of terms based on the context window around them. In detail, the generation will be carried out similarly to the ACD subtask with the input being the context window and polarity seed words.

---

3.2 Retrieval-based Data Augmentation
Sentence Bank
laok Information
To select high-quality data from unannotated data bank containing noise, select sentences with content similar to the reliable knowledge we have about the task-specific in-domain data, for example, seed words/a small in-domain dataset having certain connections with seed words. To do this, we first utilize a paraphrastic sentence en- coder to create representations for sentences in the data bank and the target task. The task embedding will be used tences in the sentence bank. Figure 2 illustrates retrieval-baae based data augmentation. The sentence encoder, task embedding, and data retrieval process occur as follows:
Information aspect Information
retrieved sentences of nearest neightors
Figure 2: Retrieval-basee augmentation (k = 3). Infor- mation for each aspect is derived from prior knowledge. including seed words and task-specific in-domain sen- tences having certain connections with seed words.
query to find high-quality sen-
shared representation learning, resulting in im- proved ACD performance. In detail, given an input sentence W = [wi....ww] with n words, we em- ploy a pretrained language model, e.g. RoBERTa (Liu et al., 2019) to generate shared sequence of contextualized embedding W [WI,..., w.] for the words (using the average of hidden vec- tors for word-pieces in the last layer of the pre- trained language model). Then, for each task t E {ACD, A,E, ATP), we feed the vector sequence W into feed-forward network to find the corresponding probability scores P Vt, E {ACD, ATP]. Note that, we treat the ACD problem sentence classification problem, while ATE and ATP are treated BIO sequence labeling problems.
Sentence Encoder: We leverage a paraphrastic encoder to generate similar representations for se- mantically similar sentences. In detail, the encoder is a Transformer pre-trained with masked language modeling (Kenton and Toutanova, 2019; Conneau and Lample, 2019), it is finetuned by a triplet loss [( y) = mar(0, a cos(xr, y) + (1,, )n)) on paraphrases from Natural Language Inference en- tailment pairs (Williams et al., 2018), Quora Ques- tion Pairs, round-trip translation (Wieting and Gim- pel, 2018), web paraphrases (Creutz, 2018), Open- Subtitles (Lison et al., 2018), and Europarl (Koehn, 2005) to maximize cosine similarity between simi- lar sentences. Positive pairs (ar, y) are either para- phrases or parallel sentences (Wieting et al., 2019), and the negative yn is selected to be the hardest in the batch.
Given the training dataset X,, which consists of task-specific in-domain data and augmented data, along with their corresponding pseudo labels of generated by Pseudo label Generation. The loss function is defined follows:
The task embedding uses shared paraphrastic encoder with sentence embed- dings, which used embed the prior knowledge about the target task (seed words, task-specific in- domain data with high-quality pseudo labels). In this task, given the prior knowledge, each repre- sentation of a seed word or sentence in the prior knowledge is considered a representation of the target task.
Task embedding:
L== 18-1X1 ΣE
Σ NEC,
dusi log(ptd) (3)
X1
in which Cr, is the set of labels of task 4 and A is a hyperparameter determining the weight of task 4.
We use task em- bedding as queries to retrieve a subset of the large sentence bank. For each task embedding, we select k nearest neighbors based on cosine similarity.
Unsupervised Data Retrieval:
4
Experiments
4.1 Datasets
Following previous works (Huang et al., 2020; Shi et al., 2021; Nguyen et al., 2021) conduct exper- iments on three benchmark datasets with different domains as described in Table
3.3 Classification
In this component, we train a neural network of multitask learning ATE, ATP, and ATE. We ex- peet that multitask learning can provide additional guidance for ACD from other tasks during the
Restauran//Laptop containing reviews about restauran//laptop (Huang et al., 2020). We use the

---

training dataset as the sentence bank and the Se- mEval training set (Pontiki et al., 2015, 2016) as task-specific in-domain data and dev set with a ra- tio of 0.83. The test set is taken from SemEval test set (Pontiki et al., 2015, 2016). Restaurant contains labeled data for all three tasks, where ACD has five aspect category types and ATP has two polarity types. Laptop only has labels for ACD with eight aspect category types, and Sentence-level ATP with two polarity types. For initial seed words, follow- ing (Huang et al., 2020; Nguyen al., 2021) we have five manual seed words and five automatic seed words for each label of ACD and Sentence- level ATP.
textblob extract nouns, noun phrases and ad- jectives. For retrieval-based data augmentation, we select: paraphrastic encoder pretrained by Du et al. (2021), and tune the number of nearest neigh- bors k in the range of [1, 20] with step 1. For the classification network, we use Roberta-large (Liu et al., 2019) with batch size 16, learning rate 1e-5, AdamW optimizer, weight decay 1e-5, task weights A is set to I, 0.8, 0.6 for ACD, ATE, and ATP spectively. We report the average performances of five different runs with random seeds.
4.3 Baselines
We compare ours to recent unsupervised ACD methods, including ABAE (He et al., 2017), CAt (Tulkens and van Cranenburgh, 2020), JASen (Huang et al., 2020), SSCL (Shi et al., 2021), UCE (Nguyen et al., 2021), and PCCT (Li et al., 2022). Currently, PCCT is state-of-the-art on all three datasets.
CitySearch containing reviews about restaurants (Ganu al., 2009), in which the test set only con- tains labeled ACD data with three aspect category types. Similarly to Restauran/LLatopp we use Se- mEval training set (Pontiki et al., 2014, 2015, 2016) as task-specific in-domain data and dev set with a ratio of 0.85. For seed words, similar to (Tulkens and van Cranenburgh, 2020), for Citysearch we use the aspect label words food, ambience, staff seed words.
Concerning the utilization of seed words to guide inference for larger LLMs, such as generative mod- els like GPT-3.5, we conducted an experiment in- volving model gpt-3. 5 5-turbo (from OpenAl) to infer aspect labels. The prompt used is described in detail in section A.
Following previous works, we remove the multi- aspect sentences from all datasets and only evaluate ATE and ATP on sentences with at least one OTE; multi-polarity sentences are removed when evalu- ating Sentence-level ATP.
Evaluation
First, we report the results of our framework on the ACD task in Table 3, and ATP and ATE in Table 4. During training, we utilize multitask learning for ACD, ATP, and ATE. However, due to limited labeled tasks in the test datasets, we only evalu- ate tasks with labels. Further details can be found in subsection 4.1. The results of prior methods were collected from the respective works. For ini- tial seed words, we employ the seed words recom- mended by (Huang et al., 2020) for the Restaurant and Laptop, as well as the seed words suggested by (Tulkens and van Cranenburgh, 2020) for City- Search. Following (Huang et al., 2020), evalu- ate the ACD performance by accuracy and macro- F1. Similarly, we evaluate ATP(s) (shorted for Sentence-level ATP) and ATP (shorted for Term- level ATP) by accuracy and macro-F1; and ATE by Fl FI-score. As can be seen, despite using less human supervision compared to manual mapping- based methods, seed word-based methods yield competi- tive results.
Dataset
Train
SemEval
Test
Citysearch 279,862
1474
1490
Restaurant
17,027
855
694
Laptop
14,683
476
307
Table 2: Statistics of the datasets.
4.2
HapeeeParameterr
For the learning framework, the best parameters and hyperparameters selected based the val- idation sets. For pseudo label generation, we use nitk' for tokenizing and Gensim² to train CBOW with embedding size of 200 and number of epochs of 10, the window of 10, negative sample size of 5. The connection measure 7 is tuned within the range of [0, 700), ATP context window size is tuned within the range of [20, 100], and m is set to 2. For the ATE task, use nitk for POS tagging and
hhtps///www.llkkorg Shaps://radimrccuuk...oo//aann

---

Restaurant
Laptop
Citysearch
Method
Acc macro-F1 Acc
macro-F1
Ace macro-F1
Methods with Manual Mapping
ABAE (2017)
67.3
45.3
59.8
56.2
85.7
77.5
SSCL (2021)
89.7
87.0
Methods with Aspect Name/Seed Words
CAt (2020)
66.3
46.2
58.0
58.6
83.6
82.5
JAsen (2020)
83.8
66.3
71.0
69.7
87.3
86.2
UCE (2021)
83.1
66.1
71.3
71.3
PCCT (2022)
85.3
79.2
74.3
73.4
90.6
89.8
ASeM 90.0
82.0
76.5
75.7
92.2
90.0
Table 3: The performance of ASeM on Aspect Category Detection.
Restaurant
Laptop ATP(s)
can be observed that removing SEC significantly impairs the accuracy ACD compared other tasks, clearly demonstrating the benefits of expand- ing seed words for ACD.
Method
ATP(s)
ATP
ATE
JASen
79.4
74.6
ASeM
80.5
33.7
48.2
78.4
W/o Retrieval-based Data Augmentation. We ig- nore data augmentation and follow previous works Huang et al. (2020); Nguyen et al. (2021) by train- ing classification the entire training data. As presented in the table, the -aug model exhibits significant performance decrease compared ASeM, particularly in out of tasks where per- forms the worst among the ablated versions. This clearly demonstrates the adverse impact of noise on the framework's 's accuracy and the limitations it imposes on the contributions of other components within the framework, well the effectiveness of the proposed method in addressing noise.
Table The performance of ASeM on Aspect Term Po- larity (ATP), Sentence-level ATP (ATP(s)) and Aspect Term Extraction (ATE). our work, Sentence-level ATP is not directly learned, but rather its labels inferred based on the polarity labels of terms in that sentence.
The results compared with GPT-3.5 are reported Table 5. detail, while GPT-3.5 performs well in terms of food, drinks, and service aspects and demonstrates superior accuracy, our model outper- forms in terms of location and ambience aspects as well as overall macro-F1 score. It is not surprising that GPT-3.5 demonstrates strong performance this dataset due to its immense underlying knowl- edge base. However, when considering factors like scalability, computational demands, complexity, and inference time, our method exhibits competi- tive results.
W/o Multi-task Learning. Next, ablate multi- task learning and train the three tasks (ACD, ATP,
Label names
Auto Extraction
Manual Extraction
100
Overall, ASeM demonstrates state-of-the-art per- formance Aspect Category Detection across var- ious domains, providing clear evidence of the ef- fectiveness of the proposed framework.
80
Apcuracy
80
Ablation Study: To investigate the impact of each proposed component for ASeM, we evaluate our ablated framework over the Restaurant dataset. Table 6 allows us assess the contribution of cach proposed component the overall performance of ASeM.
40
JASen UGE
ASeM
Figure 3: ACD performance seed words-based meth- ods with different initial seed word sets. Label names is derived from the aspect name (c.g. 'food") ), Auto ex- traction is automatically extracted from small labeled dataset (Angelidis and Lapata, 201 8), Manual extraction is manually extracted by experts (Huang et al., 2020).
W/o Seedword Enhanment Component. Ablating the SEC includes climinating the search for addi- tional seed words T2 and then assigning T, = 0.

---

FOOD
DRINKS
SERVICE
LOCATION
AMBIENCE
Overall
Method
F1
F1
F1
F1
F1
Acc
macro-Fi
ASeM
93.4
76.2
87.7
63.2
88.4
90.0
82.0
GPT-3.5
94.9
84.7
91.2
45.5
85.0
91.4
80.
Table 5: Performance ASeM vs. GPT-3.5 on Restaurant Dataset.
Methods
ACD
AsP(s)
ATP
ATE
improves. We suspect that the decline in ACD performance may be attributed to the mismapping of seed words (e.g. 'music" mapped to the aspect 'drink' instead of "servicee) and contextual ambi- guity when different aspect seed words co-occur (e.g- "we ended our great experience by having gulab jamun dessert (FOO) recommended by the (SErers ) SEC selectively adds words less connected with seed words, thereby reducing conflicts and minimizing the effect of mismapping. In addition, SEC aids in identifying previously un- extracted aspect terms (e.g. martinis, hot dogs), as evidenced by the improved ATE performance, providing insight into how SEC enhances the per- formance of ACD.
ASeM
90.0
80.5
33.7
48.2
-SEC
86.3
79.7
31.7
46.6
-aug
86.3
74.4
28.9
46.7
-joint
88.0
79.3
32.4
42.8
Table Ablation study
ATE) separately. While inflicting less damage to ACD compared to other ablated versions, the -joint exhibits substantial performance benefits for all component tasks through the straightforward ap- proach of joint learning. It results in 2% increase in accuracy for ACD, 1.3% boost in Fl-score for ATP, and a remarkable 5.4% improvement in F1- score for ATE.
In general, the elimination of any component from ASeM would significantly hurt the performance, clearly demonstrating the benefits of the proposed components.
6.2 Keyword distinction
In this section, we carry out experiments on adding weights to distinguish keywords/sentences based on uncertain connections, while ASeM considered the role of seed words and uncertain keywords equal when they contribute aspect category rep- resentation. Table 7 shows that our approach is simpler and more efficient than prior works and optimally setting weights to preserve inherent data properties is challenging. detail, weighting by
6 Analysis
In this section, we thoroughly analyze the perfor- mance of our framework concerning the quality of seed words and training data.
6.1 The quality of seed words
Firstly, we examine the effect of the initial seed words on the performance of our framework. Fig- ure 3 shows the accuracy of our framework using different initial seed word sets on the Restaurant dataset, comparing them with other saat-offthe-aa methods based on seed words. As can be seen, both JASen and UCE exhibit significant variations in per- formance when changing the initial seed word sets, highlighting their strong dependence on the quality of initial seed words. Meanwhile, ASeM demon- strates good adaptation ability to the quality of ini- tial seed words, delivering promising results across all three of initial seed words. Secondly, dis- cuss the effectiveness of the additional seed words generated by SEC, compared other methods of adding seed words. The results are reported in Fig- ure While baselines fluctuate, SEC consistently
repuiiely
POST
ARC
popularty
RCET
50c
48
ATE Accuraty
88
87
86
84 Number adsttional aoed words
45
20
40
60
50
Number addidional seed words
Figure 4: ACD and ATE performances with differ- seed word addition strategies. Our baselines: dom adds seed words randomly from the vocabulary. mapped their aspects by Eq.2, while popularity se- lects the most frequently occurring important words (noun/adjective) of each aspect based on a small labeled dataset (dev set). PCCT a set additional seed words proposed by (Li et al., 2022), extracted from the vocab- ulary of pre- pre-trained model.

---

ACD
Methods
Acc
macro-Fl
0.9
ASeM
90.0
82.0
weighted term level
88.6
77.5
weighted sentence level
89.1
75.3
0.8
w/o aug + weighted sen- tence level
86.4
74.6
0.7
Accuracy
macro-F1
Table Experimental Results for Weighted Seed Word and Sentence Assignment on Restaurant dataset. weighted term level: A variant of AseM where the aspect representation is computed by a weighted sum of seed word representations (following (Angelidis and Lapata, 2018)). weighted sentence level: A variant of AseM where the connection score multiplied into loss function (following (Nguyen al., 2)21)). w/o aug + weighted sentence level is the weighted sentence level without data augmentation.
0.6
2
3
6
7
8
10
neighbors
Figure 5: Performance of ACD with increasing num- ber of neighbors. More neighbors result in a larger training data set.
model accurately associates the sentiment of sentence with the corresponding aspect, thereby enhancing ACD performance demonstrated by rows and of Table 8.
confidence scores does consistently yield im- proved results or seed words that appear more fre- quently do necessarily play a more important role, as further illustrated by Figure 4.
6.4 Retrieval-based Augmentation
6.3 Qualitative analysis
In this subsection, we examine the impact of trans- forming the unsupervised learning problem based on seed words into data augmentation task for a low-resource task. As observed Figure 5, our framework's performance shows a substantial im- provement in the initial phase but gradually de- clines afterward. This decline can be attributed excessive increase in neighbors, which leads to the inclusion of misaligned data that does not connect well with the target task's prior knowledge (e.g. seed words). Consequently, the pseudo-label generation becomes insufficient to provide accurate predictions, resulting in compromised classifica- tion and decreased performance.
We conducted a comparison (Table 8) of the per- formance improvement of our model against the UCE baseline and observed that the progress stems from two main factors. First, our model accurately identifies aspect terms present in sentences, which in turn accurately determines the ACD, shown in rows and 2 of Table 8. Our finding is further supported by Figure 4, which illustrates that the model's s performance correlates with the accuracy Aspect Term Extraction (ATE). Additionally,
Sentences
UCE
ASeM
Save room for scrump- ambience food tious desserts. This place is famous for location food their breakfast.
Conclusion
In this work, we propose a novel framework for ACD that achieves three main goals: (1) enhane- ing aspect understanding and reducing reliance on initial seed words, (2) effectively handling noise in the training data, and (3) self-boosting supervised signals through multitask-learinnn three unsuper- vised tasks (ACD, ATE, ATP) to improve perfor- mance. The experimental results demonstrate that our model outperforms the baselines and achieves state-of-the-an performance on three benchmark datasets. In the future, we plan to extend our frame- work to address other unsupervised problems.
The waiters are very ex- perienced and helpful with pairing your drink choice to your food tastes vice versa.
food service
Can't 't believe how an ex- pensive NYC restaurant can so disrespectful to its clients.
location service
Table 8: Examples of improved accuracy by ASeM

---

Limitations
International Conference on Language Resources and Evaluation (LREC 2018).
Although our experiments have proven the effec- tiveness of our proposed method, there are still some limitations that can be improved in future work. First, our process of assigning keywords to their relevant aspects is not entirely accurate. Fu- ture work may explore alternatives to make this process more precise. Second, through the anal- ysis of the results, we notice that our framework predicts the aspect categories of sentences with im- plicit aspect terms less accurately than sentences with explicit aspect terms. This is because we pri- oritize the presence of aspect terms in sentences when predicting their aspect categories, which can be seen in the pseudo-label generation. However, sentences with implicit aspect terms do not contain aspect terms, or even contain terms of other aspects, leading to incorrect predictions. For example, the only beverage we did receive was water in dirty glasses was predicted as DRINKS instead of the golden aspect label SERVICE. Future works may focus more on the context of sentences to make bet- ter predictions for sentences with implicit aspect terms.
Steve -onnen,oowsend, Yun Zhou, and Bruce Croft. 2002. Predicting query performance. In Proceedings of the 25th annual international ACM SIGIR confer- ence on Research and development in information retrieval, pages 299 199-306.
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaud- hary, Onur Celebi, Michael Auli, Veselin Stoyanov, and Alexis Conneau. 2021. Self-training improves pre-training for natural language understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 5408-5418.
Kawin Ethayarajh. 2018. Unsupervised random walk sentence embeddings: A strong but simple baseline. In Proceedings of The Third Workshop on Represen- tation Learning for NLP, pages 91 1000.
Gayatree Ganu, Noemie Elhadad, and Amélic Marian. 2009. Beyond the stars: improving rating predictions using review text content. In WebDB, volume 9, pages 1-6. Citeseer.
Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2017. An unsupervised neural attention model for aspect extraction. In Proceedings of the 55th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages Vancouver, Canada. Association for Com- putational Linguistics.
References
Stefanos Angelidis and Mirella Lapata. 2018. Sum- marizing opinions: Aspect extraction meets senti- ment prediction and they both weakly supervised. Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 3675-3686, Brussels, Belgium. Association for Com- putational Linguistics.
Jiaxin Huang, Yu Meng, Fang Guo, Heng Ji, and Ji- awei Han. 2020. Aspect-hased sentiment analysis by aspect-sentiment joint embedding. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6989-6999 -699.
Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. simple but tough-to-beat baseline for sentence em- beddings. In International conference learning representutions.
Giannis Karamanolakis, Daniel Hsu, and Luis Gra- vano. 2019. Leveraging just few keywords for fine fineigrained aspect detection through weakly super- vised co-training. In Proceedings of the 2019 Confer- ence Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 461 1-4621, Hong Kong, China. Association for Computational Linguistics.
Samuel Brody and Noemie Elhadad. 2010. An unsu- pervised aspect-sentimen model for online reviews. In Human language technologies: The 2010 annual conference of the North American chapter of the asso- ciation for computatonall linguistics, pages 802-11.
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova, 2019. Bert: Pre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT,, pages 4171-4186.
Zhiyuan Chen, Arjun Mukherjee, and Bing Liu. 2014. Aspect extraction with automated prior knowledge learning. In Proceedings of the 52nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 347-358.
Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of machine translation summit x: papers, pages 79-86.
Alexis Conneau and Guillaume Lample. 2019. Cross- lingual language model pretraining. Advances in neural information processing systems, 32.
Yazheng Li, Pengyun Wang, Yasheng Wang, Yong Dai, Yadao Wang, Lujia Pan, and Zenglin Xu. 2022. Leveraging only the category name for aspect de- tection through prompt-based constrained clustering.
Mathias Creutz. 2018 Open subtitles paraphrase corpus for six languages. In Proceedings of the Eleventh

---

In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1552-1364, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Maria Pontiki, Dimitris Galanis, John Pavlopoulos. Har- ris Papageorgiou, lon Androutsopoulos, and Suresh Manandhar, 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (Se- mEval 2014), pages 27-35, Dublin, Ireland. Associa- tion for Computational Linguistics.
Pierre Lison, Jörg Tiedemann, and Milen Kouylekov, 2018. Opensubritles2188: Statistical rescoring of sentence alignments in large, noisy parallel corpora. In Proceedings of the Eleventh International Confer- ence on Language Resources and Evaluation (LREC 2018).
Tian Shi. Liuqing Li, Ping Wang, and Chandan Reddy. 2021. A simple and effective self-supervised con- trastive learning framework for aspect detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13815-13824.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint weXiv:1907.1166
Stéphan Tulkens and Andreas van Cranenburgh. 2020. Embarrassingly simple unsupervised aspect extrac- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3182-313 3187, Online. Association for Computational Linguisties.
Ling Luo, Xiang Ao, Yan Song, Jinyao Li, Xiaopeng Yang, Qing He, and Dong Yu. 2019. Unsupervised neural aspect extraction with sememes. In IJCAI, pages 5129-5129.
John Wieting and Kevin Gimpel. 2018. ParaNMT-SOM: Pushing the limits of paraphrastic sentence embed- dings with millions of machine translations. In Pro- ceedings of the 56th Annual Meeting of the Associa- tion for Computational Linguistics ( (Volume I: Long Pupers), pages 451-462. Melbourne, Australia. As- sociation for Computational Linguistics.
Tomas Mikolow, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pages 746-751, Atlanta, Georgia. Association for Computational Linguistics.
John Wieting, Kevin Gimpel, Graham Neubig, and Tay- lor Berg- Kirkpatrick. 2019. Simple and effective paraphrastic similarity from parallel translations. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4602- 4608.
Arjun Mukherjee and Bing Liu. 2012. Aspect extraction through semi-supervised modeling. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers),
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A croad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guisties: Human Language Technologies, Volume (Long Papers), pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguis- tics.
pages 339-348.
Thi-Nhung Nguyen, Kiem-Hieu Nguyen, Young-In Song, and Yban-Dung Cao. 2021. An uncertainty- aware encoder for aspect detection. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 797-806.
Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Moham- mad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clereq, Véronique Hoste, Marianna Apidianaki, Xavier Tannier, Na- talia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel, Salud María Jiménez-Zafra, and Gülşen Eryigit. 2016. SemEval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop Semantic Evaluation (SemEval-2016), pages 19-30, San Diego California. Association for Computational Linguistics.
A GPT-3.5 prompt
In the experiment with GPT-3.5, we use the follow- ing prompt:
Map the following sentence to the
appropriate aspect from the provided list of aspects:
Sentence: [sentence)
Aspect options: [List of n aspects}
Maria Pontiki, Dimitris Galanis, Haris Papagcorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015, ComEval-2015 task 12: Aspect based sentiment anal- ysis. In Proceedings of the 9th International Work- shop on Semantic Evaluation (SemEval 2015), pages 486-495, Denver, Colorado. Association for Compu- tational Linguistics.
For each aspect, you have the following prior knowledge words: [Aspect 1): [List of seed words}
(Aspect n): [List of seed words}
Your task is to associate the sentence

---

with the most suitable aspect using the provided prior knowledge.


Reviews:
[{"review_id": "ZSs3Dg6ogK", "reviewer": "Reviewer_3mKJ", "summary": "", "questions": "a) Does your method consider the difference between keywords with certain connections and keywords with uncertain connections? Do you think adding weights to distinguish keywords from uncertain connections (similar to Huang et al., 2020; Nguyen et al., 2021 mentioned at 072) can help the learning, or have you experimented with that?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for acknowledging the effectiveness of our method and the thoroughness of the experiments that support our claims. We appreciate your insights and would like to address the concerns you've raised regarding the potential reasons for rejection.\n \n> There are similar methods in other unsupervised learning tasks....\n\n**Our Response**:\n\n**(1) Seedword enhancement component**:\nWe want to emphasize that previous unsupervised ACD works have not been consistent in using the same prior knowledge (i.e. seed words). They have also shown significant variations in performance when using different sets of seed words (as observed in Figure 3). We believe that addressing this issue is crucial for better understanding how seed words affect performance and to compare ACD studies more effectively. Moreover, this Seedword Enhancement Component is executed through a meticulous analysis of the relationships between seed words and sentences, as well as among the seed words themselves (lines 83-97, Algorithm 1). As a result, it not only ensures consistent quality for generating pseudo-labels from any given seed word inputs but also enhances overall performance (as demonstrated in Table 5).\n\n**(2) Retrieval-based Data Augmentation**: \nWe would like to highlight that the data augmentation technique applied to this unsupervised aspect category detection (ACD) task is unique in the following ways:\n- **Purpose of use**: Data augmentation is typically utilized for two main purposes:  (a) Increasing training data diversity without additional crawling, through techniques like synonym replacement [1,2,3]. (b) Extracting more knowledge from external sources [4,5].\nIn this work, we use the data augmentation technique to **remove low-quality training data**. \n- **Application to unsupervised ACD**: We are the first to approach the challenge of addressing noise in unsupervised ACD as a data augmentation problem. Unlike previous unsupervised ACD methods that train on all available unlabeled data while trying to limit the harm of inaccurate pseudo-labels based on a certain criteria (e.g. confidence of classification model [6]), we extract a small subset from the training data to act as a task-specific in-domain data. We then proceed to collect more high-quality data from the rest through its connection with seed words by the data-augmentation technique. (line 221-235, sub-section 3.5)\n\nIntuition: In-domain sentences predictable from the seed words have strong connections in content and context to those seed words. In contrast, out-of-domain sentences and those the model cannot accurately predict from the seed words often contain irrelevant or minimal relevant information. Thus, using a data augmentation technique in this way simultaneously increases useful in-domain data, while preventing an increase of irrelevant examples and inaccurate pseudo-labels.\n\n>  the impact of this paper might be limited\n\n**Our Response**:\n\nAs this problem extends beyond ACD, our techniques could be adapted to aspect-based sentiment analysis [7], summarization [8], and related weakly supervised tasks [9] where managing noise and finding supervised signals are common challenges.\n\n> ...difference between keywords...Do you think adding weights to distinguish keywords...\n\n**Our Response**:\n\nWe considered the role of seed words and uncertain keywords as equal (line 252). Our in-house experiments about adding weights to distinguish keywords/sentences are reported in Table 6.\n\n|            **Methods**            |  **ACD** |              |\n|:--------------------------------- |--------: | ------------:|\n|                                   | **Acc**  | **macro-F1** |\n| ASeM                              | **90.0** | **82.0**     |\n| weighted term level               | 88.6     | 77.54        |\n| weighted sentence level           | 89.1     | 75.3         |\n| w/o aug + weighted sentence level | 86.4     | 74.6         |\n\nTable 6. Experimental Results for Weighted Seed Word and Sentence Assignment on Restaurant dataset.  **weighted term level**: A variant of AseM where the aspect representation is computed by a weighted sum of seed word representations (following [10]). **weighted sentence level**: A variant of AseM where the connection score is multiplied into the loss function (following [11]). **w/o aug + weighted sentence level** is the weighted sentence level without data augmentation.\n\nThe results show that our approach is simpler and more efficient than previous studies because optimally setting weights to preserve inherent data properties is challenging. As Figure 4 shows, seed words that appear more frequently do not necessarily play a more important role. Additionally, Table 6 demonstrates that weighting by confidence scores does not consistently yield improved results. Setting weights to distinguish seed words or pseudo-label quality remains an open challenge. We will expand on this issue in a revised version of the paper.\n\n> It would be easier to read if the first paragraph in section one can contain a direct description of Table 1.\n\n**Our response**:\n\nThank you for your suggestion. In the revised version, we will include a direct description of Table 1 in the first paragraph of section 1 for improved readability.\n\nCitations:\n \n[1] Asai, A. and Hajishirzi, H., 2020, July. Logic-Guided Data Augmentation and Regularization for Consistent Question Answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 5642-5650).\n\n[2] Wei, J. and Zou, K., 2019, November. EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 6382-6388).\n\n[3] Wei, J., Huang, C., Vosoughi, S., Cheng, Y. and Xu, S., 2021, June. Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 5493-5500).\n\n[4] Du, J., Grave, É., Gunel, B., Chaudhary, V., Celebi, O., Auli, M., Stoyanov, V. and Conneau, A., 2021, June. Self-training Improves Pre-training for Natural Language Understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 5408-5418).\n\n[5] Xia, M., Kong, X., Anastasopoulos, A. and Neubig, G., 2019, July. Generalized Data Augmentation for Low-Resource Translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 5786-5796).\n\n[6] Huang, J., Meng, Y., Guo, F., Ji, H. and Han, J., 2020, November. Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 6989-6999).\n\n[7] Lei Zhang and Bing Liu. 2014. Aspect and Entity Extraction for Opinion Mining, pages 1–40. Springer Berlin Heidelberg, Berlin, Heidelberg.\n\n[8] Stefanos Angelidis and Mirella Lapata. 2018. Summarizing opinions: Aspect extraction meets sentiment prediction and they are both weakly supervised. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3675–3686, Brussels, Belgium. Association for Computational Linguistics.\n\n[9] Karamanolakis, G., Mukherjee, S., Zheng, G. and Hassan, A., 2021, June. Self-Training with Weak Supervision. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 845-863).\n\n[10] Stefanos Angelidis and Mirella Lapata. 2018. Summarizing opinions: Aspect extraction meets sentiment prediction and they are both weakly supervised. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3675–3686, Brussels, Belgium. Association for Computational Linguistics\n\n[11] Thi-Nhung Nguyen, Kiem-Hieu Nguyen, Young-In Song, and Tuan-Dung Cao. 2021. An Uncertainty-Aware Encoder for Aspect Detection. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 797–806, Punta Cana, Dominican Republic. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors provide clear explanations about the methodology. The writing is easy to understand. The proposed method is a reasonable solution to the specific mentioned problem (unsupervised selection of keywords in aspect category detection). Aside from the overall accuracy and macro-F1 (outperforms existing method with large margins), they also provide many additional analyses and experiments which are important evidence for their claim (e.g., with more keywords selected by the proposed method, the accuracy increased).", "reasons_to_reject": "The main contribution of this paper is a solution to a very specific sub-problem, which is expanding the keyword set for unsupervised aspect category detection, of aspect sentiment analysis. And the problem isn’t unique to the aspect category detection. There are similar methods in other unsupervised learning tasks. Therefore, the impact of this paper might be limited.", "questions_to_authors": "a) Does your method consider the difference between keywords with certain connections and keywords with uncertain connections? Do you think adding weights to distinguish keywords from uncertain connections (similar to Huang et al., 2020; Nguyen et al., 2021 mentioned at 072) can help the learning, or have you experimented with that?", "paper_topic_and_main_contributions": "This paper proposes a method to tackle the problem of seed word selection in unsupervised aspect category detection. The proposed method adds new keywords with uncertain connections to the initial seed word set (Pseudo-label generation). Secondly, it leverages a retrieval-based data augmentation technique to search for augmented training data in unannotated data banks. Lastly, it deploys multi-task learning with tasks having similar nature, e.g., aspect term extraction and aspect term polarity tasks, which are also trained based on unsupervised pseudo-label generation.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "a) It would be easier to read if the first paragraph in section one can contain a direct description of Table 1."}, "extra_text": {}}, {"review_id": "kWky462zts", "reviewer": "Reviewer_9v2c", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We're grateful that you've recognized the efficacy of our approach. We appreciate your insights and would like to address the concerns you've raised regarding the potential reasons for rejection. \n\n>  I do not see a lot of technical novelties and insights\n\n**Our response**:\n\n**(1) Seedword enhancement component**: \nWe want to emphasize that previous unsupervised ACD works have not been consistent in using the same prior knowledge (i.e. seed words). They have also shown significant variations in performance when using different sets of seed words (as observed in Figure 3). We believe that addressing this issue is crucial for better understanding how seed words affect performance and to compare ACD studies more effectively. Moreover, this Seedword Enhancement Component is executed through a meticulous analysis of the relationships between seed words and sentences, as well as among the seed words themselves (lines 83-97, Algorithm 1). As a result, it not only ensures consistent quality for generating pseudo-labels from any given seed word inputs but also enhances overall performance (as demonstrated in Table 5).\n\n**(2) Retrieval-based Data Augmentation**: \nWe would like to highlight that the data augmentation technique applied to this unsupervised aspect category detection (ACD) task is unique in the following ways:\n- **Purpose of use**: Data augmentation is typically utilized for two main purposes:  (a) Increasing training data diversity without additional crawling, through techniques like synonym replacement [1,2,3]. (b) Extracting more knowledge from external sources [4,5].\nIn this work, we use the data augmentation technique to **remove low-quality training data**. \n- **Application to unsupervised ACD**: We are the first to approach the challenge of addressing noise in unsupervised ACD as a data augmentation problem. Unlike previous unsupervised ACD methods that train on all available unlabeled data while trying to limit the harm of inaccurate pseudo-labels based on a certain criteria (e.g. confidence of classification model [6]), we extract a small subset from the training data to act as a task-specific in-domain data. We then proceed to collect more high-quality data from the rest through its connection with seed words by the data-augmentation technique. (line 221-235, sub-section 3.5)\n\nIntuition: In-domain sentences predictable from the seed words have strong connections in content and context to those seed words. In contrast, out-of-domain sentences and those the model cannot accurately predict from the seed words often contain irrelevant or minimal relevant information. Thus, using a data augmentation technique in this way simultaneously increases useful in-domain data, while preventing an increase of irrelevant examples and inaccurate pseudo-labels.\n\nOur experiments demonstrate that our approach is both simpler and more efficient than previous studies. This is because determining the right weights to maintain the natural characteristics of the data is a complex task. For instance, seed words that occur more often might not always be more significant (see Figure 4).\n\n> I'm not sure if LLM can help with this task. It will be interesting to see an LLM baseline.\n\n**Our response**:\n\nWe highly appreciate your suggestions, and we also agree that incorporating additional concerns with state-of-the-art Large Language Models (LLMs) would enhance the significance of our research and open up new avenues for future exploration. We will include the following discussion in the revised version.\n\nFirst and foremost, we want to emphasize that the Roberta model we are utilizing serves as a foundational and reasonable LLM, particularly in ensuring fairness with related works and minimal computational resource requirements. It offers ease of expansion and reuse.\n\nRegarding the extension of using larger LLMs, there are several distinct approaches:\n- Concerning the utilization of seed words to guide inference for larger LLMs, such as generative models like GPT-3.5, we conducted an experiment involving GPT-3.5* (from OpenAI) to infer aspect labels. The outcomes are detailed in Table 7.\n\n| **Method** | **FOOD** | **DRINKS** | **SERVICE** | **LOCATION** | **AMBIENCE** | **Overall** |  |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  | **F1** | **F1** | **F1** | **F1** | **F1** | **Acc** | **Macro_f1** |\n| ASeM | 93.4 | 76.2 | 87.7 | **63.2** | **88.4** | 90.0 | **82.0** |\n| GPT-3.5 | **94.9** | **84.7** | **91.2** | 45.5 | 85.0 | **91.4** | 80.3 |\n\nTable 7 Performance of ASeM vs. GPT-3.5 on Restaurant Dataset. \n\nAs can be observed, while GPT-3.5 performs well in terms of food, drinks, and service aspects and demonstrates superior accuracy, our model outperforms in terms of location and ambience aspects as well as overall macro-F1 score.\n\nIt's not surprising that GPT-3.5 demonstrates strong performance on this dataset due to its immense underlying knowledge base. However, when considering factors like scalability, computational demands, complexity, and inference time, our method exhibits competitive results.\n\n**\\* _In this experiment, we use the following prompt:_**\n\n```\nMap the following sentence to the appropriate aspect from the provided list of n aspects:\nSentence: [Insert sentence here]\nAspect Options: [List of n aspects]\nFor each aspect, you have the following prior knowledge words:\n[Aspect 1]: [List of seed words for Aspect 1]\n[Aspect 2]: [List of seed words for Aspect 2]\n...\n[Aspect n]: [List of seed words for Aspect n]\nYour task is to associate the sentence with the most suitable aspect using the provided prior knowledge.\n```\n\n- As a potential avenue for future work, similar to our existing approach of weakly supervised learning, the prospect of substituting Roberta with a more robust LLM offers the potential for significant improvements.\n\n\n> It would be better to provide some qualitative analyses to discuss which types of cases could be better addressed with the new framework\n\n**Our response**:\n\nWe agree that adding qualitative analyses would provide further clarity on the instances where our model outperforms. We will incorporate these analyses into the revised version.\n\nWe conducted a comparison of the performance improvement of our model against the UCE baseline (Table 8) and observed that the progress stems from two main factors. First, our model accurately identifies aspect terms present in sentences, which in turn accurately determines the ACD, as shown in rows 2 and 3 of Table 8. Our finding is further supported by Figure 4, which illustrates that the model's performance correlates with the accuracy of Aspect Term Extraction (ATE). Additionally, our model accurately associates the sentiment of a sentence with the corresponding aspect, thereby enhancing ACD performance as demonstrated by rows 4 and 5 of Table 8.\n\n| **Sentences** | **UCE** | **ASeM** |\n|---|---|---|\n| Save _room_ for scrumptious **desserts**. | ambience | food |\n| This _place_ is famous for their **breakfast**. | location | food |\n| The **waiters** are very **experienced** and helpful with pairing your _drink_ choice to your _food_ tastes or vice versa. | food | service |\n| Can’t believe how an **expensive** _NYC_ restaurant can be so **disrespectful** to its clients. | location | service |\n\nTable 8: Examples of Improved Accuracy by ASeM\n\n\nCitations:\n\n[1] Asai, A. and Hajishirzi, H., 2020, July. Logic-Guided Data Augmentation and Regularization for Consistent Question Answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 5642-5650).\n\n[2] Wei, J. and Zou, K., 2019, November. EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 6382-6388).\n\n[3] Wei, J., Huang, C., Vosoughi, S., Cheng, Y. and Xu, S., 2021, June. Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 5493-5500).\n\n[4] Du, J., Grave, É., Gunel, B., Chaudhary, V., Celebi, O., Auli, M., Stoyanov, V. and Conneau, A., 2021, June. Self-training Improves Pre-training for Natural Language Understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 5408-5418).\n\n[5] Xia, M., Kong, X., Anastasopoulos, A. and Neubig, G., 2019, July. Generalized Data Augmentation for Low-Resource Translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 5786-5796).\n\n[6] Huang, J., Meng, Y., Guo, F., Ji, H. and Han, J., 2020, November. Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 6989-6999).\n\n[7] Lei Zhang and Bing Liu. 2014. Aspect and Entity Extraction for Opinion Mining, pages 1–40. Springer Berlin Heidelberg, Berlin, Heidelberg.\n\n[8] Stefanos Angelidis and Mirella Lapata. 2018. Summarizing opinions: Aspect extraction meets sentiment prediction and they are both weakly supervised. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3675–3686, Brussels, Belgium. Association for Computational Linguistics.\n\n[9] Karamanolakis, G., Mukherjee, S., Zheng, G. and Hassan, A., 2021, June. Self-Training with Weak Supervision. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 845-863).\n\n[10] Stefanos Angelidis and Mirella Lapata. 2018. Summarizing opinions: Aspect extraction meets sentiment prediction and they are both weakly supervised. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3675–3686, Brussels, Belgium. Association for Computational Linguistics\n\n[11] Thi-Nhung Nguyen, Kiem-Hieu Nguyen, Young-In Song, and Tuan-Dung Cao. 2021. An Uncertainty-Aware Encoder for Aspect Detection. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 797–806, Punta Cana, Dominican Republic. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The method is straightforward and the motivation is clear. It can achieve good performance over a few baselines included in this paper.", "reasons_to_reject": "1. Data augmentation and multi-task learning are both well-studied techniques. I'm not surprised to apply these techniques to a new application. However, I do not see a lot of technical novelties and insights.\n2. I'm not sure if LLM can help with this task. It will be interesting to see an LLM baseline.\n3. It would be better to provide some qualitative analyses to discuss which types of cases could be better addressed with the new framework.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a novel framework for unsupervised aspect category detection. The task is to identify the categories, specific aspects, and polarities of the aspects discussed. The three sub-tasks can be named ACD, ATE, and ATP respectively. The authors propose two main improvements over the previous work, one is data augmentation and the other is multi-task learning. Experiments on three benchmark datasets demonstrate the effectiveness of the proposed framework.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZLIvZ5yuSK", "reviewer": "Reviewer_UyNx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We appreciate your insights and would like to address the concerns you've raised regarding the potential reasons for rejection. \n\n> ...not be entirely convincing to demonstrate the effectiveness across different domains.\n\n**Our response**:\n\nWhile it's true that both the Restaurant and CitySearch datasets revolve around restaurant reviews, they exhibit variations in data sources, review styles, and aspect labels. Moreover, we have experimented with the **_Laptop_** domain, as detailed in lines 396-411, providing further evidence of our method's effectiveness across diverse domains.\n\n> The paper does not mention the specific computational resources used for their experiments and the time complexity of the method.\n\n**Our response**:\n\nThe computational resources employed for our experiments included the utilization of the RoBERTa-large model, running on a Tesla V100 32GB GPU. The training duration varied based on the dataset sizes of different experiments, averaging around 10 minutes/experiment. The model employed in our work is a prevalent and extensively utilized architecture, ensuring straightforward reproducibility. Consequently, we omitted this detail from our paper.\n\n> The performance of the ACD task heavily depends on the inconsistent quality of pseudo-label generation. The accuracy of the pseudo-labels can be impacted by the presence of noisy data or ambiguous sentences, leading to potential misclassification.\n\n**Our response**:\n\nWe understand that the quality of pseudo labels has an impact on the model's performance. However, our contributions are focused on tackling this matter.\n- **SEC**, outlined in lines 81-97, 142-147, and Algorithm 1, introduces enhancements to aspect comprehension and reduces dependence on the initial seed word quality. This results in improved and consistent quality of pseudo-labels with any initial seed words. Our assertion is substantiated by the compelling evidence presented in Table 5 and Figures 3, and 4.\n- **Retrieval-based Data Augmentation**, detailed in lines 97-119, 148-154, and section 3.2, is designed to automatically remove out-of-domain data and inaccurate pseudo-labels. This proactive filtering mitigates their detrimental impact on performance. Table 5 and Figure 5 provide empirical validation for our proposition.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper introduces the Seedword Enhancement Component (SEC) that expands the set of seed words to enhance their quality. Then, ASeM shows good adaptation to the quality of initial seed words during the study. Even with varying initial seed word sets, the model delivers promising results, reducing the dependency on high-quality initial seed words. \n\n2. The experimental results demonstrate that ASeM outperforms several state-of-the-art unsupervised ACD methods on three benchmark datasets, establishing it as an effective framework in ACD tasks. \n\n3. ​​The paper employs multitask learning for Aspect Term Extraction (ATE), Aspect Term Polarity (ATP), and Aspect Category Detection (ACD) tasks, which offer extra guidance for ACD by leveraging shared representation learning. As a result, this strategy leads to enhanced performance in all three tasks. ", "reasons_to_reject": "1. While the paper asserts that all three benchmark datasets are from different domains, it is important to note that the Restaurant and CitySearch datasets both revolve around restaurant reviews. This similarity in the domain might not be entirely convincing to demonstrate the effectiveness across different domains. \n\n2. The paper does not mention the specific computational resources used for their experiments and the time complexity of the method. The lack of such information on the computational resources used can make it difficult for readers to assess the scalability and reproducibility of the proposed framework. \n\n3. The performance of the ACD task heavily depends on the inconsistent quality of pseudo-label generation. The accuracy of the pseudo-labels can be impacted by the presence of noisy data or ambiguous sentences, leading to potential misclassification.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper presents ASeM, an unsupervised framework for Aspect Category Detection (ACD). ASeM addresses issues related to the quality of initial seed words and sample noise in the dataset based on previous and recent works. The contributions of the paper include enhancing aspect comprehension and reducing reliance on the quality of initial seed words, introducing retrieval-based data augmentation to handle training data noise, leveraging multi-task learning for improved ACD, and demonstrating superior performance compared to previous methods through comprehensive experiments on standard datasets. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers are in general agreement that this paper could be accepted in EMNLP-2023. The reviewers also raised some concerns about this submission. The authors have provided a rebuttal that appeared to alleviate some concerns. A suggestion was made to accept the paper as Main Conference. The reviewers' concerns are also suggested to be addressed in the final version.
Decision: Accept-Main