Adaptive Structure Induction for Aspect-based Sentiment Analysis with Spectral Perspective
Authors: Hao Niu
Yun Xiong
Xiaosu Wang
Wenjing Yu
Yao Zhang
Zhonglei Guo
Venue: EMNLP 2023 Findings (2023)
Keywords: Sentiment analysis
Spectral analysis
Structure induction
TL;DR: Exploring the role of scale information in the structure induction on the ABSA task from a spectral perspective.

Abstract: Recently, incorporating structure information (e.g. dependency syntactic tree) can enhance the performance of aspect-based sentiment analysis (ABSA). However, this structure information is obtained from off-the-shelf parsers, which is often sub-optimal and cumbersome. Thus, automatically learning adaptive structures is conducive to solving this problem. In this work, we concentrate on structure induction from pre-trained language models (PLMs) and throw the structure induction into a spectrum perspective to explore the impact of scale information in language representation on structure induction ability. Concretely, the main architecture of our model is composed of commonly used PLMs (e.g. RoBERTa, etc), and a simple yet effective graph structure learning (GSL) module (graph learner + GNNs). Subsequently, we plug in spectral filters with different bands respectively after the PLMs to produce filtered language representations and feed them into the GSL module to induce latent structures. We conduct extensive experiments on three public benchmarks for ABSA. The results and further analyses demonstrate that introducing this spectral approach can shorten Aspects-sentiment Distance (AsD) and be beneficial to structure induction. Even based on such a simple framework, the effects on three datasets can reach SOTA (state of the art) or near SOTA performance. Additionally, our exploration also has the potential to be generalized to other tasks or to bring inspiration to other similar domains.

Adaptive Structure Induction for Aspect-based Sentiment Analysis with Spectral Perspective
Hao Niu, Yun Xiong; Xiaosu Wang, Wenjing Yu, Yao Zhang, Zhonglei Guo
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University {hniu18, yunx, xswang19, yaozhang, guoz118 @euudan. www.dneeeu....... [wjyu21} Jem. fudan.edu cn
Abstract
of the task is detect the sentiment polarities (i.e.. POSITIVE, NEGATIVE, NEUTRAL. of all given as- pects. Given the sentence "The decor not a spe- cial at all but their amazing food makes up for it" and corresponding aspects "decor" and "food", the sentiment polarity towards "decor" is NEGATIVE, whereas the sentiment for "food" is POSITIVE.
Recently, incorporating structure information (e.g. dependency syntactic tree) can enhance the performance of aspect-based sentiment analysis (ABSA). However, this structure infor- mation is obtained from off-the-shelf parsers, which often sub-optimal and cumbersome. Thus, automatically learning adaptive struc- tures is conducive to solving this problem. In this work, concentrate structure induc- tion from pre-trained language models (PLMs) and throw the structure induction into a spec- trum perspective to explore the impact of scale information in language representation on struc- ture induction ability. Concretely, the main architecture of model is composed of com- monly used PLMs (e.g.. RoBERTa, etc.), and a simple yet effective graph structure learning (GSL) module (graph learner GNNs). Subse- quently, we plug in Frequency Filters with ferent bands after the PLMMs produce filtered language representations and feed them into the GSL module induce latent structures. conduct extensive experiments three public benchmarks for ABSA. The results and further analyses demonstrate that introducing this spec- tral approach can shorten Aspects--entiment Distance (AsD) and be beneficial to structure induction. Even based such simple frame- work, the effects on three datasets can reach SOTA (statc-of-the-er
Early works (Vo and Zhang, 2015; Kiritchenko et al., 2014; Schouten and Frasincar, 2016) to deal with ABSA mainly relied manually designing syntactic features, which is cumbersome and in- effective as well. Subsequently, various neural network-based models (Kiritchenko et al., 2014; Vo and Zhang, 2015; Chen et al., 2017: 7; Zhang et al., 2019b; Wang et al., 2020; Trusca et al., 2020) have been proposed to deal with ABSA tasks, to get rid of hand-crafted feature design. In these stud- ies, syntactic structures proved effective, helping to connect aspects the corresponding opinion words, thereby enhancing the effectiveness of the ABSA task (Zhang et al., 2019b; Tian et al., 2021; Veysch et al., 2020; Huang and Carley, 2019; Sun et al., 2019; Wang et al., 2020). Additionally, some research (Chen et al., 2020a; Dai et al., 2021; Zhou et al., 2021; Chen et al., 2022; Brauwers and Frasin- car, 2023) suggests there should exist task-specific induced latent structures because dependency syn- tactic structures (following that, we refer to them as external structures for convenience) generated by off- shell dependency parsers are static and sub-optimal in ABSA. The syntactic structure is not specially designed to capture the interactions between aspects and opinion words.
or near SOTA perfor- mance. Additionally, our exploration also has the potential to be generalized to other tasks or bring inspiration to other similar domains.
1 Introduction
Aspect-based sentiment analysis (ABSA) is de- signed do fine-grained sentiment analysis for different aspects of a given sentence (Vo and Zhang, 2015; Dong ct al., 2014). Specifically, one or more aspects are present in a sentence, and aspects may express difffrent sentiment polarities. The purpose
Consequently, we classify these structure-baa based ABSA models into three categories by summariz- ing prior research: (1.) external structure, (2.) semi-induced structure, and (3.) full-induced struc- ture. Works based external structures use depen- dency syntactic structures generated by dependency parsers or modified dependency syntactic structures to provide structural support for ABSA (Zhang
*Conresondingg author
'Our code is at http:://gthhh..com/annnui//////

---

al., 2019b; Sun et al., 2019; Wang et al., 2020). Studies based on semi-induced structures leverage both external and induced structures, merging them to offer structural support for ABSA (Chen et al., 2020a). The first two categories require the in- troduction of external structures, which increases the complexity of preprocessing, while the third category directly eliminates this burdensomeness,
lations scale information for structure induction in contextual representations. We employ three commonly PLMs: BERT base= RoBERTapar RoBERTaaarge Our research is based on extensive experiments and yields some in- triguing findings, which we summarize follows:
Structure Induction. By comparing three GSL methods (Attention-basee (Attn.,, Kernel-based (Knl.), and Cosine-based (Cosine)), we find that the Attention-based method is the best for structure induction ABSA.
Our research is based on full-induced structures, Works in this field intend to totally eliminate the reliance external structures aid ABSA by em- ploying pre-trained language models (PLMs) to induce task-specific latent structures (Dai et al., 2021; Zhou et al., 2021; Chen et al., 2022). These efforts, however, aim to create a tree-based struc- ture, then convert it into a graph structure and feed it to Graph Neural Networks (GNNs) to capture structural information. Our research follows this line of thought, but directly from the perspective of the graph, utilizing PLMs to induce graph structure for GNNs. In addition, studies (Tamkin et al., 2020) have shown that contextual represen- tation contains information about context tokens as well as a wide range of linguistic phenomena, including constituent labels, relationships between entities, dependencies, coreference, etc. That is, there are various scales of information (spanning from the (sub)word itself to its containing phrase, clause, sentence, paragraph, etc.) in the contextual representation. This contextual representational characteristic has rarely been explored in previous studies. Therefore, our research investigates the influence of manipulations at informational scales of contextual representation on structure induction with spectral perspective.
Frequency Filter (FLT). Heuristic operations of information scales in the contextual representation by Frequency Filters are able to influence struc- ture induction. Based on Attention-based GSL, the structure induction FLT can obtain lower Aspects-sentiment Distance (AsD) and better per- formance.
Automatic Frequency Selector (AFS), Get rid of the tediousness of the heuristic method, AFS can consistently achieve better results than the Attention- GSL method. This further demon- strates the effectiveness of manipulating scale in- formation.
2
Related Work
2.1 Tree Induction for ABSA
In ABSA, there are lot of works that aim inte- grate dependency syntactic information into neural networks (Zhang et al., 2019b; Sun et al., 2019; Wang et al., 2020) to enhance the performance of ABSA. Despite the improvement of dependency tree integration, this is still not ideal since off-the- shelf dependency parsers are static, have parsing errors, and are suboptimal for a particular task. Hence, some effort is being directed toward dy- namically learning task-specific tree structures for ABSA. For example, (Chen et al., 2020a) combines syntactic dependency trees and automatically in- duced latent graph structure by gate mechanism. (Chen et al., 2022) propose to induce an aspect- specific latent tree structure by utilizing policy- based reinforcement learning. (Zhou et al., 2021) learn aspect-specific tree structure from the per- spective of closing the distance between aspect and opinion. (Dai et al., 2021) propose to induce tree structure from fine-tuned PLMs for ABSA. How- ever, most of them fall to take the context represen- tational characteristic into account.
Specifically, we employ graph structure learn- ing (GSL) based on metric learning (Zhu et al., 2021) to induce latent structures from PLMs. We investigate three commonly used metric functions (Attention-basad) (Attn.), Kernel-based (Knl.), and based (Cosine)) and contrast their effects on the structure of induced graphs. Furthermore, we heuristically explore four types of Frequency Filters with corresponding band allocations (HIGH, MID-HIGH, MID-LOW, LOW) acting contex- tual representations, and in this way, can seg- regate the representations of different scales the level of individual neurons. Additionally, we in- troduce an automatic frequency selector (AFS) to circumvent the cumbersome hcuristic approaches. This allows us to investigate the impact of manipu-

---

2.2 Spectral Approach in NLP
Prediction Head
Concallrieee
In NLP, one line of spectral methods is used in the study of improving efficiency (Han et al., 2022; Zhang et al., 2018). For example, (Han et al., 2022) propose
Negative,
Aspoct
f1u)
Graph Learner
type of recurrent neural net- work with the help of the discrete Fourier trans- former and gain faster training. In addition, a few works investigate contextual representation learn- ing from the standpoint of spectral methods. (Kayal and sattaronis,, 2019) propose method to con- struct sentence embeddings by exploiting a spectral decomposition method rooted in fluid dynamics. et al., 2022; Tamkin et al., 2020) propose using Frequency Filters to constrain differ- ent neurons model structures at different scales. These bring new inspiration the research of lan- guage representation.
GNNS
Adjacency Matrix (A)
FLT(or AFS)
Contestuul Reprosentations (H)
Output thea)
PLM
Sentence:
Great
food
but
the
service
dreadful
Figure 1: The overall architecture of our method.
2.3 Metric Learning based GSL
cumstances. For a given input sentence S [wi, w2, w. we employ a type of PLMs to serve as the contextual encoder to obtain the hid- den contextual representation H Raxd of the imput sentence S, where d is the dimension of word representations, and n. is the length of the given sen- tence. The contextual representation H is waited for inputting into GNNs architecture as node repre- sentations. Simultaneously,
The metric learning approach is one of represen- tative graph structure learning (GSL), where edge weights are derived from learning metric func- tion between pairwise representations (Zhu al., 2021). According to metric functions, the metric learning approach can be categorized into two sub- groups: Kernel-based and Attention-based, Kernel- based approaches utilize traditional kernel func- tions the metric function to model edge weights (Li et al., 2018; Yu al., 2020; Zhao et al., 20. Attention-based approaches usually utilize atten- tion networks or more complicated neural networks to capture the interaction between pairwise rep- resentations (Velickovic et al., 2018; Jiang et al., 2019; Chen et al., 2020b; Zhao et al., 2021a). The Cosine-based method (Chen et al., 2020b) is gener- ally kind of Attention-banl method. In our exper- iments, take out representative method.
going to feed into Graph Learer to induce latent graph structures, which serve as adjacency matrices A for GNNs ar- chitecture. Then the GNNs architecture can extract aspect-specific features h, utilizing both structural information from A and pre-trained knowledge in- formation from H. Finally, resentation of [CLS] token h from PLMs as well as h, and send them into Multi-layer Perception (MLP) (served as the Prediction Head) to detect the sentiment polarities (i.e., POSITIVE, NEGATIVE, NEUTRAL) for the given aspects.
concatenate the rep-
3 Method
Here, we investigate the effectiveness of three common graph structure Icarning (GSL) methods based metric learning: intentionbbssed (Attn.), Kernel-based (Knl.), and Cosine-based (Cosine) (refer to (Zhu et al., 2021) for specific descriptions of Kernel-based and Cosine-based methods). We introduce the Attention-based GSL method to adap- tively induce graph structures. Firstly, we calculate the unnormalized pair-wise edge score Eij for the i-th and j-th words utilizing the given represen- tations h, E R and h, Rd. Specifically, the pair-wise edge score Cit is calculated as follows:
To obtain induced graph structure, propose spectral filter (FLT) approach select scale infor- mation when adaptively learning graph structure. this section, we introduce a simple but effective approach (FLT) to induce graph structures from PLMs to enhance the performance of ABSA. The overall architecture is displayed in Figure 1.
3.1 Overview
As shown in Figure the overall architecture is composed of PLMs, Graph Learner, GNNs archi- tecture, and Prediction Head under normal cir-
Cij = (W,
(1)

---

where W., W Rdxda are learnable weights for i-th and j-th word representations, where d,, the hidden dimension.
Table 1: Statistics datasets.
Pusitive
Neutral
Negative
Dataset
Truin
Test
Truin
Test
Train
Test
Then, relying on these pair-wise scores eij for all word pairs, we construct the adjacency matrices A for induced graph structures. Concretely,
Rest14
2164
728
807
196
637
196
Laptop14
994
341
870
128
464
169
Twitter
1561
173
3127
346
1560
173
if i=j otherwise
Aij
expleey) Eke exples)
(2)
as semantic roles, coreference, and constituent la- bels, (Tenney et al., 2019). Prism (Tamkin et al., 2020) demonstrates these word representa- tions contain multi-scale information ranging from (sub)word to phrase, clause, sentence, and so forth. Hence in this work, we explore the impact of struc- ture induction ability by operating on scale-specific information of contextual representations.
where the adaptive adjacency matrix is A E R and Aij is the weight score of the edge between the i-th and j-th words.
For simplicity, we employ Vallina Graph Neural Networks (GCNs) (Kipf and Welling, 2017) served GNNs architecture (other variants of graph neu- ral networks can also be employed here). Given the word representations and the adaptive adjacency matrix A, we can construct an induced graph struc- ture consisting of words (each word acts as node in the graph) and feed it into GCNs. Specifically,
To achieve this goal, we introduce Frequency Filter (FLTT based on Discrete Fourier Transform (DFT) to conduct disentangling operations the frequency domain. To be specific, given word rep- resentations H E Raxd we feed them into the FLT before the Graph Learner. For the specific i-th and j-th word representations h, Rd and hy E Rd, the pair-wise edge score eij is calculated as fol- lows:
h
Aij Wh + bb),,
(3)
gf (x) =F (V((F())
where a is activation function (e.g. ReLU), w and b' are the learnable weight and bias term of the 1-th GCN layer. By stacking several layers of Graph Learner and GNNs architectures, we can obtain structure information enhanced word repre- sentations Hg for the downstream task. should be noted that the induced graph structure is dynam- ically updated while training.
(4)
Cij f (W::), 1 (W,n;)
(5)
where F(.) and (-) denote the Fast Fourier Transform (FFT) and its inverse, v indicates the filtering operation, and & denotes the Frequency Filter (FLTT) We carry out filtering at the sentence level. Subsequent operations are consistent with Section 3.1. We conduct experiments and analyses on four band allocations (HIGH, MID-HIGH, MID- LOW, LOW)). The specific band allocations are displayed in Table 5, and the analysis experiments refer to Section 4.7 and 4.10.
After we get aspect representations h, from Hg. we feed them along with the pooler output hass of PLMs (the output representation of [CLS) token) into a task-specific Prediction Head to acquire re- sults for the downstream task.
3.2 Frequency Filter (FIT)
Experiment
Furthermore, inspired by (Tamkin et al., 2020), we introduce a spectral analysis approach to enhance the structure induction ability of the Graph Learner. Intuitively, we tend to import a Frequency Filter on contextual word representations to manipulate scale information, and then feed them into the Graph Learner module improve the structure in- duction capability. Contextual representations have been investigated not only convey the meaning words in context (Peters et al., 2018), but also carry a large range of linguistic information such
To prove the effectiveness of our approach, we demonstrate experimental results conducted on three datasets for ABSA and compare them with previous works. We show the details follows.
4.1 Dataset
We conduct experiments on Semlival 2014 task (Rest14 and Laptop14) (Pontiki et al., 2014) and Twitter (Dong et al., 2014) datasets, which are widely used. Each of the three datasets contains

---

Table 2: Overall performance ABSA on the three datasets, According to the categorization of structure (Dep.: external structures (dependency syntactic tree), Semi.: semi-induced structures, Full: full-induced structures, and None: no structure information used), we classify the baselines accordingly, which are in the 'Structure' column.
Rest14
Laptop14
Twitter
Embedding
Model
Structure
Accuracy
Macro-FI
Accuracy
Macro-F1
Accuracy
Macro-FI
depCCC
Dep.
75.55t
Dep.
82.30
74.02-
77.19
72.99F
KumaGCN
Semi.
81,43
73.64
76.12
72.42
72.45
70.77
Static Embedding
RGAT
Dep.
83.30
76.08
77.42
73.76
75.57
73.82
FT-ROBERTH
Full
82.31
73.53
76.33
72.76
73.84
72.66
FT POWERT
Full
82.40
73.95
76.95
73.21
73.84
71.43
T-RoBERTA(RGAT)
Full
82.76
75.25
77.43
74.21
75.43
74.04
BERT
None
85.621
78.28F
77.58F
72.38-
75.28
74.1
SAGAT
Dep.
85.08
77.94
80.37
76.94
75.40
74.17
Dep.
80.00
79.80
77,90
.0
BERT base
depGCN-BERI
Dep.
78.79
RGAT-BERT
Dep.
86.60
81.35
78.21
74.07
76.15
74.88
KumaGCN BERT
Semi.
86.43
80.30
81,98
78.81
77.89
77.03
dotGCN-BEa BERT
Full
86.16
80.49
81.03
78.10
78.11
77.00
Roberta MLP
None
87.32
81.01
82,60
79.33
77.17
76.20
(oC(e)ta-ASC(eep)
Dep.
82.82
75.12
74.12
70.52
(DPS-
Dep.
86.71
80.31
80.52
77.13
Dep(ASGCN)
Dep.
86.90
80.75
81.66
78.31
75.28
74.38
Dep(PWCN)
Dep.
87.41
81.07
84.16
81.18
76.63
75.60
RoBERTTNuE
Dep(RGiAT)
Dep.
80.61
74.42
FT-ROBERIDASSGEN
Full
86.87
80.59
83.33
76.10
75.07
FT PPWER
Full
87.35
80.85
84,01
81.08
77.02
75.52
FTRoBBERTARGAT)
Full
87.52
81.29
83.33
79.95
75.81
74.91
FLT
Full
88.57
83.27
85.42
83.01
77.02
75.83
RoBERTaaarg
FLT
Full
90.27
85.20
86.05
84.68
77.89
77.20
three sentiment label categories: POSITIVE, NEU- TRAL, and NEGATIVE. Statistics of these datasets are displayed in Table 1, where (Train/Test) denotes the number of instances on the training, and testing set for each dataset.
External Structure. This line of works utilizes dependency syntactic structure generated by exter- nal dependency parsers (c.g. Spacy and Standford CoreNLP etc.) to offer structural information supplements for ABSA. Its delegate works as fol- lows:
4.2 Experiment Settings
depGCN (Zhang et al., 2019a) combines BiL- STM to capture contextual information regarding word orders with multi-layered GCNs.
We utilize the popular Pre-trained Language Mod- els (PLMs) based on Transformer Encoder architec- ture (BERTiuse (Devlin al., 2019), RoBERTapes and RoBERTavarge (Liu al., 2019) for word rep- resentations. Moreover, the hidden dimensions of all Graph Learners are 60. The dropout rate is 0.2, the batch size is 32. The number of the epoch is 60 for RoBERTabee and RoBERTajarge and 30 for BERTbase We use Adam optimizer (Kingma and Ba, 2015) while training with the learning rate initialized by le-5. Following previous works, Accuracy and Macro-FI scores for metries. All experiments are conducted on NVIDIA Tesla P100.
CDT (Sun et al., 2019) encodes both dependency and contextual information by utilizing GCNs and BiLSTM.
RGAT (Wang et al., 2020) feeds reshaped syn- tactic dependency graph into RGAT to capture aspect-centric information.
SAGAT (Huang et al., 2020) uses graph atten- tion network and BERT to explore both syntax and semantic information for ABSA.
DGEDT (Tang et al., 2020) jointly consider BERT outputs and dependency syntactic represen- tations by utilizing GCNs.
LCFS-ASC-CDW (Phan and Ogunbona, 2020) combine dependency syntactic embeddings, part- of-speech embeddings. and contextualized embed- dings to enhance the performance of ABSA.
4.3 Baselines
We categorize the existing structure-based ASBA models into three genres: external structure, semi- induced structure, and full-induced structure. Be- low, we introduce each of them in detail.

---

Table 3: Results of ablation studies.
Rest14
Laptop14
Twitter
Embedding
Model
Structure
Accuracy
Macro-F1
Accuracy
Macro-FI
Accurucy
Macro-F1
Attn.
Fall
85.43
78.04
80.54
77.06
76.22
75.04
BERT base
FLT
Full
87.04
81.46
81.17
77.97
77.55
76.66
Attn.
Full
87.59
81.72
83.86
80.53
75.72
73,92
RoBERTappa
FLT
Full
88.57
83.27
85,42
83.01
77.02
75.83
Alln.
Full
89.46
84.12
84.80
82,19
77.02
75.75
RoBF.RTa (auuge
FLT
Full
90.27
85.20
86.05
84.68
77.89
77.20
Semi-induced Structure. Works in this line tend to exploit both dependency syntactic structure from off-the-shelf parsers and induced structure from PLMs, the representative works are follows: KumaGCN (Chen et al., 2020a) combine latent graphs induced by self-attention neural networks and dependency syntactic structure for ABSA.
Table 4: The impact of different metric functions based on RoBERTabaee
Restid
14
Twitter
Accunscy
Macro-F1
Accurscy
Mocre-F]
Accunscy
Macro-F1
Aimn
87.59
K1.72
83,86
NILS3
25.72
73.92
Cosine
87.14
79.94
83.39
79.91
74.25
72.80
4.5 Ablation Study
Full-induced Structure. Works in this line in- tend to get totally rid of external parsers and induce task-specific latent structures from PLMs for down- stream tasks. Its delegate works follows:
We conduct ablation studies to highlight the effec- tiveness of FLT, which is based on Attention-bases (Attn.) GSL module and utilizing Frequency Filter. Thus, we compare Attn. and FLT on three PLMs (BERT bases RoBERTatasss and RoBERTatarge) show the impact of introducing Frequency Filter. Results are shown in Table 3. Compared Attn., FLT has achieved significant improvements con- sistency across three datasets utilizing different PLMs. Therefore, it can be seen that the manipula- tion of scale information is beneficial for enhancing performance.
FT-RoBERTa (Dai al., 2021) induce tree structures from the fine-tuned RoBERTa (fine-tune RoBERTa on the ABSA datasets advance) by utilizing a dependency probing approach.
dotGCN (Chen et al., 2022) induce aspect- specific opinion tree structures by using Reinforce- ment learning and attention-based regularization.
4.4 Overall Performance
The overall results of competitive approaches and FLT on the three benchmarks are shown in Table
4.6 Different Metric Function
2. We categorize baselines according to the embed- ding type (static embedding (GloVe), BERT bascrr RoBERTalsse and RoBERTatarge) and the struc- ture they used (None, Dep., Semi., and Full). The parameters of PLMs are trained together with the GSL module for FLT. Compared with baselines, FLT obtains the best results except on Twitter, which obtains comparable results. We speculate that the reason is that the expression Twitter more casual, which leads to a limited improvement of the structure on Twitter, which is consistent with the result in (Dai et al., 2021). Compared with FT-RoBERTa-seriee works, the most relevant work of ours, FLT outperforms them a lot on the three datasets. And it worth noting that FT-RoBERTa- series works need fine-tuning PLMs on the ABSA datasets in advance (Dai al., 2021), 1), but FLT does not need it. Therefore, FLT is simpler and more effective than FT-RoBERTa-ssrre works.
In this section, we contrast the impact three representative metric functions: Attention- (Attn.), Kernel-based (Knl.), and Cosine- based (Co- sine) on structure induction. From the insight of graph structure learning (Chen et al., 2020b; Zhu et al., 2021), the common options for metric learn- ing include attention mechanism (Vaswani et al., 2017; Jiang et al., 2019), radial basis function ker- nel (Li et al., 2018; Yeung and Chang, 2007), and cosine similarity (Wojke and Bewley, 2018). We follow these previous works to implement the coun- terpart metric functions (Knl. and Cosine) for com- parison, the results are shown in Table 4. The performance of attention- based (Attn.) the three benchmarks gains the best results except on Twitter. But the margin between Attn. and Knl. not big (0.29% for Accuracy and 0.06% for Macro-FI) on
Twitter, thus we select the metric function Attn. for later analysis.

---

Table 5: The spectral bands we consider in this work. Since the task considered in this work is at the sentence level, we only take the scale from word to sentence into account. Here, denotes the sentence's 's length.
the detailed frequency selection and results. Our model performs better in HIGH and MID-HIGH bands on Rest14 and Laptop1 4 but performs better in LOW and MID-LOW bands on Twitter. Com- bined with Figure 2, we find that the distribution of sentence length in Twitter is very distinct from that of Rest14 and Laptop14, the sentences in Twitter are generally longer, which leads to the fact that the clause- and sentence-scale information is more beneficial the effect improvement.
Band
Scale
Period (Toks)
DFT index
HIGH
Word
+
1/2 - L
MID-HIGH
Phrase
2 2
L/6 L/2
MID-LOW
Clause
6
L/14 - L/6
LOW
Sentence
14 -
1 L/14
4.7
Different Frequency Filters
4.8 Aspects-sentiment Distance
Table 6: Band impact based RoBERTa There are statistical results for heuristic frequency selection, and the results follow the form mean( standard deviation).
To illustrate the effectiveness of induced struc- ture, following (Dai et al., 2021), we introduce the Aspects-sentiment Distance (AsD) to quantify the average distance between aspects and sentiment words in the induced structure. The AsD is calcu-
Filter
Best4
Lapisp14
Tvilter
Acchmnay
Marn-P
Accanacy
Mictn-Fi
Accamis
K7 SEBESS
41.900117
7% (N.53 14)
MID HIGH
57.5510.53)
$1.31(1 (65)
54.390.75)
$1.099.95)
75.7100.75)
74.68 72.6800722
lated as follows:
MID-LOW
67 22)0272)
8L.150.71 71)
(3.7400.52)
91.0000.55)
7k.73/0.33)
75.6411122)
LOW
N7 9100000
802250
81.400.155
(0.000115)
1603.2
14 1414 19)
=== =
(6)
42 dist(am Gg)
Hb Reatin
AsD(S)) = |A||**
(7)
Σ (S,(S))
AsD(D) D
(8)
Icl Twitter
D
where C. = (c.,.... eg) is sentiment words set (following the setting from (Dai et al., 2021)), S; denotes each sentence in dataset D, and A (a1, aap) denotes the set of aspects for each sentence. We utilize dist (nn, to calculate the relative distance between two nodes (n1 and nı2) on the graph structure, and represent the number of elements in the given set. For a detailed setting. please refer to Appendix B.
Figure 2: The distribution of sentence length on datasets (we combine training and testing sets for this statistic).
This section analyzes the impact of four differ- ent spectral bands (HIGH, MID-HIGH, MID-LOW, LOW) on structure induction. Each band reflects a diverse range of linguistic scales from word level to sentence level, the detailed setting is shown in Table 5. The different spectral bands are revealed by their period: the number of tokens it takes to complete cycle. For example, the word scale sug- gests the period of 1 2 tokens, thus the spectral band should be L/2 + if the sentence's length denotes L.
The results are displayed in Table 7, and the less magnitude indicates the shorter distance be- tween aspects and sentiment words. Compared to dependency structure (Dep.), attention-based GSL (Attn), and our method (FLT) shorten the Aspects- sentiment Distance greatly, which shows that GSL method encourages the aspects to find sentiment words. Furthermore, in comparison with Attn., FLT has a lower AsD score, which proves reason- able adjustment on the scale level can obtain better structures.
Then, we conduct analysis experiments on the three datasets to explore the impact of different spectral bands. The length I. in our experiments is 100, which fits the length distribution of all samples in these datasets. We perform multiple frequency selections in different frequency bands heuristi- cally, and the performance of our model in different frequency bands on the three datasets is summa- rized in Table 6. Please refer to Appendix A for
4.9 Structure Visualization and Case Study
Structure Visualization. As shown in Figure 3, we visualize the difference of distinct structures: (a) is from the Spacy parser, (b) is from Attn., and (c)

---

Structure
Rest14
Laptop14
Twitter
Ground Poccrtis Pocte
Dep.
8.19
8.02
8.33
Alln.
2.26
2.55
2.64
FLT
1.97
2.15
2.16
Frankey
the
(Strase (1o0) (wees (1) Leresnrry X
Table 7: The Aspects-sentiment Distance (AsD) of dif- ferent trees in all datasets. The dependeney tree struc- ture (Dep.) comes from the Spacy parser ?
la The unsr son- CS. inkued one RooERTabesa
hegative
Th Cntress
the wtweoonbmeee inbcel TeEETTI tarel 302-000-1
Figure 4: A case of Rest dataset. The colored words denote aspects. The golden label for Chinese food is NEGATIVE.
Ialatol
Table 8: The results of AFS based on RoBERTabaee
Paterel
(wutter
oooked
ths
Rest14
Laptop14
Twitter
Model
121 The amtent con-baadd SOL use foo Rol
Arcunecy
Maca-FF
Avcurecy
Macu-FF
Avcurecy
Maroo-F/
Aim.
87.59
51.72
83.86
50.53
75.72
73.92
$1.63
(aateer
coooed
the optimization of the overall model. In this way, for different datasets, the information of the corre- sponding scale (HIGH, MID-HIGH, etc.) can be adaptively selected improve the effect of struc- ture induction. Here we briefly describe the AFS, and for a detailed description, please refer to Ap- pendix C.
Figure 3: case is from the Restl4 dataset. The col- ored words aspects. The golden label for falafal is NEGATIVE, and for chicken is POSITIVE.
is the result from FLT. This case is from the Rest14 dataset. In comparison with (a), aspects more directly connected important sentiment words (e.g. cooked, dried, and fine) in (b) and (c), which is consistent with the results of AsD in Section 4.8. In this case, both (b) and (c) obtained correct judgment results, hence from the perspective of structure, they are relatively similar.
Model Description. Following the operation of FLT, for an input sentence representation И Baxd we conduct Discrete Fourier Transform (DFT) F to transform H into the frequency domain. Then, we utilize AFS aauto to adaptively select fre- quency components, where AFS gauto is realized by using Multi-layer Perceptron (MLP) architec- ture, please refer to the Appendix C for details. After AFS and inverse Discrete Fourier Transform F-1 we can obtain the sentence representation Hafs RRaxd The subsequent operations are con- sistent with the attention-based GSL.
Case Study. In Figure 4, we provide case to compare Attn. in (a) and FLT in (b). In this case, the structures induced by the two are quite differ- ent, and for the aspect (Chinese food), Attn, gives a wrong judgment. From the comparison of struc- tures, it can be found that although the aspect word Chinese in (a) pays attention to the key information I can make better at home, they may not under- stand the semantics expressed by this clause. From the perspective of structure, FLT in (b) is obviously better able understand the meaning of this clause.
Results. We utilize AFS instead of FLT to con- duct experiments on the three datasets, the results are shown in Table 8, Compared to Attn., AFS is consistently improved. This further illustrates the operation scale information is conducive to im- proving the effectiveness of GSL. on ABSA. Com- pared with the heuristic FLT method, AFS avoids the burden brought by manual frequency selection, making the method more flexible.
4.10 Automatic Frequency Selector (AFS).
Furthermore, in order to illustrate the impact of the operation of the scale information on the GSL, we introduce an Automatic Frequency Selector (AFS) to select helpful frequency components along with
Frequency Component Analysis. Furthermore, we conducted an in-depth analysis of the intermedi-

---

Table 9: Frequency Component Analysis. The spectral bands consider in this work. Since the task consid- ered in this work is at the sentence level, we only take the scale from word to sentence into account. Here, denotes the sentence's length.
can enhance the effect of GSL on ABSA, Addition- ally. our exploration is also beneficial provide inspiration for other similar domains.
Limitations
Rand
Taplop144:5)
Twiller(()
Stale
DFT
HIGH
$4.77
25.64
K7.22
Word
1/2 L
Though we verify the operation on various infor- mation scales can be beneficial to structure induc- tion on ABSA. there are still some limitations. Al- though the heuristic FLT has achieved excellent results, it requires some manual intervention. The AFS method reduces manual participation, but its effect is worse than the optimal FLT method. How- ever, still meaningful to explore the impact of scale information on the contextual representation of downstream tasks.
MID-HICH
92.61
Phose
MID-LOW 91.82
41.02
95.87
Clause
U/4 L/6
LOW 99.61
88.08
99.19
Serience
1 1/14
ate results obtained from the Automatic Frequency Selector (AFS). From Table 8, observe that incorporating AFS consistently enhances model performance without manual adjustments to Fre- quency Components. This suggests that the auto- mated Frequency Components selection process is effective. Based on AFS's Frequency Component selection outcomes, we performed statistical anal- yses across three datasets in accordance with the spectral band distribution outlined in Table 5. Ta- ble 9 illustrates the percentage of Frequency Com- ponents selected by AFS within different spectral bands, while "Overall" represents the percentage selected Frequency Components across all four bands.
Acknowledgements
This work is funded in part by the National Natural Science Foundation of China Project (No. U1936113,,,, and the Major Key Project of PCL (PCL2021A06)
References
It is evident that the results are not uniformly 100%, indicating that AFS indeed performs selec- tion Frequency Components, thereby adjusting information at various scales achieve consistent improvements. Moreover, the percentage of se- lected Frequency Components varies across differ- ent datasets, implying adaptive adjustments by AFS to cater to the diverse demands of distinct samples. Notably, the LOW band exhibits the highest per- centage of selected Frequency Components, under- scoring the significance of sentence-level informa- tion for token-level tasks (such as Structure Induc- tion for ABSA, which can be considered token- level task). This observation also aligns with the conclusion drawn in reference EMüller-Eberrstiin et al., 2022).
Gianni Brauwers and Flavius Frasincar. 2023. A sur- vey aspect-based sentiment classification. ACM Comput. Surv., 1-655377
Chenhua Chen, Zhiyang Teng, Zhongqing Wang, and Yue Zhang. 2022. Discrete opinion tree induction for aspect-based sentiment analysis. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 2051 1051-2064. Association for Computational Linguis- tics.
Chenhua Chen, Zhiyang Teng, and Yue Zhang. 2020a. Inducing spreet-specifie latent structures for aspect sentiment classification. Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5596-56666 5607. Association for Com- putational Linguistics.
5 Conclusion
Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. 2017. Recurrent attention network on memory for aspect sentiment analysis. In EMNLP, pages 452- 461. Association for Computational Linguistics.
In this work, we propose utilizing GSL to induce la- tent structures from PLMs for ABSA and introduce spectral methods (FLT and AFS) into this prob- lem. We also explore the impact of manipulation on scale information of the contextual representa- tion for structure induction. Extensive experiments and analyses have demonstrated that the operation of scale information of contextual representation
Yu Chen, Lingfci Wu, and Mohammed J. Zaki. 2020b. Iterative deep graph learning for graph neural net- works: Better and robust node embeddings. In Ad- vances in Neural Information Processing Systems 33: Annual Conference Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, viriual.

---

Jungi Dai, Hang Yan, Tianxiang Sun, Pengfei Liu, and Xipeng Qiu. 2021. Does syntax matter? A strong baseline for aspect-based sentiment analysis with roberta. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Techmologies, NAACI-III.T 2021, Online, June 6-11, 2021. pages 1816-1829. Association for Computa- tional Linguistics.
for Computational Linguisties, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Pa- pers, pages 4536-4546. Association for Computa- tional Linguistics.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster).
Thomas N. Kipf and Max Welling, 2017. Semi- supervised classification with graph convolutional networks. In 5th International Conference on Learn- ing Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Jacob Devlin, Ming-Wei Chang. Kenton Lee, and
Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In NAACL-HLT (1), pages 4171-4148 As- sociation for Computational Linguistics.
Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif M. Mohammad. 2014. Nre-canada-2014: 4: De- tecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@ COLING 2014, Dublin, Ireland, August 23-24, 2014, pages 437-442. The Association for Computer Linguistics.
Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment clas- sification. In ACL (2), pages 49-54, The Association for Computer Linguistics.
Bing Han, Cheng Wang, and Kaushik Roy. 2022. Oscil- latory fourier neural network: A compact and ef- ficient architecture for sequential processing. In Thirty-Sixth AAAI Conference on Artificial Intelli- gence, AAAT 2022, Thirty-Fourth Conference on In- novative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22- March 1, 2022, pages 6838- 6846. AAAI Press.
Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. 2018. Adaptive graph convolutional neural networks. In Proceedings of the Thirty-Secowel AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educa- tional Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 3546-3566663 0366335, AAAI Press.
Binxuan Huang and Kathleen M. Carley. 2019. Syntax- aware aspect level sentiment classification with graph attention networks. In EMNLP/UCNNPP (1), pages 5468-5476. Association for Computational Linguis- tics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1900. 11692.
Lianzhe Huang, Xin Sun, Sujian Li, Linhao Zhang, and Houfeng Wang. 2020. Syntax- aware graph at- tention network for aspect-level sentiment classifica- tion. In Proceedings of the 28th International Confer- ence on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 799-810. International Committee on Compu- tational Linguistics.
Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. 2017. The concrete distribution: A continuous re- laxation of discrete random variables. Sth Inter- national Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Con- ference Track Proceedings. OpenReview.net.
Max Müller-Ebersted Rob van der Goot, and Barbara Plank. 2022. Spectral probing. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 7730-7741. Association for Computational Linguis- tics.
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categori- cal reparameterization with gumbel-softmax. In 5th International Conference on Learning Representa- tions, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Bo Jiang, Ziyan Zhang, Doudou Lin, Jin Tang, and Bin Luo. 2019. Semi-supervised learning with graph networks. In IEFE Confer- ence Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 11313-11320. Computer Vision Founda- tion/ / IEEE.
Matthew E. Peters, Mark Neumann, Mohit Tyyer, Matt Gardner, Christopher Clark. Kenton Lee, and Luke Zettlemoyer. 2018 Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACI-IILT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237 Association for Computational Linguistics.
Subhradeep Kayal and George Tsatsaronis. 2019. Eigensent: Spectral sentence embeddings using hidher-order dynamic mode decomposition. Pro- ceedings of the 57th Conference of the Association

---

Minh-Hier Phan and Philip O. Ogumbona. 2020. Mod- elling context and syntactical features for aspect- based sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 321 1-3220. Association for Computa- tional Linguisties.
Conference, ICWE 2020, Helsinki, Finland, June 9- 12. 2020. Proceedings, volume 12128 of Lecture Notes in Computer Science, pages 365-380. Springer.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS, pages
Maria Pontiki, Dimitris Galanis, John Paviopoulos, Har- ris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In SemEval @ COLING. pages 27-35. The Association for Computer Linguis- tics.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference Learning Representations. ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net.
Kim Schouten and Flavius Frasincar. 2016. Survey on aspect-level sentiment analysis. IEEE Trans. Knowl. Data Eng., 28(3):813-38)
Amir Pouran Ben Veyseh, Nasim Nouri, Franck Dernon- court, Quan Hung Tran, Dejing Dou, and Thien Huu Nguyen. 2020. Improving aspect-based sentiment analysis with gated graph convolutional networks and syntax-based regulation. In Findings of the As- sociation for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL, pages 4543-4548. Association for Computational Linguisties.
Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, and Xudong Liu. 2019. Aspect-level sentiment analysis via convolution over dependency tree. In EMNLP/JCCNPP (1), pages 5678-5687. Association for Computational Linguistics.
Alex Tamkin, Dan Jurafsky, and Noah Goodman. 2020. Language through prism: A spectral ap- proach for multiscale language representations. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Pro- cessing Systems 2020, NewrlPS 2020, December 6- 12. 2020, virtual.
Duy-Tin Vo and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. Proceedings of the Twenty-Fourd Inter- national Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages
AAAI Press.
Hao Tang, Donghong Ji, Chenliang Li, and Qiji Zhou.
Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020. Relational graph attention net- work for aspect-based sentiment analysis. In ACL, pages 3229-3238. Association for Computational Linguistics.
2020. Dependency graph enhanced dual transformer structure for aspect-based sentiment classification. In ACI, pages 6578-6588. Association for Computa- tional Linguisties.
lan Tenney, Patrick Xia, Berlin Chen, Alex Wang. Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das. and Ellie Pavlick. 2019. What do you learn from context? probing for sentence structure in contextu- alized word representations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenRe- view.net.
Nicolai Wojke and Alex Bewley. 2018 Deep cosine metric learning for person re-identification. In 2018 IEEE Winter Conference on Applications of Com- puter Vision, WACV 2018, Lake Tahoe, NV, USA, March 12-15, 2018, pages 748-756. IEEE Computer Society.
Dit- Rit-Yan Yeung and Hong Chang. 2007. A kernel ap- proach for semisupervised metric learning. IEEE Trans. Neural Nenworks, 18(1):141-149
Yuanhe Tian, Guimin Chen, and Yan Song. 2021. Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, NAACL-HLI 2021, Online, June 6- 11, 2021, pages 2910-2922, Association for Computational Linguis- tics.
Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, and Yiming Yang. 2020. Graph-revised convo- lutional network. In Machine Learning and Knowl- edge Discovery in Databases- European Conference, ECML PKDD 2020, Ghent, Belgium, September 14- 18, 2020, Proceedings, Part III, volume 12459 of Lecture Notes in Computer Science, pages 378-393. Springer.
Maria Mihacla Trusca, Daan Wassenberg, Flavius Fras- incar, and Rommert Dekker. 2020. A hybrid ap- proach for aspect-based sentiment analysis using deep contextual word embeddings and hierarchical attention. In Web Engineering- 20th International
Chen Zhang, Qiuchi Li, and Dawei Song. 2019a. Aspect-based sentiment classification with aspect- specific graph convolutional networks. In EMNLP/JCNLPP (1), pages 4567-4577. Association for Computational Linguistics.

---

Chen Zhang, Qiuchi Li, and Dawei Song. 2019b. Syntax-awe aware aspect-level sentiment classification convolution network. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, pages 144--148. ACM.
100 for each dataset and place sentences of sim- ilar length in one batch, with a batch size of 32. Each batch is batched according to the maximum sentence length in that batch. For simplicity, we did not design specific spectral bands for differ- ent sentence lengths. Instead, we set the spectral bands based on the maximum sentence length (L) in each dataset. We only change the hyperparame- ter 'Bands' settings, while all other settings remain the same. For specific experimental settings, refer to Section 4.2. It can be observed that different spectral band selections indeed lead to different results, and an appropriate heuristic spectral band selection can significantly improve the results.
with proximity-
Jiong Zhang, Yibo Lin, Zhao Song, and Inderjit S. Dhillon. 2018. Learning long term dependencies via fourier recurrent units. Proceedings of the 35th In- ternational Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 58 10-5818. PMLR.
Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guo- jie Song, and Yanfang Ye. 2021a. Heterogencous graph structure learning for graph neural networks. In l'hirty-Fftth AAAI Conference on Artificial Intel- ligence, AAAI 2021, Thirty-Third Conference on In- novative Applications of Artificial Intelligence, IAAl 2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAT 2021, Virtual Event, February 2-9, 2021, pages 4697-4705, AAAI Press.
B The Settings of AsD analysis
Here, we provide a detailed introduction to the relative distance calculation (ist n2) for AsD. For given sentence S,, with its aspect words A = (a.,... , ap)) sentiment word set C = (cr, Cg), and the adjacency matrix Ag of the induced graph structure, we calculate the shortest hops from Op to Cg. If the value of the corresponding position of ap and Gy on the adjacency matrix Ag is greater than the threshold y, then we call the distance between Op and Cy to be 1. Otherwise, finding the shortest hops between dp and Cq on the Ag as its shortest path. We also use y to judge whether there is edge between two nodes. Here, 7 is set to the average value of all values of Ag. If ag and cy are not directly connected, we set the distance between ap and c, to the maximum number of hops, where the maximum number of hops is set to 10.
Tong Zhao, Yozen Liu, Leonardo Neves, Oliver J. Wood- ford, Meng Jiang, and Neil Shah. 202 Data aug- mentation for graph neural networks. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thine Third Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Ar- tificial Intelligence, EAAI 2021, Virtual Event, Febru- ary 2-9, 2021, pages 1015-1 AAAI Press.
Yuxiang Zhou, Lejian Liao, Yang Gao, Zhanming Jie, and Wei Lu. 2021. To be closer: Learning to link up aspects with opinions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November. 2021, pages 3899 1899-3909. Association for Computa- tional Linguistics.
C Automatic Frequency Selector (AFS)
Furthermore, it is not affirmed that information in just one band (e.g. HIGH, MID-HIGH, etc.) is helpful, and information in other bands may also provide a gaining effect. Therefore with this in mind, we introduce an Automatic Frequency Selec- tor (AFS) to select helpful frequency components along with the optimization of the overall model.
Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2021. Deep graph struc- ture learning for robust representations: A survey. CoRR, Abs/2103.030336
A Different Frequency Selection
We heuristically select spectral bands (HIGH, MID- HIGH, MID-LOW, LOW) to observe the impact of different spectral bands on structure induction for ABSA. The specific selection of spectral bands at different frequencies and their results are shown in Table 10. The range of spectral bands corresponds to the description in Table 5. Here, based on the distribution of sentence lengths in the dataset (refer to Figure/ 2), we set the maximum length (L. to
To achieve this goal, we design the Frequency Se- lection operation under a probabilistic scenario T. To be specific, we map each frequency component f into Bernoulli parameter space by employing Multi-layer Perceptron (MLP) architecture to pa- rameterize this mapping process. Firstly, we bring in a set of learnable parameters & C &xxx to pa- rameterize frequency components, where denotes the number of frequency components, and dy de-

---

notes the dimension of component representations. Then, utilize the MLP architecture (composed of two linear projection layers Proji and Proj2. and an activation function a (i.e. ReLU)) to map frequency components Ç into the Bernoulli param- eter space.
2B MLP(E) = Projz (roiz(((Prron (Proji(())) (9)
En <
loo( 22 (oo(-)og()))
(10)
where EB denotes the success probabilities of Bernoulli distributions, and yp denotes the Softmax function. We utilize the Gumbel reparameteriza- tion proposed by (Jang et al., 2017; Maddison et al., 2017) to address the differentiable difficulty during training, where ~ (4, 1) is random variables of a uniform distribution on the interval (0, 1). The hy- perparameter - is the annealing temperature, which is adjusted to zero progressively in practice. Next, we can obtain the values of Bernoulli random variables mB ~ Bern(EB), where mB C {0, 1)%, and Bern denotes Bernoulli distributions. During the non-training phase, we set hyperparameter threshold y to control the sparsity of my. (For the Rest14 dataset, we set the threshold y to 0.65. For the other two datasets, the threshold is set at 0.75.) Subsequently, for the i-th and j-th word repre- sentations h; Rd and hy E Rd,
calculate
the pair-wise edge score Eij as follows:
Tal* = =
(11)
Gij =
(W,h,)" (12)
where indicates the Frequency Selection opera- tion, and Φafa denotes the Automatic Frequency Selector (AFS). Subsequent operations are consis- tent with Section 3.1.

---

Table 10: Detailed results of the band impact based on RoBERTa for heuristic frequency selection. For real sequence, the spectrum obtained by the Discrete Fourier Transform is symmetrical, so we only take half of the spectral bands for analysis. Negative values indicate that the frequency selected from the high-frequency band, and positive values mean that the frequency is selected from the low-frequency band. Additionally, x means that the frequency selection is between two values (r and y). The values bold indicate superior performance compared to the Attn, method.
Rest14
Laptop14
Twitter
Filter
Bands
Accuracy
Macro-F1
Accuracy
Macro-F1
Accuracy
Macro-Fl
-1
87.32
80.76
84.48
81.54
75,43
74.88
87.32
80.79
84.17
81.13
75.72
74.45
87.23
81.56
83.86
81.20
74.34
86.88
80.44
84.01
81.34
75.43
74.78
-
87.77
81.62
83.54
80.53
76.30
75.00
87.77
81.71
82.76
79.93
75.58
74.34
87.77
80.74
84.80
82.27
76.16
75.52
-10
87.05
80.79
83.86
81.37
75.58
74.41
-12
87.77
80.74
84.48
81.38
75.87
74.45
HIGH
87.75
81.86
84.80
82.21
75.43
74.69
-16
88.57
82.95
84.32
81.87
76.45
75.46
-18
86.43
79.26
83.54
80.54
75.43
74.08
88.13
82.33
84.01
81.06
76.01
75.23
-22
88.57
83.27
84.48
81.82
75.58
74.91
-24
87.14
80.63
84.17
81.65
76.30
75.18
-26
87.50
80.85
84.64
82.04
76.01
74.46
+ 10
88.21
82.41
84.48
81.90
74.57
74.19
8 11
87.86
81.69
85.42
83.01
75.29
74.59
8 + 12
87.50
80.66
83.39
80.49
75.29
74.68
+ 13
87.23
80.13
83.86
81.06
76.88
75.70
14
86.88
80.75
84.48
81.70
75.72
74.90
MID-HIGH
8 16
87.95
81.69
83.70
80.92
77.02
75.84
8 - 18
87.50
82.16
85.27
82.67
75.72
74.48
88.48
83.32
83.70
80.81
75.14
73.63
8 22
87.05
79.81
83.54
80.50
76.45
75.16
8 + 24
86.88
80.53
84.33
81.65
75.00
73.62
+
86.96
80,50
84.01
81.14
76.45
75.50
87.14
80.40
83.70
81.05
76.59
75.61
MID-LOW
87.14
81.71
84.33
82.10
77.02
75.64
+
87.68
81.99
82.92
79.72
76.87
75.82
87.41
81.27
83,39
80.44
76.16
75.03
87.86
81.06
83.39
80.55
76.15
75.16
LOW
87.23
80.51
83.70
80.80
76.45
74.90
86.96
80.14
84.01
81.64
75.87
74.65
Atin.
87.59
81.72
83.86
80.53
75.72
73.92


Reviews:
[{"review_id": "srtgSrbryg", "reviewer": "Reviewer_n3xo", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## 1: Hyperparameter in FLT\n\nFor different manually selected frequency components in FLT, please refer to Table 8 in the appendix. For your convenience, we have also presented the results from Table 8 below, with bold values indicating the frequency selections that demonstrate improvements of FLT relative to Attn.\n\n  (Detailed results of the band impact based on RoBERTa(base) for heuristic frequency selection. For real\nsequence, the spectrum obtained by the Discrete Fourier Transform is symmetrical, so we only take half of the spectral\nbands for analysis. Negative values indicate that the frequency is selected from the high-frequency band, and\npositive values mean that the frequency is selected from the low-frequency band. Additionally, x → y means that\nthe frequency selection is between the two values (x and y))\n\n| Filter | Bands | Rest14 Accuracy | Rest14 Macro-F1 | Laptop14 Accuracy | Laptop14 Macro-F1 | Twitter Accuracy | Twitter Macro-F1 |\n|--------|-------|-----------------|-----------------|-------------------|-------------------|------------------|------------------|\n| HIGH   | -1    | 87.32           | 80.76           | **84.48**             | **81.54**             | 75.43            | **74.88**            |\n|        | -2    | 87.32           | 80.79           | **84.17**             | **81.13**             | **75.72**            | **74.45**            |\n|        | -3    | 87.23           | 81.56           | **83.86**             | **81.20**             | **76.01**            | **74.34**            |\n|        | -4    | 86.88           | 80.44           | **84.01**             | **81.34**             | 75.43            | **74.78**            |\n|        | -5    | **87.77**           | 81.62           | 83.54             | 80.53             | **76.30**            | **75.00**            |\n|        | -6    | **87.77**           | 81.71           | 82.76             | 79.93             | 75.58            | **74.34**            |\n|        | -8    | **87.77**           | 80.74           | **84.80**             | **82.27**             | **76.16**            | **75.52**            |\n|        | -10   | 87.05           | 80.79           | **83.86**             | **81.37**             | 75.58            | **74.41**            |\n|        | -12   | **87.77**           | 80.74           | **84.48**             | **81.38**             | **75.87**            | **74.45**            |\n|        | -14   | **87.75**           | **81.86**           | **84.80**             | **82.21**             | 75.43            | **74.69**            |\n|        | -16   | **88.57**       | **82.95**           | 84.32             | **81.87**             | **76.45**            | **75.46**            |\n|        | -18   | 86.43           | 79.26           | 83.54             | **80.54**             | 75.43            | **74.08**            |\n|        | -20   | **88.13**           | **82.33**           | **84.01**             | **81.06**             | **76.01**            | **75.23**            |\n|        | -22   | **88.57**       | **83.27**       | **84.48**             | **81.82**             | 75.58            | **74.91**            |\n|        | -24   | 87.14           | 80.63           | **84.17**             | **81.65**             | **76.30**            | **75.18**            |\n|        | -26   | 87.50           | 80.85           | **84.64**             | **82.04**             | **76.01**            | **74.46**            |\n|--------|---------|-----------------|-----------------|-------------------|-------------------|------------------|------------------|\n| MID-HIGH | 8 \\to 10 | **88.21**    | **82.41**           | **84.48**             | **81.90**             | 74.57            | **74.19**            |\n|          | 8 \\to 11 | **87.86**         | 81.69           | **85.42**         | **83.01**         | 75.29            | **74.59**            |\n|          | 8 \\to 12 | 87.50         | 80.66           | 83.39             | 80.49             | 75.29            | **74.68**            |\n|          | 8 \\to 13 | 87.23         | 80.13           | **83.86**             | **81.06**             | **76.88**           | **75.70**            |\n|          | 8 \\to 14 | 86.88         | 80.75           | **84.48**             | **81.70**             | **75.72**            | **74.90**            |\n|          | 8 \\to 16 | **87.95**         | 81.69           | 83.70             | 80.92             | **77.02**        | **75.84**        |\n|          | 8 \\to 18 | 87.50         | **82.16**           | **85.27**             | **82.67**             | **75.72**            | **74.48**            |\n|          | 8 \\to 20 | **88.48**         | **83.32**           | 83.70             | 80.81             | 75.14            | 73.63            |\n|          | 8 \\to 22 | 87.05         | 79.81           | 83.54             | 80.50             | **76.45**            | **75.16**            |\n|          | 8 \\to 24 | 86.88         | 80.53           | **84.33**             | **81.65**             | 75.00            | 73.62            |\n|--------|---------|-----------------|-----------------|-------------------|-------------------|------------------|------------------|\n| HIGH   | 4 \\to 5 | 86.96           | 80.50           | **84.01**             | **81.14**             | **76.45**            | **75.50**            |\n|        | 4 \\to 6 | 87.14           | 80.40           | 83.70             | **81.05**             | **76.59**            | **75.61**            |\n|        | 4 \\to 7 | 87.14           | 81.71           | **84.33**             | **82.10**             | **77.02**        | **75.64**            |\n|        | 4 \\to 8 | **87.68**           | **81.99**           | 82.92             | 79.72             | **76.87**            | **75.82**            |\n|  LOW   | 1       | 87.41           | 81.27           | 83.39             | 80.44             | **76.16**            | **75.03**            |\n|        | 2       | **87.86**           | 81.06           | 83.39             | **80.55**             | **76.15**            | **75.16**            |\n|        | 3       | 87.23           | 80.51           | 83.70             | **80.80**             | **76.45**            | **74.90**            |\n|        | 4       | 86.96           | 80.14           | **84.01**             | **81.64**             | **75.87**            | **74.65**            |\n| Attn.  | -       | 87.59           | 81.72           | 83.86             | 80.53             | 75.72            | 73.92            |\n\n\n## 2: Benchmark Dataset and Results\n\nBased on prior related research and our experimental findings, ensuring a consistent random seed guarantees reproducibility, making each run of the model yield identical results. Furthermore, the datasets we employed are well-established and commonly used in previous related works. Comparing the results against all baselines was conducted under uniform conditions, employing the same datasets and experimental settings. Consequently, our experimental outcomes are both reliable and stable.\n\n## 3: References\n\nBased on your suggestion, we will include all the references you pointed out in the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed approach achieved good performance on English benchmark datasets.\n2. The paper is easy to follow.\n", "reasons_to_reject": "1. Many techniques (e.g., attentions, graph neural networks, discrete Fourier transform) used in the paper are not new.\n2. It is not clear how the hyper-parameters are tuned. It seems the results reported in Table 2 and 3 comes from a single run (the authors do not report the average and standard deviation of the results). Given the test sets of the benchmark datasets are small, it is possible that the model overfits the test set.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes to improve aspect-based sentiment analysis (ABSA) through the induction of structure information, where graph structure learning (GSL) module that consists of a graph learner and Graph Neural Networks (GNNs) is proposed. The paper incorporated Frequency Filters to produce filtered language representations and feed these into the GSL module to generate latent structures. Experiments on three ABSA benchmark datasets demonstrated that this approach reduced the aspect-sentiment distance (AsD), leading to good performance. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "1. Binxuan Huang and Kathleen M Carley. SyntaxAware Aspect Level Sentiment Classification with Graph Attention Networks. EMNLP-2019\n2. Amir Pouran Ben Veyseh, Nasim Nouri, Franck Dernoncourt, Quan Hung Tran, Dejing Dou, Thien Huu Nguyen. Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation. EMNLP-2020: findings\n3. Yuanhe Tian, Guimin Chen, Yan Song. Aspect-based Sentiment Analysis with Type-aware Graph Convolutional Networks and Layer Ensemble. NAACL-2021 "}}, {"review_id": "iZw8US4Vit", "reviewer": "Reviewer_ubU3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, thank you so much for your detailed comments and affirmation of our approach.\n\n## 1: Model Description\n\nWe will carefully polish the camera-ready version and describe the input and output of the model in detail in the camera-ready version. This task (ABSA) is a sentiment classification task for aspect words in a sentence. Since the paper describes the focus on structure induction and Frequency Filter (FLT), there are fewer descriptions of the input and output of the model and the definition of the problem. We will The description of this part will be added in the camera-ready version. \n\n## 2: Citation Format\n\nWe will modify the citation format issue in the camera-ready version, and if we check for the same similar issues, we will also modify them.\n\n## 3: Figure 1 in our manuscript\n\nThe input of the model is only the binary group (sentence, aspect word), and the emotional polarity label (Positive, Negative, and Neutral) of the aspect word is used as the classification label. The output of the model prediction head is the logits for classification. The cross-entropy loss will be calculated with the aspect word sentiment polarity label so that the whole model can perform gradient backpropagation according to the loss. Regarding the problem of Figure 1, we will improve it in the camera-ready version to make the diagram more clearly represent data transformation and input and output.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed model demonstrates strong theoretical foundations.\n\nThe experimental results provide evidence of the model's effectiveness.", "reasons_to_reject": "1. The writing requires polishing. It's unclear whats the input and output of the proposed model. From section 4, it seems the model's output is the sentiment label, but the model description in Section 3 doesn't mention anything about classification.  \n\n2. The citation format is incorrect. For example, in line 285, it should be written as \"inspired by Tamkin et al., (2020),\" instead of \"inspired by (Tamkin et al., 2020).\"\n\n3. Figure 1 is not clear as it lacks input/output arrows to illustrate how the data is transformed. The role and purpose of the prediction head in this architecture are not well-defined, making it difficult to understand its function within the model. More detailed explanations and visual annotations would improve the clarity and comprehension of the figure.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper introduces an ABSA parsing model that generates a latent structure. Specifically, the model employs a pre-trained language model (RoBERTa) to generate language representations. These representations are then filtered using frequency filters. Next, a GSL module is utilized to induce latent structure from the filtered representations. The paper thoroughly evaluates the proposed model through extensive experiments on three public benchmarks for ABSA.\n\n\n\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0MK3acYS3l", "reviewer": "Reviewer_sCU9", "summary": "", "questions": "1. What is the relation between frequency and context? How is this related to DFT?\n2. Did you try to combine multiple graphs based on the previous similarity functions and then using a fusion gate?\n3. The comparison between FLT and AFS is not clear, as in the paper FLT is claimed to be the best while in conclusions AFS is claimed to be the best w.r.t. to the accuracy. What is correct?\n\nThank you for the answers to my questions. I am satisfied with the answers.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, thank you so much for your detailed comments and affirmation of our approach.\n\n## 1: Relation between Frequency, DFT and Context\n\n### Related Works and Motivation\n\nDrawing from previous research ([1],[2]), it is well-established that contextual representations with varying scales of information in pre-trained models exert distinct influences on downstream tasks. For instance, reference [1] demonstrates the advantage of utilizing low-frequency signals for topic classification (document-level), while medium-frequency signals prove more beneficial for tasks such as dialog acts classification (utterance-level). Hence, it can be inferred that operations on different scale information within context representations may impact task performance downstream.\n\nIn this study, we concentrate on conducting Structure Induction from contextual representations of pre-trained models. We apply the induced structure to tasks like Aspect-based Sentiment Analysis (ABSA) that require structural information. Our focus is on exploring whether operations concerning scale information are conducive to enhancing the efficacy of induced structures, thus yielding benefits for downstream tasks. To achieve this objective, segregating and extracting information of varying frequencies becomes imperative. This naturally leads us to consider Discrete Fourier Transform (DFT) as a spectral analysis tool, aligning with the methodology employed in prior research endeavors.\n\n[1] Alex Tamkin, Dan Jurafsky, and Noah D. Goodman. 2020. Language through a prism: A spectral approach for multiscale language representations. NeurIPS 2020, December 6-12, 2020, virtual.\n\n[2] Max Müller-Eberstein, Rob van der Goot, and Barbara Plank. 2022. Spectral probing. EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 7730–7741. Association for Computational Linguistics\n\n### Further Experiment\n\nFurthermore, we conducted an in-depth analysis of the intermediate results obtained from the Automatic Frequency Selector (AFS). From Table 7 in the manuscript, we observe that incorporating AFS consistently enhances model performance without manual adjustments to frequency components. This suggests the effectiveness of the automated Frequency Components selection process. Based on AFS's frequency component selection outcomes, we performed statistical analyses across three datasets in accordance with the spectral band distribution outlined in Table 5 of the manuscript. The table below illustrates the percentage of Frequency Components selected by AFS within different spectral bands, while \"Overall\" represents the percentage of selected Frequency Components across all four bands.\n\nIt is evident that the results are not uniformly 100%, indicating that AFS indeed performs selection on Frequency Components, thereby adjusting information at various scales to achieve the consistent improvements highlighted in Table 7 of the manuscript. Moreover, the percentage of selected Frequency Components varies across different datasets, implying adaptive adjustments by AFS to cater to the diverse demands of distinct samples. Notably, the LOW band exhibits the highest percentage of selected Frequency Components, underscoring the significance of sentence-level information for token-level tasks (such as Structure Induction for ABSA, which can be considered a token-level task). This observation aligns with the conclusion drawn in reference [2].\n\n| Band | Rest14(%) | Laptop14(%) | Twitter(%) | Scale | DFT index |\n| --- | --- | --- | --- | --- | --- |\n| HIGH | 84.77 | 25.64 | 87.22 | Word | L/2 → L |\n| MID-HIGH | 89.82 | 28.68 | 92.61 | Phrase | L/6 → L/2 |\n| MID-LOW | 91.82 | 41.02 | 96.87 | Clause | L/14 → L/6 |\n| LOW | 99.61 | 88.08 | 99.19 | Sentence | 1 → L/14 |\n| Overall | 88.88 | 35.21 | 91.41 | - | - |\n\nThe specific configuration of spectral bands in our study is detailed in Table 5 of the manuscript.\n\n## 2： Combine multiple graphs based on the previous similarity functions and then using a fusion gate\n\nIndeed, we previously delved into inducing multiple graphs simultaneously, while also avoiding the use of off-the-shelf parsers. We achieved this integration through gating mechanisms. Typically, such an approach yields improvements. However, it's crucial to ensure that each induced graph remains distinct. Moving forward, we are committed to further exploration in this realm to refine our understanding and methodologies in inducing multiple graphs effectively.\n\n## 3: The comparison between FLT and AFS\n\nAFS is a variant of FLT. They are based on the scale information operation of the context. For FLT, it is necessary to heuristically select Frequency Components. AFS avoids this burden and makes the method more flexible and convenient. Although the effect of carefully selecting Frequency Components under FLT may exceed AFS, the effect of AFS is still greater than Attn, and it is at the head of all possible Frequency Component selection results of FLT.\n\n## 4: Other Concerns\n\n- **References:** Based on your suggestion, we will include all the references you pointed out in the camera-ready version.\n- **Typos Grammar Style And Presentation Improvements:** Thank you very much for pointing out our representation problems; we will fix all of them in the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ technical paper\n+ good results\n+ novel solution", "reasons_to_reject": "- sometimes not clear explanations\n- sometimes not clear results\n- missing relevant literature", "questions_to_authors": "1. What is the relation between frequency and context? How is this related to DFT?\n2. Did you try to combine multiple graphs based on the previous similarity functions and then using a fusion gate?\n3. The comparison between FLT and AFS is not clear, as in the paper FLT is claimed to be the best while in conclusions AFS is claimed to be the best w.r.t. to the accuracy. What is correct?\n\nThank you for the answers to my questions. I am satisfied with the answers.", "paper_topic_and_main_contributions": "The paper presents a solution for ABSC based on fully-automatic Graph Structure Learning (GSL) using PLMs. The graphs are induced using three similarities: attention, kernel, and cosine. In addition a Frequency Filter (FLT) based on DFT selects frequencies from 5 bandwidths. Alternatively, an Automatic Filter Selector (AFS) helps with the automatic selection of the frequencies. Using three datasets and an ablation experiment the authors show the usefulness of the proposed approach and its components.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "None."}, "extra_text": {"Missing_References": "How does approach compare to hybris solutions as HAABSA++:\n\nMaria Mihaela Trusca, Daan Wassenberg, Flavius Frasincar, Rommert Dekker: A Hybrid Approach for Aspect-Based Sentiment Analysis Using Deep Contextual Word Embeddings and Hierarchical Attention. ICWE 2020: 365-380\n?\n\nAs a paper on ABSA and ASBC, these two surveys need to be referred to:\n\n-For ABSA:\n\nKim Schouten, Flavius Frasincar: Survey on Aspect-Level Sentiment Analysis. IEEE Trans. Knowl. Data Eng. 28(3): 813-830 (2016)\n\n-For ABSC:\n\nGianni Brauwers, Flavius Frasincar: A Survey on Aspect-Based Sentiment Classification. ACM Comput. Surv. 55(4): 65:1-65:37 (2023)", "Typos_Grammar_Style_and_Presentation_Improvements": "-throughout the paper: all bold text followed by a paragraph needs a dot at the end (e.g., “External Structure.” Instead of “External Structure”)\n-throughout the paper: “e.g.,” instead of “e.g.”\n-page 1: “etc.” instead of “etc”\n-page 3: “a simple” instead of “the simple”\n-page 4: h_{CLS instead of h_cls\n-page 5: “CoreNLP” instead of “Corenlp”\n-page 5: “uses graph” instead of “use graph”\n-page 6: “to users” instead of “of users”\n-page 7: “mean (” instead of “mean(”\n-page 7: “our method” instead of “Our method”\n-page 9: “ABSA” instead of “ABAS”"}}]

Author rebuttal:

Meta-review: The paper introduces an approach to enhance aspect-based sentiment analysis (ABSA) by incorporating graph structure learning (GSL) techniques. The proposed method utilizes a Graph Neural Network (GNN) within a GSL module to induce latent structures from filtered language representations, generated using Frequency Filters. These latent structures aim to improve the accuracy of ABSA. The paper evaluates the approach on three benchmark ABSA datasets and reports favorable results. 
Key strengths of the paper include its solid theoretical foundation and promising experimental results. However, reviewers raised concerns about the clarity of the paper's explanations (lack of clarity regarding hyper-parameter tuning), the novelty of the techniques used, and issues with presentation (need for writing revisions to improve clarity), which could be improved for the camera-ready version if accepted.
Decision: Accept-Findings