On the Strength of Sequence Labeling and Generative Models for Aspect Sentiment Triplet Extraction
Year: 2023

Abstract:
Generative models have achieved great success in aspect sentiment triplet extraction tasks. However, existing methods ignore the mutual informative clues between aspect and opinion terms and may generate false paired triplets. Furthermore, the inherent limitations of generative models, i.e., the token-by-token decoding and the simple structured prompt, prevent models from handling complex structures especially multi-word terms and multi-triplet sentences. To address these issues, we propose a sequence labeling enhanced generative model. Firstly, we encode the dependency between aspect and opinion into two bidirectional templates to avoid false paired triplets. Secondly, we introduce a marker-oriented sequence labeling module to improve generative models’ ability of tackling complex structures. Specifically, this module enables the generative model to capture the boundary information of aspect/opinion spans and provides hints to decode multiple triplets with the shared marker. Experimental results on four datasets prove that our model yields a new state-of-art performance. Our code and data are available at https://github.com/NLPWM-WHU/SLGM.

heading: 1 Introduction
text: Aspect sentiment triplet extraction (ASTE) aims at extracting all triplets in a sentence, consisting of the aspect/opinion terms and the sentiment polarity on them. Given the example “Their twist on pizza is healthy, but full of flavor.” in Fig. 1 (a), the goal is to extract two triplets (twist on pizza, healthy, positive) and (flavor, full, positive).
Conventional approaches to ASTE include pipeline (Peng et al., 2020), table filling (Chen et al., 2022), sequence tagging (Xu et al., 2020; Wu et al., 2020b), and hybrid ones (Xu et al., 2021). More recently, there is an emerging trend in adopting generative models for ASTE (Yan et al., 2021;
* Corresponding author.
𝑋𝑋𝑎𝑎
𝑋𝑋𝑜𝑜
𝑌𝑌𝑎𝑎
𝑌𝑌𝑜𝑜
Previous Methods
Our Method
ENC DEC
MOSL
𝑋𝑋
Zhang et al., 2021b,a; Lu et al., 2022) to alleviate error propagation and exploit full label semantics.
Current generative ASTE models employ a classical encoder-decoder architecture and follow a paradigm that first generates a target sequence Y and then recovers the triplets T from the sequence Y . The model needs to pre-define an output template ψ(·) to convert ASTE into text generation and then calculates the loss between the triplet and the generated sequence for model training, as shown in Fig. 1 (b). The template ψ(·) constructed by existing methods is in the form of ψa→o or ψo→a, reflecting the unidirectional dependency from aspect to opinion, or vice versa. However, the aspect and opinion terms that appear together in one sentence might hold informative clues to each other (Chen and Qian, 2020b) and there is no intrinsic order between them (Chen et al., 2021). Hence, modeling unidirectional dependency may mislead the model to generate false paired triplets like (twist on pizza, full, positive).
12038
Existing generative ASTE models also suffer from another challenging problem, i.e., lacking the ability to handle complex structures especially multi-word terms and multi-triplet sentences. On one hand, the token-by-token decoding manner makes the model focus only on the next token at each time step of decoding without grasping the whole information of the aspect/opinion term with multiple words. On the other hand, generative models often deploy the simple-structured prompt template to ensure the generation quality. When handling the sentence with multiple triplets, a generative model needs to invoke a template several times, which may lead to an information confusion for the same marker in the template.
To address the aforementioned issues, we propose a sequence labeling enhanced generative model for ASTE.
Firstly, we design two bidirectional templates with different decoding orders to simultaneously capture the mutual dependency between the aspect and opinion terms. In particular, we add two types of prompt prefix before the input sentence to indicate the decoding order, and we also present two output templates ψa→o and ψo→a, both consisting of the markers {aspect, opinion, sentiment} and the corresponding labels {a, o, s}. In this way, the decoder can generate two sentences reflecting dependency from aspect to opinion and that from opinion to aspect.
Secondly, we propose a marker-oriented sequence labeling (MOSL) module, which can enhance the generative model’s ability to handle complex structures. Specifically, the decoding is conducted after the MOSL module at the training stage. Hence the BIO tags obtained in MOSL help the generative model capture the boundary information of multi-word aspect/opinion terms in advance. Moreover, while the generative model needs to invoke the output templates several times for the multitriplet sentence, we adopt different marker vectors in MOSL for the same marker in the generative model. By doing this, we can share the markers without causing confusion. Since the markers encode information across multiple triplets in one sentence, previous markers can contribute to the decoding of subsequent triplets. The illustration of our proposed method is shown in Fig. 1 (c).
We conduct extensive experiments on four datasets with both full supervised and low-resource settings. The results demonstrate that our model
significantly outperforms the state-of-art baselines for the ASTE task.

heading: 2 Related Work
text: Aspect-based sentiment analysis traditionally involves three basic tasks, including aspect extraction (Xu et al., 2018; Dai and Song, 2019; Chen and Qian, 2020a), aspect-level sentiment classification (Zhang and Qian, 2020; Zhou et al., 2021; Li et al., 2021), and opinion extraction (Wu et al., 2020a).
To meet the practical need, some recent studies propose to extract two or more elements simultaneously, including aspect opinion pair extraction (Zhao et al., 2020; Wu et al., 2021; Gao et al., 2021), end-to-end aspect-based sentiment analysis (Hu et al., 2019; Chen and Qian, 2020b; Oh et al., 2021), and aspect sentiment triplet extraction. Among them, ATSE is regarded as a near complete task and is of the most challenge.
Earlier work in ATSE can be sorted into four streams, i.e., pipeline (Peng et al., 2020), table filling (Chen et al., 2022), sequence tagging (Xu et al., 2020; Wu et al., 2020b), and hybrid ones (Xu et al., 2021; Chen et al., 2021; Mao et al., 2021). These methods do not fully utilize the rich label semantics and some of them may encounter the error propagation problem.
Another line of research in ASTE performs this task in a generative manner (Zhang et al., 2021a,b). For example, Yan et al. (2021) model the extraction and classification tasks as the generation of pointer indexes and class indexes. Lu et al. (2022) introduce the structured extraction language and structural schema instructor to unify all information extraction tasks. While getting better performance, current generative models are prone to generate false paired triplets and are not suitable for tackling complex structures. Our generative model addresses these issues with the proposed bidirectional templates and the marker-oriented sequence labeling module.

heading: 3 Our Method
text: Given a review sentence X with L words, the goal of ASTE is to extract all triplets T = {(a, o, s)}Ni=1 in X , where N is the number of triplets, and a, o, and s denotes aspect term, opinion term, and sentiment polarity, respectively.
We first introduce the overall architecture of our proposed sequence labeling enhanced generative
Template-Guided Generation Model
Encoder
Decoder
aspect : i5 processor , opinion : speeds things , sentim ent : up positive 

M O SL
O
B I O
M O SL
I
O B I
O
model (SLGM) in Fig. 2, which has the following distinguished characteristics.
(1) To capture the mutual information between the aspect and opinion terms, we construct two bidirectional templates at both the input and output ends, shown as Xa/Xo and ψa→o/ψo→a in Fig. 2.
(2) To handle complex structures, we propose a marker-oriented sequence labeling (MOSL) module to capture the boundary information of multiword aspect/opinion terms and the shared marker information of multi-triplets.

heading: 3.1 Bidirectional Template
text: Our bidirectional templates are used to guide the generation model in an end-to-end way.
For the input review X , we construct two sentences Xa and Xo by adding two types of prompt prefix, i.e, “aspect first:” and “opinion first:”. Such prefix can prompt the model to generate target sequence with specific decoding order when we finetune the model with these templates.
To get the output triplets T in a generative manner, an essential step is linearizing triplets T into a target sequence during training and de-linearizing triplets from the predicted sequence during inference. In particular, a good output template is expected to: 1) ensure that the linearized target sequence can be easily de-linearized into a collection of triples, 2) contain specific markers to prompt the decoding process of labels, 3) free to change the order of labels. Based on the above considerations, we propose two marker-based templates ψa→o and ψo→a with different decoding orders between aspect and opinion terms as follows:
ψa→o → aspect : a,opinion : o, sentiment : s ψo→a → opinion : o,aspect : a, sentiment : s
Our output templates consist of two parts: the markers {aspect, opinion, sentiment} and the corresponding labels {a, o, s}. The markers can guide the model to generate the specific type of label at the next step. When the input review contains several triplets, we need to sort the triplet order to ensure the uniqueness of the target sequence. For the template ψa→o, we sort triplets by the end index of aspect term in an ascending order. If some triplets share the same aspect term, we further sort them by the end index of opinion term. After obtaining text segments of triplets, we use a special symbol [SSEP] to concatenate these segments to form the final target sequence.

heading: 3.2 Template-Guided Text Generation
text: We employ a standard transformer-based encoderdecoder architecture for the text generation process, and we initialize the model’s parameters with the pre-trained language model T5 (Raffel et al., 2020). For simplicity, we take the sentence Xa and the corresponding target sequence Ya based on the template ψa→o as an example for illustration. We first feed Xa into the transformer encoder to get contextual features Henc:
Henc = Encoder(Xa) (1)
We then use a transformer decoder to generate the target sequence Ya. At the t-th time step, the decoder will calculate the decoder hidden states ht based on the contextual features Henc and the previously decoded tokens y[1:t−1].
ht = Decoder(y[1:t−1],H enc) (2)
Next, ht is used to compute the conditional probability of the token yt:
p(yt|Henc; y[1:t−1]) = softmax(WTht), (3)
where W is the transformation matrix. Finally, we calculate the cross-entropy loss La→og between the decoder output and the target sequence Ya:
La→og = − L∑
i=1
log p(yt|Henc; y[1:t−1]) (4)

heading: 3.3 Marker-Oriented Sequence Labeling (MOSL)
text: The marker-based templates can prompt the generative model with the label types including aspect, opinion, and sentiment. However, the classic encoder-decoder architecture prevents the model from handling complex structures. On one hand, the decoding process is performed in a token-bytoken manner, which cannot provide clear bound-
ary information for multi-word aspect/opinion terms. On the other hand, the model needs to invoke the output templates repeatedly when the sentence contains multiple triplets. The duplicate template based decoding may cause an information confusion and sacrifice the quality of the generated text. Therefore, we propose a marker-oriented sequence labeling (MOSL) module to solve these problems. The goal is to allow the model to incorporate the prompt information of aspect and opinion terms during the generation of the specific marker 1. Fig. 3 illustrates the text generation process enhanced by the marker-oriented sequence labeling (MOSL) module.
In MOSL, we will tag aspect and opinion terms through sequence labeling. We first use two linear transformations to extract aspect features Ha = {ha1,ha2, · · · ,haL} ∈ RL×d (L is the sentence length) and opinion features Ho = {ho1,ho2, · · · ,hoL} ∈ RL×d from the contextual features Henc:
Ha = MLPa(H enc), Ho = MLPo(H enc) (5)
Then, we take the last hidden state of the decoder corresponding to the markers as the marker features, including aspect marker features Ma = {ma1,ma2, · · · ,maN} (N is the number of triplets) and opinion marker features Mo = {mo1,mo2, · · · ,moN}. We then calculate the marker-oriented features for mai ∈ Ma or moi ∈ Mo for sequence labeling:
qaij = σ(W1(h a j ⊕mai ) + b1), qoij = σ(W1(h o j ⊕moi ) + b1),
(6)
where σ(·) is the selu activation function, haj ∈ Ha and hoj ∈ Ho are the aspect/opinion features2. W and b are the transformation matrix and bias.
Note that we deploy a tag-then-generate mechanism at the training stage, which means the MOSL module will predict the BIO tags for tokens in a sentence, and then the generation model will start to decode the tokens. Such a mechanism can force the text generation module to capture the boundary information of multi-word aspect/opinion terms.
When the input sentence contains multiple triplets, the aspect/opinion marker features in different positions correspond to different tagged
1Note only aspect and opinion terms are derived from the input sentence, thus the sentiment marker is not used in MOSL.
2The tokenizer may split the word into several tokens. Here, we obtain the word-level features for MOSL by taking the maxpooling of the token-level features.
Template-Guided Generation Model
𝑋𝑋𝑎𝑎 aspect first: And the fact … speeds things up
𝜓𝜓𝑎𝑎→𝑜𝑜 𝑇𝑇𝑎𝑎 aspect: i5 processor, opinion: speeds things up, sentiment: positive 𝜓𝜓𝑜𝑜→𝑎𝑎 𝑇𝑇𝑜𝑜 opinion: speeds things up, aspect: i5 processor, sentiment: positive
𝑋𝑋𝑜𝑜 opinion first: And the fact … speeds things up
𝑇𝑇𝑓𝑓𝑓𝑓𝑓𝑓𝑎𝑎𝑓𝑓
𝑋𝑋 And the fact … processor definitely speeds things up
∩𝑇𝑇𝑎𝑎 𝑇𝑇𝑜𝑜
Encoder
Decoder MO SL Template-Guided Text Generation
Marker-Oriented Sequence Labeling
sequences in the MOSL module, e.g., Y mai = {ymai1 , ymai2 , · · · , ymaiL } for mai and Y moi = {ymoi1 , ymoi2 , · · · , ymoiL } for moi , where Y ma and Y mo are the BIO tags in sequence labeling. Hence the same marker in the generation module can share information without causing confusion since it has different pointers referring to multiple aspect/opinion terms in MOSL, which consequently benefits the decoding of the sentence containing multiple triplets. Then, we feed the markeroriented features into a fully connected layer to predict the tags of aspect/opinion terms and get the predicted probabilities over the label set:
pmaij = softmax(W2q a ij + b2), pmoij = softmax(W2q o ij + b2),
(7)
The training loss for MOSL is defined as the cross-entropy loss:
La→om = − N∑
i=1
L∑
j=1
∑ c∈C I(ymaij = c) · log(pmai,j|c)
− N∑
i=1
L∑
j=1
∑ c∈C I(ymoij = c) · log(pmoi,j|c),
(8)
where I(·) is the indicator function, ymaij and ymoij are the ground truth labels, and C denotes the {B, I, O} label set. Training For a better understanding of bidirectional dependency and also for less space cost, we jointly optimize two bidirectional templates for the sentence and label pair (X , T ):
L = λ(La→og + La→om ) + (1− λ)(Lo→ag + Lo→am ), (9)
where λ is a hyper parameter to control the contributions of different templates.

heading: 3.4 Inference
text: Constrained Decoding (CD) During inference, we employ a constrained decoding (CD) strategy to guarantee the content and format legitimacy, which is inspired by Bao et al. (2022); Lu et al. (2021). The content legitimacy means that aspect/opinion terms should be a single word or multiple continuous words in the input sentence, and the sentiment must be either positive, neutral, or negative. The format legitimacy means that the generated sequence should meet the formatting requirements defined in the template.
Both types of legitimacy can be viewed as the constraint on the candidate vocabulary during the decoding process. Before decoding, we enumerate the candidate vocabulary for each token in the input sentence and templates. We then use the constrained decoding strategy to adjust the candidate vocabulary according to the current input token at each decoding time step. For example, when we input the start token “
” to the decoder, the candidate token should be “aspect”/“opinion” to guarantee the format legitimacy. When we input “:”, the model needs to determine which is the first word of the aspect/opinion term, and the candidate tokens should be consistent with those in the input sentence. Triplet De-linearization So far, we have generated two sequences Ya and Yo based on two input sentences Xa and Xo with the constrained decoding strategy. We then de-linearize them into two triplet sets Ta and To according to pre-defined templates ψa→o and ψo→a. We take the intersection of Ta and To as the final prediction results.

heading: 4 Experiments

heading: 4.1 Datasets
text: Our proposed model is evaluated on four ASTE datasets released by Xu et al. (2020) which correct the missing triplets that are not explicitly annotated in the previous version (Peng et al., 2020). All
datasets are based on SemEval Challenges (Pontiki et al., 2014, 2015, 2016) and consist of reviews in the laptop and restaurant domains. Table 1 shows the statistics of four benchmark datasets.

heading: 4.2 Implementation Details
text: As mentioned in Sec. 3.2, T5-Base (Raffel et al., 2020) is used to initialize the parameters of our model. We train our model using AdamW optimizer with an initial learning rate 3e-4 and linear learning rate decay. The number of training epoch is set to 20 for full supervised settings and 200 for low-resource and few-shot settings. When encoding the bidirectional dependency jointly, we set the batch size to 32 and λ to 0.5. The results for supervised and low-resource settings are averaged over five and ten runs with different random initialization, respectively. All experiments are conducted on an NVIDIA RTX 3090 GPU.

heading: 4.3 Baselines
text: To validate the effectiveness of our proposed model, we compare it with 14 state-of-art baselines. We divide the baselines into three categories. (1) pipeline methods: CMLA+, RINANTE+, Liunified-R, and Peng-two-stage are proposed by Peng et al. (2020). (2) unified non-generative methods: JET-BERT (Xu et al., 2020), OTE-MTL (Zhang et al., 2020), GTS-BERT (Wu et al., 2020b), SPAN-ASTE (Xu et al., 2021), BMRC (Chen et al., 2021), EMC-GCN (Chen et al., 2022). (3) generative methods: BART-GEN (Yan et al., 2021), GAS (Zhang et al., 2021b), PARAPHRASE (Zhang et al., 2021b), SSI+SEL (Lu et al., 2022).

heading: 4.4 Main Results
text: Supervised settings Table 2 shows the triplet extraction performance under supervised settings. Our proposed SLGM method beats all baselines in terms of F1 scores. Specifically, our SLGM outperforms the best text-generation method SSI+SEL by 2.48, 2.64, 4.72, and 2.54 points on four datasets, respectively. Moreover, our SLGM can exploit knowledge for triplet extraction directly from training data, contradicting to SSI+SEL’s pre-training
method which relies on extra data like Wikipedia and Wikidata.
The generative methods like GAS which use the classic encoder-decoder architecture can outperform most non-generative methods without complicated architectures through learning label semantics. We also find that the non-generative method BMRC achieves competitive precision scores on four datasets because it also considers the bidirectional dependency. By combining the text generation and sequence labeling in training for tackling the complex extraction scenarios, our SLGM method improves the precision of GAS by more than 7 points and the recall of BMRC by more than 10 points. Low-resource settings To validate the model’s performance in the low-resource scenarios, we fol-
low the settings in SSI+SEL (Lu et al., 2022) to conduct experiments on six different partitions of the original training sets (1/5/10-shot, 1/5/10%- ratio) and report averaged scores over random 10 runs. SSI+SEL adopts a pre-training process which can help the model capture general information from additional data. However, as shown in Table 3, our SLGM achieves much better results than SSI+SEL by a large margin on all partitions without such a pre-training process. The performance gap between our SLGM and SSI+SEL becomes more impressive under the low-resource settings than that under the supervised ones. This clearly demonstrates that our SLGM model can be quickly adapted to the low-resource scenarios with very few samples, which is an extremely good property of our model.

heading: 5 Analysis

heading: 5.1 Ablation Study
text: To examine the impacts of three key components in our model, including marker-oriented sequence labeling (MOSL), bidirectional templates (ψa→o and ψo→a), and constrained decoding (CD), we conduct the ablation study on four datasets under supervised settings, The results are shown in Table 5. We make the following notes.
Firstly, removing one of two bidirectional templates will cause a performance drop, and ψa→o contributes more to the model than ψo→a.
Secondly, the extraction performance decreases dramatically after removing MOSL. This clearly proves the effectiveness of MOSL module. We will make more exploration about the impacts of MOSL in Sec. 5.3.
Thirdly, “w/o CD” denotes that we directly take the whole vocabulary instead of taking the format and content constraints into account. We find that the performance slightly degrades on Lap14, Res14, and Res15, but increases on Res16. The reason might be that limiting the size of candidate vocabulary leads the model to generate some wrong but legal triplets. However, the large amount of
training data under the supervised settings allows the model to adaptively fit to the target text.
To confirm this hypothesis, we further investigate the impacts of CD under the low-resource settings on the Res16 dataset 3. The results are shown in Table 6. We can see that as the number of training samples decreases, the performance gain from CD becomes more significant. This infers that the CD strategy plays a more important role in data scarcity scenario.

heading: 5.2 Impacts of Bidirectional Templates
text: We model the mutual dependency between aspect and opinion terms using the bidirectional templates. Our purpose is to avoid generating false paired aspect-opinion triplets. We investigate the impacts of bidirectional templates and show the results in Table 4. Besides, we also plot the performance under different settings of λ to further validate the importance of bidirectional dependency as shown in Fig. 4.
It can be seen that the unidirectional decoding order Ta/To gets better recall scores but generates many false triplets, and thus has low precision. By capturing the mutual dependency and taking the intersection of Ta and To, our model can effectively filter false paired triplets and significantly enhance
3We have similar observations on other datasets. We omit those results for clarity.
the precision and F1 scores. Moreover, when λ is biased towards ψa→o or ψo→a, the performance tends to decrease. Meanwhile, when λ is set to 0.5, the model achieves optimal results on most of the datasets. This further confirms that the bidirectional dependency is of the same importance.

heading: 5.3 Impacts of Marker-Oriented Sequence Labeling (MOSL)
text: Table 1 shows that multi-word triplets account for roughly one-third of all triplets while about half of the sentences are multi-triplet ones. Our MOSL module allows the model to learn the prompt information of aspects and opinions based on our tagthen-generate mechanism during training, which improves the model’s ability of handling complex structures. We verify the effects of MOSL in this section 4.
Table 7 shows the performance with two different evaluation modes, where “Single-Word” denotes both aspect and opinion terms in a triplet are single-word spans, and “Multi-Word” denotes that at least one of the aspect or opinion terms in a triplet is a multi-word span. We find that the model obtains more significant improvements for multi-word triplets than that for single-word triplets after adding the MOSL module. It shows that the model can learn the boundary information of aspect/opinion terms and generate the complete terms with the guidance of MOSL.
Table 7 also presents the results for “Single-” or “Multi-” triplets in a sentence, where the MOSL
4Note that the sentences with multi-word triplets and the multi-triplet sentences overlap in many cases. Hence the impacts of MOSL may not clearly present as expected on some datasets like Res15 or Res16.
module makes the similar contributions. As can be seen, the model with MOSL gains more improvements when the review contains multiple triplets. In addition, we attempt to mix the test sets of datasets Res14, Res15, and Res16 to evaluate the performance of the model under multi-triplet setting 5. The ratio of the averaged improvement of the multi-triple to the single-triple setting on three single dataset is 1.77 while it increases up to 3.15 on the mixed dataset. This is because all aspect/opinion features in MOSL point to the same marker “aspect/opinion”. This allows the marker to share knowledge across different aspect/opinion features, thus the text generation module holds the clue from the shared marker about the subsequent aspect/opinion term when generating the prior ones.

heading: 5.4 Analysis on Computational Cost
text: To demonstrate that our model does not bring too much computational cost, we compare it with GAS in terms of the number of parameters and inference time as shown in Table 8. We also analyze the costs of the key components in our model to show their impact on complexity. Firstly, the MOSL module adds only about 2.3M parameters compared with GAS. Secondly, we find that the constrained decoding algorithm increases the inference time as our implementation of constrained decoding algorithm requires determining the candidate vocabulary according to the current input token at each decoding time step, which undermines the parallelism of the generation model during inference. Moreover, bidirectional templates require the model to generate target sequences based on two different decoding orders which also increases inference time to some extent. However, SLGM does not show significant differences from GAS in terms of model parameters and inference time because GAS needs to take a prediction normalization strategy to refine the
5Here we still take the training set of Res15 for training.

heading: At home , so built in screen size is not terribly important .

heading: At home , so built in screen size is not terribly important .
text: At home and the office it gets plugged into an external 24 ” LCD screen, so built in screen size is not terribly important.
prediction results.

heading: 5.5 Case Study
text: We conduct a case study on two reviews to compare typical generative methods, including PARAPHRASE (Zhang et al., 2021a), SSI+SEL (Lu et al., 2022), and our method. The results are as shown in Fig. 5.
For the first review (the left one in Fig. 5), SSI+SEL and PARAPHRASE cannot recognize the opinion term “cheap”, whereas “not very sensitive” is recognized by all methods. In contrast, our SLGM can identify both terms. To have a close look, we further visualize the BIO probabilities output by MOSL in Fig. 6. As we can see in the left part of Fig. 6, the opinion marker in MOSL focuses on two opinion terms simultaneously when the generation module generates the first triplet, which helps the model know that there are two related opinion terms for the aspect term “keyboard”.
For the second review (the right one in Fig. 5), both SSI+SEL and PARAPHRASE find the approximate locations of the aspect and opinion terms, but neither of them gets correct pairs due to incomplete decoding. The reason is that these two methods lack the corresponding prompt information for boundary identification. Meanwhile, as can be seen from the right part of Fig. 6, the aspect marker in MOSL focuses on the complete aspect term, which contains the boundary information that can help our generation module to decode the complete aspect term.

heading: 6 Conclusion
text: In this paper, we exploit the power of text generation and sequence labeling for ASTE. We propose two bidirectional templates to reflect the mutual aspect-opinion dependency for filtering false paired triplets. We also present a marker-oriented sequence labeling module to help the text generation module tackle complex structures in the subsequent decoding process. Experiment results show that our framework consistently outperforms all generative and non-generative baselines under both the full supervised and low-resource settings.
Limitations
Although our proposed method achieves the stateof-art performance, it still has a few limitations. Firstly, we only consider the dependency between aspect and opinion in the target text yet ignoring the order influence in the input text, which may bring more improvements. Secondly, there are three label types for ASTE, including aspect, opinion, and sentiment. Currently, we only utilize the aspect and opinion markers in the marker-oriented sequence labeling module. We believe that the specific design for the sentiment marker can further improve the performance, which can be a future direction.

heading: Acknowledgments
text: This work was supported by a grant from the National Natural Science Foundation of China (NSFC) project (No. 62276193).

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitations
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4.1

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4.1
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4.1
C 3 Did you run computational experiments? Section 4, Section 5.1-5.4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 5.4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4.2

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4.2
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We believe that the specific design for the sentiment marker can further improve the performance, which can be a future direction