Refining and Synthesis: A Simple yet Effective Data Augmentation Framework for Cross-Domain Aspect-based Sentiment Analysis (2024)

Abstract: Aspect-based Sentiment Analysis (ABSA) is extensively researched in the NLP community, yet related models face challenges due to data sparsity when shifting to a new domain. Hence, data augmentation for cross-domain ABSA has attracted increasing attention in recent years. However, two key points have been neglected in prior studies: First, target domain unlabeled data are labeled with pseudo labels by the model trained in the source domain with little quality control, leading to inaccuracy and error propagation. Second, the label and text patterns of generated labeled data are monotonous, thus limiting the robustness and generalization ability of trained ABSA models. In this paper, we aim to design a simple yet effective framework to address the above shortages in ABSA data augmentation, called Refining and Synthesis Data Augmentation (RSDA). Our framework roughly includes two steps: First, it refines generated labeled data using a natural language inference (NLI) filter to control data quality. Second, it synthesizes diverse labeled data via novel label composition and paraphrase approaches. We conduct experiments on 4 kinds of ABSA subtasks, and our framework outperforms 7 strong baselines, demonstrating its effectiveness.

heading: 1 Introduction
text: Aspect-based Sentiment Analysis (ABSA) is a fundamental sentiment analysis task that aims to analyze sentiments at the aspect level (Liu, 2012; Xue and Li, 2018). It usually involves extracting several sentiment elements, including aspects, opinions, and sentiment polarities. For example, given a sentence: “It is the best sushi I ever had”, the aspect term is “sushi”, the corresponding opinion term is “best” and the sentiment polarity is “positive”. ABSA has attracted more and more attention in the past decade (Nguyen and Shirai,
* Corresponding author.
2015; Zhang et al., 2023b), with the development of deep learning, many models and methods can achieve good results on the aspect-level sentiment analysis dataset (Yadav et al., 2021; Zhang et al., 2023b). Most methods for model training only use same domain data and require fine-grained labeled data (Ding et al., 2017). This is problematic in nascent domains with little labeled data, impeding robust performance. Some studies focus on developing models with domain migration capabilities to address these challenges (Zhang et al., 2022) . Other works employ domain adaptation technology to transfer learned knowledge from labeled source domains to unlabeled target domains (Deng et al., 2023). However, the majority of these studies are based on discriminative models (Zhang et al., 2021a), necessitating customized design for specific tasks. In addition, other works resort to domain-specific dictionaries, using rule-based or neural network-based methods (Marcacini et al., 2018; Howard et al., 2022) to obtain external semantic dictionaries. While these approaches have demonstrated commendable performance on par-
10318
ticular datasets, their excessive reliance on external knowledge impairs their generalization capacity.
Recently, the methods, which integrate various tasks into a unified framework by formalizing each task as a sequence-to-sequence problem, have achieved promising results (Wang and Wan, 2023). In the cross-domain low-resource scenario, the feasibility of the cross-domain data augmentation method based on such framework has also been verified (Yu et al., 2023; Ghosh et al., 2023a). Particularly, Deng et al. (2023) proposes a data augmentation framework, which extracts pseudolabels from target domain sentences and then generates new sentences based on these pseudo-labels to synthesize labeled data. Despite achieving promising results, the existing data augmentation frameworks primarily have the following shortcomings:
• Low-quality Samples and Error Propagation. The target data generated by pseudolabels is error-prone because the extraction model is trained using labeled data from the source domain. Figure 1(a) shows an incorrectly generated sentence due to error propagation caused by a pseudo label.
• Limited Data Diversity. The diversity of the generated labeled data in the target domain is limited due to the constraints imposed by the scales of text generation models and the categories of pseudo-labels. Figure 1(b) shows that when two identical pseudo-labels are extracted, it often results in the generation of identical new sentences, even if their sources are different.
Towards this end, we propose a novel two-step data augmentation framework for cross-domain ABSA tasks named Refining and Synthesis Data Augmentation (RSDA). In the first step, our framework follows previous work (Deng et al., 2023) by extracting a pseudo label l from an unlabeled sentence t in the target domain. Subsequently, it generates a new sentence t′ aligned with the pseudo label l, thereby producing a labeled sample (t′, l) in the target domain. Then, our framework further employs an approach based on natural language inference (NLI), named the NLI filter (Sileo, 2023), to eliminate invalid samples by determining whether t and t′ are in an entailment relationship. By employing this approach, we can obtain higher-quality labeled samples in the target domain.
In the second step, we design two novel diversity enhancement modules to mitigate the duplication
and oversimplification of model-generated target domain labeled samples. The first module is called composition-based diversity enhancement, which combines two selected labels into a longer one and then generates a new sentence by the generation model. The compositions of various labels will definitely increase the diversity of generated data. On the other hand, we propose a paraphrase-based diversity enhancement module that tackles data augmentation diversity from two perspectives, namely label-variant paraphrase and label-invariant paraphrase. The former directly paraphrases the unlabeled text in the target domain and extracts pseudolabels from it, similar to the process in the first step. In contrast, the latter focuses on paraphrasing the target domain labeled text while retaining the original labels but altering their contextual representation. These two paraphrase methods complement each other effectively.
To validate the effectiveness of our framework, we conduct extensive cross-domain experiments on 4 ABSA subtasks. Our framework outperforms several strong baselines by at least 1.64%, 1.39%, 1.45% and 2.04% in averaged F1s of 4 subtasks.
Our main contributions can be summarized as follows:
• We introduce RSDA, a novel data augmentation framework designed for cross-domain ABSA. Unlike previous studies, RSDA prioritizes both data quality and diversity, which have been neglected in prior studies.
• To address the issue of limited diversity in generated data, our diversity enhancement method focuses on improving diversity from two angles: information density and expression variety.
• Our framework has been tested on 32 crossdomain experiments and the superior performances compared with 7 strong baselines demonstrate its effectiveness.

heading: 2 Related work

heading: 2.1 Aspect-Based Sentiment Analysis
text: Aspect-based Sentiment Analysis (ABSA) (Liu, 2012; Xue and Li, 2018) is a well-established sentiment analysis task that encompasses various subtasks such as Aspect Sentiment Classification (AE), Aspect Extraction and Sentiment Classification (AESC), Aspect Opinion Pair Extraction (AOPE),
and Aspect Sentiment Triple Extraction (ASTE) (Yan et al., 2021). Recent advancements, particularly with models like Bart or T5, have shifted towards end-to-end architectures (Lewis et al., 2020; Raffel et al., 2020). This transition has streamlined task forms, enhancing model adaptability to ABSA’s multiple subtasks (Lu et al., 2022). However, in cross-domain settings, these methods encounter challenges due to limited training data in the target domain. The domain adaptation capability of these models requires further investigation.

heading: 2.2 Cross Domain Data Augmentation
text: Data augmentation is a widely used technique to address training data scarcity (Fadaee et al., 2017; Chao et al., 2023), proven effective in computer vision (Ye et al., 2019). However, for textual data with discrete and complex semantics, augmentation becomes challenging. Current methods include rule-based synonym replacement (Rennes and Jönsson, 2021) through conditional constraint generation or template-based approaches (Chen et al., 2021). Yet, these methods often rely on rigid rules or fixed templates, limiting model generalization and sample diversity. In the realm of cross-domain data augmentation, some approaches generate new samples by regenerating pseudo labels (Toledo-Ronen et al., 2022; Deng et al., 2023). While showing progress, these methods often neglect the issue of pseudo-label error propagation and struggle to accurately simulate real data distribution due to model and data limitations.

heading: 3 Methodology

heading: 3.1 Problem Definition
text: In this work, we focus on the unsupervised crossdomain setting (Sharma et al., 2018; Zhang et al., 2023a), which means that we only have labeled data in the source domain, namely Ds = {t, l} where t and l denote a sentence and its corresponding label (e.g., aspect, opinion, sentiment polarity). Only unlabeled data is available in the target do-
main, denoted as Dt = {t}. The objective is to leverage the training data from both the source and target domains to predict the labels of the test set in the target domain. To validate the effectiveness of our framework, we employ 4 ABSA subtasks for experiments, as shown in Table 1.

heading: 3.2 Overview
text: Our RSDA framework mainly consists of two steps as shown in Figure 2: (1) The first step is data generation and quality control. Firstly, we obtain pseudo labels and corresponding generated samples of the target domain from the extraction and generation models trained on the source domain. Then, we use a Natural Language Inference (NLI) model as a filter to remove incorrect samples. (2) The second step is data diversity augmentation. In this step, we employ composition-based diversity enhancement to make generated samples contain multiple aspects to improve information density, and paraphrase-based diversity enhancement to generate new labels or change their context.

heading: 3.3 Data Generation and Quality Control
text: Data Generation: Following previous work (Deng et al., 2023), we train both the label extraction model l = Me(t) and text generation model t = Mg(l) using the source domain data Ds = {t, l}. Both of them are based on T5-base (Raffel et al., 2020). Then, we utilize the extraction model Me to extract the pseudo label l′ from a target domain sentence t. Based on the pseudo label l′, the sample generation model Mg can generate a new sentence t′. After data generation, we obtain a new target domain labeled dataset Dtp = {t′, l′}. NLI-based Quality Control: As the generation model was trained on the source domain, it tends to generate text more aligned with the source domain, resulting in less fluent data. Moreover, the noise introduced by the extracted pseudo-labels can propagate into the generated text samples. To address these problems, we employ a Natural Language Inference (NLI) filter (Sileo, 2023) for quality control in the generation of data.
Concretely, we take the original target domain text t as the premise and the newly generated text t′ as the hypothesis. The NLI filter can determine the relationship between a pair of premises and hypotheses, formulated as:
P (y | t, t′) = f(t, t′) (1)
where y can be Entailment, Contradiction and Neutral, f denotes the NLI filter. When a contradiction relation arises between t and t′, it suggests that the generated text t′ is less likely to be inferred from the original text, which should be filtered out. After quality control, certain problematic samples are removed and the remaining high-quality labeled data are denoted as Dtf = {t′, l′}. The examples are in Appendix B.1.

heading: 3.4 Data Diversity Augmentation
text: In the previous step, we filtered model-generated target domain-labeled samples through the NLI filter. However, based on our manual observation, we identified two shortages that should be improved:
(1) As shown in Figure 1(b), the generation model tends to generate simple or duplicated sentences due to training resource limitations.
(2) Although the quality of generated data can be enhanced using the NLI filter, it sacrifices text expression and pattern diversity by filtering out part of the samples.
To address these issues, we focus on diversifying the data from two dimensions in the second step, namely information density and expression variety.

heading: 3.4.1 Composition-based Diversity Enhancement
text: First, we perform lexical clustering using the labels l′ of the labeled target domain data Dtf = {t′, l′}.
Specifically, we employ MiniLM-L61 from Sentence Transformers to encode a label l′i into its vector representation hl′i . Subsequently, the K-means clustering algorithm is applied to partition labels into K clusters, where the value of K is determined by the silhouette coefficient method (Rousseeuw, 1987). The calculation method is as follows:
s(i) = b(i)− a(i)
max{a(i), b(i)} (2)
K = argmax k   1 |D| |D|∑
i=1
s(i)   (3)
where a(i) and b(i) represent the average distance between sample i and all other points within the same cluster and in different clusters, respectively. The silhouette score for sample i is denoted by s(i), and |D| represents the total number of samples.
Then, the semantic similarity between the text of each pair of data points in the same cluster is measured by the cosine similarity, calculated as:
Sim(t′i, t ′ j) = ht′i · ht′j ∥ht′i∥∥ht′j∥
(4)
where ht′i and ht′j denote the vector representation of the text pair t′i and t ′ j also encoded by MiniLML6. Concretely, we select the data points with the 1https://huggingface.co/sentence-transformers/allMiniLM-L6-v2
lowest semantic similarity to increase the diversity of data.
Next, we concatenate l′i and l ′ j and feed them into the generation model to obtain a synthesized text t′′ = Mg(l′′) = Mg(l′i ⊕ l′j). For example, after clustering and similarity calculation, we select two labels “<pos> food <opinion> yummy” and “<pos> fish <opinion> fresh”. Then we concatenate them and input them into Mg to generate “The food in here is yummy, especially the fresh fish”. In fact, since we choose the two farthest labels in the same cluster, the merged two labels do not guarantee the same sentiment polarity, which provides more possibilities for the merged samples. By combining different labels, information density and label diversity can be enhanced.

heading: 3.4.2 Paraphrase-based Diversity Enhancement
text: For paraphrase-based diversity enhancement, we design two methods to augment the target domain labeled data. Our core idea is to use paraphrase to rewrite labels or their context, thus generating new data. The details are explained below.
Label-variant Paraphrase Due to the nonlinguistic nature of labels, we perform an indirect approach to implement label-variant paraphrase. As shown in Table 2, we apply a paraphrasing tool (Vladimir Vorobev, 2023) to the original target domain unlabeled text t and a piece of new paraphrased text can be generated, called t′. Then a pseudo label l′ can be extracted using the extraction model Me introduced in Section 3.3. Note that in this phase, because the paraphrase tool is directly applied to the raw text, all the words could be rewritten thus the extracted pseudo label could also be different from the original one. Afterwards, the generation model Mg will generate a new sentence t′′ based on the pseudo label l′. This approach not only aligns with the label-invariant paraphrase procedure but also enhances the diversity and expression of Dtp.
Label-invariant Paraphrase We also applied paraphrasing to the target domain labeled samples generated in previous steps to enrich their text patterns and avoid simple sentence structures. As shown in Table 2, we utilize prompts to encourage the paraphrase tool to include the label l′ when it rewrites the text t′ as t′′. Then, we use postprocessing methods to make sure that the paraphrased text t′′ includes the label l′. In this phase, we try to keep the label in a target domain labeled sample unchanged and meanwhile transform its context, thus more diverse data could be synthesized.

heading: 4 Experiments

heading: 4.1 Experimental Settings
text: Datasets. To validate the effectiveness of our framework in cross-domain settings, we conduct extensive experiments on benchmark datasets from four domains: Restaurant (R), Laptop (L), Device (D) and Service (S). The specific dataset statistics are illustrated in Figure 3, which are widely used for ABSA and sourced from the SemEval challenge 2014, 2015, and 2016 (Pontiki et al., 2014, 2015, 2016), reviews about digital devices (Toprak et al., 2010) and comments about web services (Hu and Liu, 2004).
Considering the domain similarities among datasets, we select several distinct source-to-target
domain pairs for experimentation. In AE and AESC tasks, following prior work (Yu et al., 2021; Gong et al., 2020; Yu et al., 2023), we exclude experiments involving transfers between L and D domains due to their high similarity. Our experiments exclusively utilize the R and L datasets for AOPE and ASTE tasks, owing to limitations in data sources. Additionally, the experiments between R14, R15, and R16 were omitted because they share the same domain (Deng et al., 2023). Consequently, there are a total of 10 transfer experiments for AE, AESC and 6 transfer experiments for AOPE and ASTE. Our final training dataset includes the original domain label dataset and the target domain dataset processed by the RSDA framework. Evaluation Metrics. We choose Micro-F1 as the primary evaluation metric for our experiments, considering a predicted label correct only when it fully matches the gold label (Lu et al., 2022). Additionally, to assess the diversity of enhanced samples, we employ diversity evaluation metrics, following Ghosh et al. (2023b); Yu et al. (2023). Specifically, Da indicates the percentage of unique aspect terms among all aspect terms, while Do represents the percentage of unique opinion terms among all opinion terms. For AE and AESC tasks, the diversity of generated samples is determined solely by Da, while for AOPE and ASTE tasks, the diversity is the average of Da and Do. Parameter Settings. For extraction and generation models, we choose T5-base checkpoint from Hugging Face2. The architecture of the T5 model is based on the transformer model, comprising
2https://huggingface.co/google-t5/t5-base
encoder and decoder components. We employ a T5-base paraphraser (Vladimir Vorobev, 2023) for paraphrase-based diversity enhancement. Moreover, we utilize the Adam optimizer with a learning rate of 3e-4 and a batch size of 16 for all tasks. All experiments are conducted on a single NVIDIA 3090 GPU. For further details, please refer to Appendix A.1.

heading: 4.2 Comparison with Other Approaches

heading: 4.2.1 Approach Introduction
text: For AE and AESC tasks, we follow previous work (Yu et al., 2021, 2023) and choose baselines including BERT-UDA (Gong et al., 2020), CDRG (Yu et al., 2021), GAS (Zhang et al., 2021b), DA2LM (Yu et al., 2023), BGCA (Deng et al., 2023). Both BERT-UDA and CDRG utilize the BERT base, while GAS, and BGCA are built upon the T5-base. The base models of baselines are of the same order of magnitude as the one we use.
As for ASTE, our selected baselines include RoBMRC (Liu et al., 2022), SpanASTE (Xu et al., 2021), GAS (Zhang et al., 2021b), and BGCA (Deng et al., 2023). For AOPE, we adapted RoBMRC and SpanASTE by excluding sentiment polarities to accommodate the task, following Deng et al. (2023).

heading: 4.2.2 Result Comparison
text: The overall results of AE and AESC tasks in the cross-domain setting are presented in Table 4. It can be observed that our proposed framework outperforms the state-of-the-art method BGCA in the majority of domain pairs across ten different crossdomain settings. Overall, our approach achieves a 1.64% absolute improvement in averaged Micro-F1
compared to BGCA in the AE task, and a 1.39% improvement in the AESC task.
For the AOPE and ASTE tasks, we conduct experiments on six different domain pairs as shown in Table 5. Our framework shows an averaged F1 improvement of 1.45% in the AOPE task and 2.04% in the ASTE task. Notably, it also achieves 1.02% ∼ 3.58% improvement in F1 score where R serves as the source domain and L as the target domain.
Additionally, through the experiments presented in Tables 4 and 5, we note the following observations:
(1) Our proposed framework consistently outperforms BGCA across all four tasks on average. We attribute this improvement to the incorporation of quality filtering and diverse enhancement strategies throughout the entire process, ensuring the quality of generated samples. Moreover, our framework is fully compatible with BGCA and serves as its further optimization.
(2) We observe that methods based on encoderdecoder structures such as T5 perform better than those based on BERT. We speculate that generative models, with their encoder-decoder architecture, excel in handling abstract tasks by better capturing contextual information. They comprehensively understand the entire text through self-attention mechanisms and recursive mechanisms.
(3) Our framework performs less effectively than DA2LM in certain domain pairs, particularly in cross-domain experiments where S serves as the source domain. We identify a challenge in experiments where the data volume in the source domain is significantly lower than that in the target domain. We hypothesize the possible reason is that the model fails to receive sufficient training in both extraction and generation, limiting subse-
quent results in terms of extraction and generation capabilities.

heading: 4.3 Ablation Study
text: To analyze the effectiveness of our framework, we conduct ablation experiments using micro-F1 and diversity as metrics, and the specific results are shown in Table 6. Firstly, when we remove the NLI filter, we observe a significant drop of approximately 3.98% in F1 scores across all four tasks. This indicates the effectiveness of NLI-based quality control, as the NLI filter eliminates examples with semantic and format errors. The removal of composition-based diversity enhancement leads to an average decrease of approximately 2.25% in F1 scores, with particularly notable impacts observed in the ASTE and AOPE tasks. We speculate that the composition-based diversity enhancement has a more noticeable impact on tasks with richer label entailment information. Thirdly, removing paraphrase-based diversity enhancement leads to an average F1 score decrease of approximately 0.55% across all four tasks.
In addition, to assess the contributions of composition-based diversity enhancement, we conduct ablation experiments for it, the results are as shown in Figure 4. We use the proportion of generated samples with multi-aspect as a metric. Removing the composition-based diversity enhancement resulted in varying degrees of reduction in this metric across all four tasks, with ASTE and
AOPE tasks experiencing nearly a 50% decrease, demonstrating that the composition-based diversity enhancement indeed enhances the information density of samples.

heading: 4.4 Further Analysis

heading: 4.4.1 Quality Assessment of Generated Data
text: To demonstrate the effectiveness of our framework in the cross-domain setting, we conduct quality assessments of generated samples for the AESC task across ten domain pairs.
We employ perplexity to measure the fluency of the generated samples and adopt GPT-23 for perplexity calculation following Yu et al. (2023). Given that the BGCA method did not generate additional data, resulting in a limited number of generated samples, for fairness, our experiments randomly select and test 500 samples generated by each method in perplexity testing. The results in Table 7 indicate that the perplexity of the samples generated by our framework is significantly lower than those of other methods. We speculate that the NLI filter effectively alleviates the issue of nonfluent generated samples caused by domain shift phenomena.
We also employ Da to assess the distribution of generated samples. As shown in the last four rows of the table 7, our model exhibits higher diversity than other methods. It is noteworthy that, in the D→S task, although the DA2LM method has a higher diversity value than our approach, our framework achieves an F1 score 12.91% higher. This suggests that our approach not only enhances the diversity of generated samples but also covers more aspect terms in the target domain.

heading: 4.4.2 Visual Case Study on Clustering
text: As described in Section 3.4.1, we conduct clustering on the filtered data and visualize the results as shown in Figure 3. The dataset for the target domain is denoted as L, focusing on the ASTE task. In the illustration, we have highlighted two samples from the same cluster with the farthest distance. It is evident from the figure that our algorithm not only ensures coherent labeling but also enhances diversity in the combinations, thereby achieving a balance between logical label pairing and increased variety.

heading: 5 Conclusion
text: In this paper, we propose a two-step data augmentation framework for cross-domain ABSA tasks. The first step controls sample quality and filters low-quality pseudo labels using the NLI filter. The second step enhances the diversity of augmented data using label composition and paraphrase methods. We conduct 32 experiments in cross-domain
3https://huggingface.co/evaluate-measurement
settings to demonstrate the effectiveness of our framework, which outperforms 7 strong baselines.4 Our approach not only mitigates error propagation caused by incorrect pseudo-labels but also enhances the diversity and fluency of the generated labeled data in the target domain. It is simple yet effective to implement and extend to other domains and tasks without much effort. In the future, we will explore the generalization ability of our framework to other structural information extraction tasks.

heading: Acknowledgments
text: This work is supported by the National Key Research and Development Program of China (No. 2022YFB3103602), the National Natural Science Foundation of China (No. 62176187), the open project of Sichuan Provincial Key Laboratory of Philosophy and Social Science for Language Intelligence in Special Education (No. YYZN-2023-1).
Limitations
While our method has achieved promising results, there are still several limitations to be addressed. Although our approach effectively enhances information density through composition-based diversity enhancement, this advantage is more pronounced when label information is abundant. Further investigation is needed on how to improve performance in scenarios with limited label information. Additionally, our framework has only been tested on sentiment analysis datasets, and its applicability to other tasks remains to be explored.
Ethics Statement
We conduct extensive experiments on benchmark datasets from four domains: Restaurant (R), Laptop (L), Device (D) and Service (S), which are widely used for ABSA tasks. These datasets do not include personal information or contain sensitive content. In the process of generating data, we employ constrained decoding and quality control methods, which to some extent mitigate the presence of harmful content. However, human review is necessary when using these data in real-world applications.
4Our codes are available at RSDA.

heading: A Experimental Details and Additional Experiments
text: A.1 Training Settings
For the extraction model, we adopted constrained decoding, confined to the vocabulary of the target domain. In selecting the training epochs for the four tasks, we drew inspiration from the approaches of Zhang et al. (2021b) and Deng et al. (2023), and extended our experimentation beyond the {15, 20, 25, 30} epochs. As for the paraphrasing model, to generate more diverse text, we set the temperature to 0.7. To ensure fairness, the amount of data generated by our method is kept in the same order of magnitude as the BGCA method. Furthermore, we performed post-processing(Deng et al., 2023) on the generated data to further ensure sample accuracy, including deduplication, format checking, and regeneration as needed. Ultimately, our training dataset includes both the source domain data and the target domain dataset obtained after RSDA framework processing.
A.2 Supplementary Experiments
Figure 4 shows the ablation experiments of composition-based diversity enhancement on four tasks, with the metric being the proportion of samples with multiple aspects in the generated data. It can be seen from the graph that the compositionbased diversity enhancement we adopted has greatly improved the performance, especially on tasks AOPE and ASTE.

heading: B Case Studies
text: B.1 Applications of NLI Filter
Figure 5 illustrates two applications of the NLI filter. As for (a) in Figure 5, the diagram depicts how the generation of new samples in the target domain can be influenced by data from the source domain, resulting in domain shift phenomena that may significantly affect the fluency of generated samples. However, the NLI filter effectively identifies and filters out such examples promptly. In (b), the diagram shows an entailment relationship between the unlabeled text in the target domain and the newly generated text, indicating that such examples should be retained. Through NLI filter processing, we can filter out samples with semantic inconsistencies, such as those influenced by the source domain or model hallucinations.
B.2 Examples for Composition-based Diversity Enhancement
The merging process is illustrated in the Figure 6. We concatenate the labels and texts of the farthest two examples in the same cluster, resulting in lc and tc. Then, we utilize the generation model Mg to obtain a tn that is smoother than tc. This constitutes
a newly labeled sample (tn, lc). Finally, all samples undergo quality control again with the NLI filter. For tasks like AE with scarce label information, we adopt a random token selection approach to augment the label information in both training and inference processes.

heading: Future Work
text: In the future, we will explore the generalization ability of our framework to other structural information extraction tasks