A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis (2024)

Abstract: Most existing work on aspect-based sentiment analysis (ABSA) focuses on the sentence level, while research at the document level has not received enough attention. Compared to sentence-level ABSA, the document-level ABSA is not only more practical but also requires holistic document-level understanding capabilities such as coreference resolution. To investigate the impact of coreference information on document-level ABSA, we conduct a three-stage research for the document-level target sentiment analysis (DTSA) task: 1) exploring the effectiveness of coreference information for the DTSA task; 2) reducing the reliance on manually annotated coreference information; 3) alleviating the evaluation bias caused by missing the coreference information of opinion targets. Specifically, we first manually annotate the coreferential opinion targets and propose a multi-task learning framework to jointly model the DTSA task and the coreference resolution task. Then we annotate the coreference information with ChatGPT for joint training. Finally, to address the issue of missing coreference targets, we modify the metrics from strict matching to a loose matching method based on the clusters of targets. The experimental results not only demonstrate the effectiveness of our framework but also reflect the feasibility of using ChatGPT-annotated coreferential entities and the applicability of the modified metrics. Our source code is publicly released at https://github.com/NUSTM/DTSA-Coref.

## introduction aspect-based sentiment analysis (absa) aims to extract opinion targets from review texts and the sentiment towards each opinion target @xcite . for example, in the sentence "the food is delicious.", the opinion target is "food" and its sentiment is positive. early work @xcite employed rules or machine learning methods to solve the task on their annotated datasets. after the introduction of absa evaluation tasks and benchmark datasets in @xcite @xcite , subsequent work deepened the research on absa, exploring different model paradigms in effects on absa @xcite @xcite , also by improving the task definition, providing rich and diverse absa datasets to extract opinion targets and their sentiments more accurately @xcite @xcite . despite these advancements, most existing work focuses on the sentence level, overlooking the fact that product reviews are typically presented in the form of documents, which is not only more practical but also enables the utilization of valuable document-level information for absa tasks. recently, @xcite introduced a task of document-level target sentiment analysis (dtsa), hierarchically depicting the opinion target. as shown in figure 1 , the review mentions three opinion targets: "work shoes", the "leather" of the work shoes (represented as "work shoes-leather"), and the "toe area" of the work shoes (represented as "work shoes-toe area"). it expresses a positive sentiment towards "leather", a negative sentiment towards "toe area", and a mixed sentiment towards "work shoes". a typical phenomenon in the dtsa task is the coreference of opinion targets, especially across multiple sentences. as the example shown in figure 1 , the "work shoes" have two coreferential mentions "they" and "they", and the three aspect terms "work shoes", "they", and "they" refer to the same opinion target. combining context understanding of these pronouns' referent may help differentiate the sentiment of different opinion targets and accurately identify the hierarchical relation among opinion targets. although such coreference phenomenon is prevalent in review texts and can affect the accuracy of opinion target extraction in the absa research, very few studies have investigated the coreference problem @xcite , especially in the dtsa task, which is relatively complex and has a higher incidence of coreference in review texts. currently, there is no relevant research addressing coreference problems in this task. to explore the impact of coreference resolution on the dtsa task, the following three questions should be addressed: first, in the end-to-end dtsa task, does the coreference information contribute to the accurate identification of hierarchical relations among opinion targets and their sentiments? second, coreference annotation requires substantial human effort and time. is it possible to mine coreference information without manual efforts? third, the same opinion target may have different forms of textual expression, yet existing absa datasets often select only one expression as the ground truth label. this approach may treat coreferent expressions of the same target as incorrect during evaluation. how can we mitigate the evaluation errors caused by the omission of coreference annotations? to address the first question, we annotate the coreferential targets of the opinion targets, proposing a coreference-aware joint model to investigate whether the coreference information can improve the model's ability to identify hierarchical opinion targets and their sentiments. the model employs a dual-path architecture, utilizing multi-task learning to jointly handle the dtsa task and coreference resolution task. the left path is designed to identify opinion targets, their sentiments, and the hierarchical relations between them. the right path focuses on identifying the opinion targets and their coreferential relations. by sharing parameters and aligning tokens between the two paths, the model can learn to leverage coreference information and enhance its performance on the dtsa task. in response to the second question, we leverage chatgpt to annotate the coreference information and verify the efficacy of the coreference annotation in the joint model. to answer the third question, we modify the metric to employ a cluster matching method that takes into account coreferent opinion targets. the experimental results demonstrate the effectiveness of introducing coreference information through a dual-path model. additionally, the coreference annotations provided by chatgpt can achieve comparable dtsa performance to those of human annotations. finally, the cluster-level evaluation metric is also proven to be more reasonable in assessing the effectiveness of opinion target extraction in consideration of the coreference problem. ## related work in recent years, absa has been extensively studied by researchers, and works are primarily focused on the sentence level, emphasizing either singleelement extraction or multi-element extraction. sentence-level absa. early sentence-level absa @xcite often constructed datasets sourced from twitter. following the introduction of the absa task through semeval evaluations by @xcite @xcite , subsequent research proposed numerous methods for sentencelevel absc @xcite @xcite . additionally, a portion of the work further focused on the end-to-end absa task, mainly aiming to jointly extract the aspects and their sentiments within sentences. @xcite @xcite have introduced various solutions based on encoder-only models for this task. with the development of generative language models, @xcite subsequently proposed solutions based on bart or t5. document-level absa. a few absa work was conducted at the document level. @xcite annotated documentlevel absa datasets across multiple domains and researched multiple aspect extractions on these datasets. @xcite ) investigated the con-sistency of the same aspect's sentiments across documents as well as the correlation between sentiments across different aspects. @xcite introduced the dtsa task and proposed a framework based on bart to solve this task. furthermore, @xcite introduced an encoderbased sequence-to-structure framework to explicitly model the hierarchical relations between entities. our work, taking dtsa as a starting point, investigates the impact of coreference on dtsa. coreference resolution. coreference resolution aims to identify all expressions that refer to the same entity from the text. since @xcite first introduced a deep learning method for end-toend coreference resolution, coreference resolution has been increasingly integrated into related downstream tasks. @xcite ) marked entities requiring pronoun resolution in absa datasets. @xcite introduced a coreference classification task for objects and entities in comparative reviews. building on this, @xcite proposed a method for automatic mining and utilizing domain-specific knowledge to address coreferences of entities. moreover, @xcite investigated whether the coreference resolution of entity is beneficial for the absc task. ## problem definition in traditional sentence-level absa tasks, given a review sentence s, the goal is to identify the opinion targets (fine-grained entities or their aspects, collectively referred to as entities) mentioned in the sentence and their corresponding sentiment {. . . , (a i , p i ), . . .}, where a i represents the extracted i-th entity, typically presented as a continuous text span in the sentence, and p i denotes the sentiment towards a i , such as positive, negative, or neutral. in the dtsa task, the input extends from a single sentence to a document @xmath0, . . . , s n ] consisting of n sentences. the output aims to identify all hierarchical entities and their corresponding sentiments {. . . , (t i , p i ), . . .}. unlike the flat entity a i in sentence-level absa, t i represents a multi-level entity composed of multiple flat entities. we will use hierarchical entities to represent multi-level opinion targets in the following. it can be seen that extracting hierarchical entities in the dtsa task is more challenging than sentence-level absa tasks, particularly when the metric requires an exact match with the ground truth. considering the entity coreference in documents, we also modify the metric from the exact entity-level matching to a cluster-level matching. ## methodology to investigate whether the information of entity coreference can aid in identifying hierarchical entities and their sentiments, we first annotate the coreferential entities for each entity a i , forming a coreference cluster c @xmath1, . . . , a i k } ( §4.1). next, we incorporate the coreference information into the dtsa task through multi-task learning, leveraging both human-annotated and machine-annotated coreference information to improve the model's ability to recognize coreference ( §4.2). ## conclusion most research on absa focuses on the sentence level. however, dtsa remains an underexplored area. a significant difference between dtsa and sentence-level absa is that dtsa requires richer coreference information, necessitating models to possess stronger contextual understanding capabilities. to explore the impact of coreference information on the dtsa task, we annotate coreferential entities with human and chatgpt and design a multi-task learning framework to verify the positive role of coreference information in dtsa. additionally, we revise the metrics from exact entity-level matching to a more lenient cluster-level matching to mitigate the bias caused by missing coreferential entities.