Revisiting Structured Sentiment Analysis as Latent Dependency Graph Parsing (2024)

Abstract: Structured Sentiment Analysis (SSA) was cast as a problem of bi-lexical dependency graph parsing by prior studies.Multiple formulations have been proposed to construct the graph, which share several intrinsic drawbacks:(1) The internal structures of spans are neglected, thus only the boundary tokens of spans are used for relation prediction and span recognition, thus hindering the model’s expressiveness;(2) Long spans occupy a significant proportion in the SSA datasets, which further exacerbates the problem of internal structure neglect.In this paper, we treat the SSA task as a dependency parsing task on partially-observed dependency trees, regarding flat spans without determined tree annotations as latent subtrees to consider internal structures of spans.We propose a two-stage parsing method and leverage TreeCRFs with a novel constrained inside algorithm to model latent structures explicitly, which also takes advantages of joint scoring graph arcs and headed spans for global optimization and inference. Results of extensive experiments on five benchmark datasets reveal that our method performs significantly better than all previous bi-lexical methods, achieving new state-of-the-art.

## introduction structured sentiment analysis (ssa) aims to extract the complete opinion tuple from a sentence. as shown in figure 1 (a), the complete opinion tuple includes an opinion expression e with sentiment polarity p, an opinion holder h, and the corresponding target t. given the complexity of detecting three items and classifying one, ssa presents more challenges than other related tasks, such as opinion mining @xcite , absa (aspect-based sentiment analysis) @xcite @xcite , towe (target-oriented opinion words extraction) @xcite , aste (aspect sentiment triplet extraction) @xcite @xcite @xcite , etc. recent works of ssa mainly cast it as a problem of bi-lexical dependency graph parsing and propose multiple formulations: (1) @xcite proposed formulations namely head-first/head-final as illustrated in figure 1 @xcite . their method cannot resolve the problem because head-first/head-final treats the first/final word as the head of the span and strictly restricts any word inside the span directly head to span head, which decreases the height of the converted trees to 2 and excludes the latent structures completely. (2) another label strategy was proposed by @xcite , which simplifies the label set to only arcs linking spans boundaries, as shown in figure 1(c ). despite the special label for discontinuous span decoding, @xcite utilize the same label set. without distinct formulation about inside words, they attempt to utilize the powerful neural models like graph attention network @xcite or axial-based attention network @xcite to implicitly encoder the inside structure information, which is found lagging behind large with explicitly modeling with graph-based parsing methods @xcite @xcite . it is evident that previous work does not address the key challenge focused on the prediction of boundary words (first/final/both) of spans, and neglect the words and structures inside spans, which hinders the model expressiveness seriously. should we neglect the structure inside spans as previous works have done? tang conceded there had been "twists and turns" following us president george w. bush 's accession to the us presidency in early last year. from this example, it is evident that the target span is considerably long, making it challenging for the boundaries ("us" and "year") to effectively represent the entire span. conversely, the internal word "accession" provides significant clues for identifying it as the target of the expression "conceded". furthermore, the other words within the target span act as modifiers, aiding in the detection of the span's boundaries. based on these observations, the necessity of addressing the internal structure of spans in the ssa task is clear. it remains a significant challenge to develop an effective and unified method that can handle such structures. to address this issue, we propose a novel approach to structured sentiment analysis: treating flat spans as latent subtrees. this perspective considers expression-holder/target structures as partially-observed trees, where the exact subtrees for each span are yet to be determined. we employ treecrf @xcite to model these partially-observed trees. this involves enumerating all possible arcs and spans in a latent tree form of a flat span and calculating their probabilities using a constrained inside algorithm. the scores for all possible arcs and spans are determined using biaffine or triaffine attention methods @xcite , commonly applied in dependency parsing. during the decoding stage, we parse the highest-scoring dependency tree and reconstruct the ssa structure from it. span boundaries are accurately inferred from the descendant words of a head word. additionally, using the labels from the dependency tree, we can globally predict span-span relationships. this method not only explicitly accounts for internal span structures but also maintains the end-to-end nature characteristic of previous work. we conduct extensive experiments on five benchmarks, including norec fine @xcite , multib eu , multib ca @xcite , mpqa @xcite and ds unis @xcite . the results affirm that our model achieves new state-of-the-art in performance for ssa task. our contributions are summarized as follows: ## related work as a key topic in the sentiment analysis community @xcite , structured sentiment analysis (ssa) encompasses several sub-tasks, each targeting a specific component of the goal tuple (holder, target, expression, polarity). ssa also involves closely with structure predictions @xcite and relevant tasks @xcite . om opinion mining primarily aims to extract the (h, t, e) tuple. existing om studies generally adopt one of two approaches: 1) bio-based approach, which views om as a sequence labeling task @xcite ; and 2) span-based approach, which jointly predicts all span pairs and their interrelations @xcite . additionally, @xcite ; @xcite introduced a transition-based model for om. however, these methodologies neglect the sentiment polarity classification sub-task. absa aspect-based sentiment analysis is another important sentiment analysis task. various methods have been proposed to address absa, including: 1) pipeline @xcite , which sequentially predicts spans and their relationships; 2) end-to-end (chen and qian, 2020), utilizing interactive information from each pair of sub-tasks; and 3) mrc @xcite , which employ a machine reading comprehension framework to extract triplets. however, these methodologies neglect the sentiment holder extraction subtask. ssa @xcite propose the conversion named head-first/head-final and applied first-order parsing method. shi et al. (2022) proposed a new label strategy and apply graph attention network for aggregation on span boundaries for decoding, samuel et al. (2022) apply transformer to predict the graph directly from the text, zhai et al. (2023) add new labels to model the boundaries of discontinuous spans and apply axial-attention encoder and table filling scheme to decode the relations. however, all these works neglect the internal structure of the spans and rely on powerful encoders to implicitly incorporate internal span structure information. different from previous work, we are the first to cast ssa task as partially-observed dependency tree and apply dependency parsing method to explicitly model the internal structures of spans. owing to the structural similarities, leveraging nlp tasks such as parsing proves to be an effective strategy for structured prediction. our research builds upon the successes observed in partially-observed tree reduction and parsing methods across various nlp tasks including named entity recognition (ner) @xcite , nested ner @xcite @xcite and semantic role labeling @xcite . ## ssa as latent graph parsing scheme in the formulation of ssa as latent dependency graph, we treat each sentiment span as latent tree. to build the dependency graph, we deal with each expression span separately and assumes each of them corresponds to a single-root tree. in this tree, the sentiment head word of the expression span serves as the dependency root, with each subtree of the corresponding holder/target span attaching to it. consequently, our proposed latent dependency parsing task can be divided into two subtasks: (1) expression extraction; (2) corresponding holder/target extraction. note that both of these subtasks are solved by a consistent graph-based and headed-span-based parsing method, trained jointly and decoded step-by-step, which is named as two-stage parsing foot_1 . for the rest of this section, we show how our method : (1) converting ssa to latent dependency graph; (2) training to get span/tree structural representation; (3) decoding to find the best tree; (4) recovering resulting tree to sentiment tuple. ## methodology following previous work on dependency parsing @xcite @xcite , our model consists of a contextualized encoder and scoring modules. we further propose a constraint treecrf to compute the probabilities of the partially-observed trees of ssa. ## discussion ssa as a latent tree formulation effectively addresses not only latent tree modeling but also the management of overlap and discontinuous scenarios, as highlighted by @xcite . we offer a succinct proof as follows: (1) overlap case: our approach handles overlaps efficiently by treating each expression separately. this allows different expressions to share the same target/holder without conflict in our framework. (2) discontinuous case: this more complex scenario, predominantly found in expressions foot_3 , is also addressed in our scheme. we introduce a new label, "exp-incomplete", during the expression extraction subtask. this label is based on the observation that discontinuous cases often include modifiers like adverbs of degree, affirmation, probability, or negation, which can be identified independently. for instance, in "it is by no means a diploma mill", the discontinuous expression ("by no means" and "diploma mill") is handled by recognizing "by no means" as an incomplete negation, which, when attached to a complete expression like "diploma mill" (negative), alters the overall sentiment polarity to positive in our framework. foot_4 6.2 does the latent tree structure benefit for long spans/tuples? we address the question through results from two key experiments: (1) experiment on norec fine (figure 3 ) investigates the model's performance in longer spans/tuples. trees, we examine three variants of our first-order method: 1) first: utilizes the first word as the span headword. 2) last: employs the last word as the span headword. 3) flat: similar to the headfirst method by @xcite , attaches span words directly to the first word. results in ## conclusion in this study, we approach structured sentiment analysis (ssa) through latent dependency graph parsing, conceptualizing flat sentiment spans as latent subtrees. we introduce an innovative parsing methodology grounded in treecrf, designed to effectively integrate span structures. our experimental findings demonstrate that this method surpasses all previous approaches across five benchmark datasets. comprehensive analyses validate the efficacy and consistency of our method in enhancing ssa.