P o D : Positional Dependency-Based Word Embedding for Aspect Term Extraction (2020)

Abstract: Dependency context-based word embedding jointly learns the representations of word and dependency context, and has been proved effective in aspect term extraction. In this paper, we design the positional dependency-based word embedding (PoD) which considers both dependency context and positional context for aspect term extraction. Specifically, the positional context is modeled via relative position encoding. Besides, we enhance the dependency context by integrating more lexical information (e.g., POS tags) along dependency paths. Experiments on SemEval 2014/2015/2016 datasets show that our approach outperforms other embedding methods in aspect term extraction.

## introduction aspect term extraction aims to extract expressions that represent properties of products or services from online reviews @xcite @xcite . understanding the context between words in reviews, such as through conditional random fields @xcite @xcite , is the key to superior results in aspect term extraction. word embeddings are effective to capture the contextual information across a wide range of nlp tasks @xcite @xcite . however, they only produce moderate results in aspect term extraction. recent studies (e.g., @xcite ) indicate that this is due to the distributed nature of the word embedding @xcite , which ignores the rich context between the words, such as syntactic information. in this paper, we propose positional dependency-based word embedding (pod) to enhance the context modeling capability for aspect term extraction. pod explicitly captures two types of contexts, dependency context and positional context. inspired by the simple-yet-effective position encoding in transformer @xcite , pod models the positional context via relative position encoding @xcite between words within a fixed window. besides, the dependency context is defined as the dependency path as well as the attached lexical information (e.g., pos tags and words) along the path. moreover, pod is able to incorporate more lexical information into the semantic compositional model via the dependency context, making representations of dependency paths more informative than the ones that only consider grammatical information @xcite . we then linearly combine the dependency and positional context to produce the positional dependencies among words. we also define a margin-based ranking loss to efficiently optimize pod. our contributions are two-fold, (i) we propose positional dependency-based word embedding pod, which incorporates both positional context and dependency context, (ii) we compare pod with existing aspect term extraction methods and demonstrate that pod yields improved results on aspect term extraction datasets. ## related work association rule mining is used in @xcite to mine aspect terms. opinion words are used to extract infrequent aspect terms. the relationship between opinion words and aspect words is crucial to extract aspect terms, which are deployed in many follow-up studies. in @xcite , the predefined dependency paths are utilized to iteratively extract aspect terms and opinion words. pod instead learns the representation of the dependency context. dependency-based word embedding @xcite ) encodes dependencies into word embeddings, and has been shown effective in aspect term extraction as well @xcite . however, only grammatical information is considered among the dependency paths. we instead introduce a positional dependency-based embedding method which considers both dependency context and positional context. end-to-end aspect term extraction @xcite @xcite based on neural networks and attention mechanism, have been recently developed. compare to these methods, pod is an embedding method, can thus be applied to more applications. compare to deep word representations @xcite , pod is more efficient, which is crucial to aspect term extraction. ## conclusion in this paper, we develop a specific word embedding method for aspect term extraction. our method considers both positional and dependency context when learning the word embedding. meanwhile, the lexical information along dependency path is encoded into representations of dependency context. compared with other embedding methods, our method achieves better results in aspect term extraction.