SlowGuess/ACL_2020_OCR huggingface dataset:
Shaowei Chen, Jie Liu*, Yu Wang, Wenzheng Zhang, Ziming Chi
College of Artificial Intelligence, Nankai University, Tianjin, China
{chenshaowei, yuwang17, wzhang}@mail.nankai.edu.cn
jliu@nankai.edu.cn
Opinion entity extraction is a fundamental task in fine-grained opinion mining. Related studies generally extract aspects and/or opinion expressions without recognizing the relations between them. However, the relations are crucial for downstream tasks, including sentiment classification, opinion summarization, etc. In this paper, we explore Aspect-Opinion Pair Extraction (AOPE) task, which aims at extracting aspects and opinion expressions in pairs. To deal with this task, we propose Synchronous Double-channel Recurrent Network (SDRN) mainly consisting of an opinion entity extraction unit, a relation detection unit, and a synchronization unit. The opinion entity extraction unit and the relation detection unit are developed as two channels to extract opinion entities and relations simultaneously. Furthermore, within the synchronization unit, we design Entity Synchronization Mechanism (ESM) and Relation Synchronization Mechanism (RSM) to enhance the mutual benefit on the above two channels. To verify the performance of SDRN, we manually build three datasets based on SemEval 2014 and 2015 benchmarks. Extensive experiments demonstrate that SDRN achieves state-of-the-art performances.
Opinion entity extraction, which aims at identifying aspects and/or opinion expressions in review sentences, is an important task in fine-grained opinion mining. Recently, there have been considerable studies focused on this task. Specifically, Liu et al. (2012), Li and Lam (2017) and Li et al. (2018) explored aspect term extraction, and Fan et al. (2019) extracted opinion phrases with given aspects. Meanwhile, many studies dealt with aspect and opinion
Figure 1: An example of task comparisons. The aspects and the opinion expressions are marked with red and blue, respectively.
term co-extraction (Xu et al., 2013; Liu et al., 2015; Wang et al., 2017; Yu et al., 2019; Wang and Pan, 2019; Dai and Song, 2019). These studies have shown the importance of opinion entity extraction and achieved great progress. However, they neglect to recognize the relations between aspects and opinion expressions.
While aspect-opinion relation detection is one of the key parts of an opinion mining system (Hu and Liu, 2004; Popescu and Etzioni, 2005; Zhuang et al., 2006), it is neglected or assumed given beforehand, which leaves a significant gap to subsequent opinion mining tasks. For instance, as shown in Figure 1, we can obtain the aspect  $\{food\}$  and the opinion expressions  $\{\text{nice-looking, delicious}\}$  from opinion entity extraction. Although both nice-looking and delicious express positive sentiment, they further describe food from the appearance and taste perspectives, respectively. Therefore, only with the relations between aspects and opinion expressions, e.g., the pair  $\langle food, delicious \rangle$ , can the more fine-grained subsequent tasks be executed, such as pair-level sentiment classification, pair-level opinion clustering, etc.
To bridge the gap between opinion entity extraction and subsequent tasks, we explore Aspect-Oinion Pair Extraction (AOPE) task, which aims at extracting aspects and opinion expressions along
6515
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6515-6524 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
with their relations. Specially, AOPE is not only necessary for subsequent tasks, but also beneficial to both opinion entity extraction and relation detection. However, the studies on AOPE are very limited. Early works (Hu and Liu, 2004; Zhuang et al., 2006) approach aspect-opinion pair extraction in a pipeline manner by dividing it into two isolated tasks. Yang and Cardie (2013), Klinger and Cimiano (2013b) and Katiyar and Cardie (2016) attempted to extract opinion entities and relations jointly without considering the interaction between opinion entity extraction and relation detection, which limits the performance.
Therefore, AOPE remains a rather challenging task. First, the relational structure of aspects and opinion expressions within a sentence can be complicated, requiring the model to be effective and flexible in detecting relations. For example, the relations can be one-to-many, many-to-one, and even embedded or overlapped. Second, opinion entity extraction and relation detection are not two independent tasks as in other multitask learning problems but rely on each other, hence posing a key challenge on how to fuse and learn the two subtasks properly. Third, how to synchronize opinion entity extraction with relation detection and make them mutually promotion is another primary challenge.
To address the aforementioned challenges, we propose Synchronous Double-channel Recurrent Network (SDRN). Specifically, we first utilize Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al., 2019) to learn context representations. Then, the double-channel recurrent network, which consists of an opinion entity extraction unit and a relation detection unit, is constructed to extract aspects, opinion expressions, and relations simultaneously. To enable the information interaction between the above two channels, we design a synchronization unit which contains Entity Synchronization Mechanism (ESM) and Relation Synchronization Mechanism (RSM). Extensive experiments verify that our model achieves state-of-the-art performances. In summary, our contributions are three-fold:
- We explore AOPE task, which is valuable and critical for downstream tasks but remains under-investigated.
- We propose an end-to-end neural model, SDRN<sup>1</sup>. By adopting BERT as the encoding
layer, SDRN can learn richer context semantics. By designing the double-channel network and two synchronization mechanisms, SDRN could process opinion entity extraction and relation detection jointly and make them mutually beneficial.
- We manually build three datasets based on SemEval 2014 and 2015 benchmarks for AOPE task. Extensive experiments are conducted to verify that our model achieves state-of-the-art performances.
Aspect-opinion pair extraction is a critical task in fine-grained opinion mining. Early studies approach this task in a pipeline manner. Hu and Liu (2004) used association mining to identify aspects and extract the adjacent adjectives as opinions. Zhuang et al. (2006) extracted aspects and opinion expressions first, and then mined the relations with dependency relation templates. Popescu and Etzioni (2005) proposed an unsupervised model to extract aspects and corresponding opinions from reviews with pre-defined rules. Although the above methods achieved great progress, they generally suffered from error propagation.
To avoid error propagation, recent studies propose joint learning methods. Klinger and Cimiano (2013a) adopted Imperatively Defined Factor graph (IDF) to analyze the inter-dependencies between aspects and opinion expressions. Klinger and Cimiano (2013b) presented a joint inference model based on IDF to extract aspect terms, opinion terms, and their relations. Yang and Cardie (2013) employed Integer Linear Programming (ILP) to identify opinion-related entities and their associated relations jointly. However, these works generally based on shallow machine learning methods and depended on hand-crafted features.
To automatically capture features, neural network methods have been applied to various fine-grained opinion mining tasks. Xu et al. (2018) used Convolutional Neural Network (CNN) to extract aspects. Wang et al. (2016), Wang et al. (2017), Yu et al. (2019) and Wang and Pan (2019) used deep learning methods to deal with aspect and opinion term co-extraction. Li et al. (2018) focused on aspect term extraction and adopted attention mechanism to exploit the latent relations between aspect and opinion terms. Hu et al. (2019) took BERT to extract aspects and corresponding sentiments.
$^{1}$ https://github.com/NKU-IIPLab/SDRN
Figure 2: The framework of Synchronous Double-channel Recurrent Network (SDRN).
For AOPE, Katiyar and Cardie (2016) explored LSTM-based models to jointly extract opinion entities and their relations with three optimization methods. But this method neglects to learn the interaction between opinion entity extraction and relation detection.
Therefore, AOPE is still under-investigated and needs more researches. In this paper, we further explore this task and propose a neural model SDRN.
Given a review sentence  $S$ , Aspect-Oinion Pair Extraction (AOPE) task aims to obtain a collection of aspect-opinion pairs  $C = [\langle a_m, o_m \rangle]_{m=1}^M$  from  $S$ , where  $a_m$  and  $o_m$  represent the aspect and the opinion expression, respectively<sup>2</sup>.
To deal with AOPE task, we propose Synchronous Double-channel Recurrent Network (SDRN). The overall framework of SDRN is illustrated in Figure 2. Specifically, we first adopt BERT as the encoding layer to learn the context representations. Then, an opinion entity extraction unit and a relation detection unit are constructed as double channels to extract aspects, opinion expressions, and relations simultaneously. Furthermore, a synchronization unit is designed to enable information interaction between the double channels. To capture high-level representations, we recurrently execute the above units. After multiple recurrent steps, we adopt an inference layer to obtain aspect-opinion pairs.
3.1 Encoding Layer
Given a review sentence  $S$ , we first tokenize it using the WordPiece vocabulary (Wu et al., 2016) and add tokens [CLS] and [SEP] to the beginning and the end of the tokenized sentence, respectively. As a result, we obtain the input sequence  $X = \{x_{1}, x_{2}, \dots, x_{N}\}$  with  $N$  tokens for each sentence.
Inspired by the success of BERT (Devlin et al., 2019), we adopt it as the encoder to learn the contextual semantics. For each token  $x_{i}$ , the initial embedding  $\mathbf{e}_i$  is constructed by summing the corresponding token embedding  $\mathbf{e}_i^w$ , segment embedding  $\mathbf{e}_i^s$ , and position embedding  $\mathbf{e}_i^p$ . Then, the embedding sequence  $E = \{\mathbf{e}_1, \mathbf{e}_2, \dots, \mathbf{e}_N\}$  is fed into BERT, which consists of stacked Transformer blocks with multiple self-attention heads (Vaswani et al., 2017). We take the output of the last Transformer block as the context representation sequence  $H^{s} = \{\mathbf{h}_{1}^{s}, \mathbf{h}_{2}^{s}, \dots, \mathbf{h}_{N}^{s}\}$ .
3.2 Double-channel Recurrent Network
3.2.1 Opinion Entity Extraction Unit
The opinion entity extraction unit, which aims at extracting the aspects and the opinion expressions, is developed as a channel of SDRN. To deal with this sequence labeling task, we couple Conditional Random Field (CRF) (Lafferty et al., 2001) upon the encoding layer, which serves as the opinion entity extraction unit. Formally, CRF adopts a state score matrix  $P \in \mathbb{R}^{N \times K}$  to model the mappings between tokens and labels, and a transition score matrix  $Q \in \mathbb{R}^{K \times K}$  to model the relations between adjacent labels, where  $K$  denotes the dimension of the label space<sup>3</sup>. For a sequence of predicted labels  $Y^{t} = \{y_{1}^{t}, y_{2}^{t}, \dots, y_{N}^{t}\}$  at the  $t$ -th recurrent step, we define its score as follows:
$$
S (X, Y ^ {t}) = \sum_ {i = 1} ^ {N} Q _ {y _ {i - 1} ^ {t}, y _ {i} ^ {t}} + \sum_ {i = 1} ^ {N} P _ {i, y _ {i} ^ {t}} ^ {t}, \tag {1}
$$
$$
P ^ {t} = H _ {t} ^ {o} W _ {p} + b _ {p}, \tag {2}
$$
where  $H_{t}^{o} = \left\{\mathbf{h}_{t,1}^{o},\mathbf{h}_{t,2}^{o},\dots,\mathbf{h}_{t,N}^{o}\right\}$  denotes the input hidden representation sequence at the  $t$  -th recurrent step for the opinion entity extraction unit, which is calculated with the context representation sequence  $H^{s}$  and the relation synchronization semantics  $R_{t - 1}$ . The details will be described in
3Following the BIO tagging scheme, we define five labels, including BA (beginning of aspect), IA (inside of aspect), BP (beginning of opinion expression), IP (inside of opinion expression), and O (others).
2Note that  $a_{m}$  or  $o_m$  could be a single word or a phrase.
6517
Section 3.3.2. The matrices  $W_{p} \in \mathbb{R}^{d_{o} \times K}$  and  $b_{p} \in \mathbb{R}^{N \times K}$  are model parameters, where  $d_{o}$  denotes the dimension of hidden representation  $\mathbf{h}_{t,i}^{o}$ .
Then, the probability of the predicted sequence  $Y^{t}$  can be calculated as follows:
$$
p \left(Y ^ {t} \mid X\right) = \frac {\exp \left(S \left(X , Y ^ {t}\right)\right)}{\sum_ {\widetilde {Y} ^ {t} \in Y _ {X} ^ {t}} \exp \left(S \left(X , \widetilde {Y} ^ {t}\right)\right)}, \tag {3}
$$
where  $Y_{X}^{t}$  denotes all possible label sequences. During training, we maximize the likelihood probability  $p(Y\mid X)$  of gold label sequence at the last step. During decoding, we use the Viterbi algorithm to find the label sequence with the maximum score.
3.2.2 Relation Detection Unit
To extract opinion entities and relations simultaneously, we design a relation detection unit as another channel of SDRN. Considering the complicated relations between aspects and opinion expressions, we devise a supervised self-attention mechanism as the relation detection unit to flexibly model token-level relations without the sequential limitation.
At the  $t$ -th recurrent step, we first compute the attention matrix  $G^{t} \in \mathbb{R}^{N \times N}$  whose element  $g_{i,j}^{t}$  represents the degree of correlation between the  $i$ -th token and the  $j$ -th token as follows:
$$
g _ {i, j} ^ {t} = \frac {\exp (\gamma (\mathbf {h} _ {t , i} ^ {r} , \mathbf {h} _ {t , j} ^ {r}))}{\sum_ {k = 1} ^ {N} \exp (\gamma (\mathbf {h} _ {t , i} ^ {r} , \mathbf {h} _ {t , k} ^ {r}))}, \tag {4}
$$
$$
\gamma \left(\mathbf {h} _ {t, i} ^ {r}, \mathbf {h} _ {t, j} ^ {r}\right) = \tanh  \left(\mathbf {h} _ {t, i} ^ {r} W _ {r} ^ {1} + \mathbf {h} _ {t, j} ^ {r} W _ {r} ^ {2}\right) W _ {r} ^ {3}, \tag {5}
$$
where  $\gamma$  is a score function, and  $\mathbf{h}_{t,i}^{r}$  denotes the input hidden representation of the  $i$ -th token for the relation detection unit. Note that the hidden representation sequence  $H_{t}^{r} = \left\{\mathbf{h}_{t,1}^{r}, \mathbf{h}_{t,2}^{r}, \dots, \mathbf{h}_{t,N}^{r}\right\}$  is calculated with the context representation sequence  $H^{s}$  and the entity synchronization semantics  $U_{t-1}$ . The details will be described in Section 3.3.1. The matrices  $W_{r}^{1} \in \mathbb{R}^{d_{r} \times d_{r}}$ ,  $W_{r}^{2} \in \mathbb{R}^{d_{r} \times d_{r}}$ , and  $W_{r}^{3} \in \mathbb{R}^{d_{r} \times 1}$  are model parameters, where  $d_{r}$  is the dimension of hidden representation  $\mathbf{h}_{t,i}^{r}$ .
At the last step  $T$ , we further introduce supervision information into the calculation of the attention matrix  $G^T$  by maximizing the likelihood probability as follows:
$$
p \left(Z | X\right) = \prod_ {i = 1} ^ {N} \prod_ {j = 1} ^ {N} p \left(z _ {i, j} \mid x _ {i}, x _ {j}\right), \tag {6}
$$
where the standard relation matrix  $Z \in \mathbb{R}^{N \times N}$  consists of element  $z_{i,j}$ , and the relation probability  $p(z_{i,j}|x_i, x_j)$  can be calculated as follows:
$$
p \left(z _ {i, j} \mid x _ {i}, x _ {j}\right) = \left\{ \begin{array}{l l} g _ {i, j} ^ {T}, & i f \quad z _ {i, j} = 1 \\ 1 - g _ {i, j} ^ {T}, & i f \quad z _ {i, j} = 0 \end{array} , \right. \tag {7}
$$
where  $z_{i,j} = 1$  denotes the fact that there is a relation between the  $i$ -th token and the  $j$ -th token, and vice versa. With this supervision information, the attention can be guided to capture the correlations between the tokens more effectively.
3.3 Synchronization Unit
Since the above two channels are interdependent, it is important to synchronize their information and make them mutually beneficial. To this end, we design Entity Synchronization Mechanism (ESM) and Relation Synchronization Mechanism (RSM) to update the hidden representation sequences  $H_{t}^{o}$  and  $H_{t}^{r}$  by exchanging the high-level information.
3.3.1 Entity Synchronization Mechanism
Considering that opinion entities are generally phrases, both opinion entity semantics and token-level interactions are crucial in detecting relations. For instance, given an aspect 'hot dog' and an opinion expression 'tasty', there is no relation between 'hot' and 'tasty' when only token-level interaction is considered, but it is easy to detect the relation if we utilize the semantics of aspect 'hot dog'.
Accordingly, we design ESM to capture the corresponding entity semantics for each token and integrate these semantics into the hidden representation sequence  $H_{t + 1}^{r}$ . Specifically, based on the predicted label sequence  $Y^{t}$  and its probability obtained from the opinion entity extraction unit, each entity semantics  $\mathbf{u}_{t,i}$  of the  $i$ -th token at the  $t$ -th recurrent step can be calculated as follows:
$$
\mathbf {u} _ {t, i} = \sum_ {j = 1} ^ {N} \varphi \left(B _ {i, j} ^ {t}\right) \mathbf {h} _ {j} ^ {s}, \tag {8}
$$
$$
\varphi \left(B _ {i, j} ^ {t}\right) = \frac {B _ {i , j} ^ {t}}{\sum_ {k = 1} ^ {N} B _ {i , k} ^ {t}}, \tag {9}
$$
where  $B_{i,j}^{t}$  is the label probability of the  $j$ -th token if the  $i$ -th token and the  $j$ -th token belong to the same entity; otherwise,  $B_{i,j}^{t}$  is zero. And  $\varphi(\cdot)$  is a normalization function.
To integrate both the context representation  $\mathbf{h}_i^s$  and the entity semantics  $\mathbf{u}_{t,i}$ , we calculate the hidden representation  $\mathbf{h}_{t + 1,i}^{r}$  as follows:
$$
\mathbf {h} _ {t + 1, i} ^ {r} = \sigma \left(\mathbf {u} _ {t, i} W _ {r} ^ {4} + \mathbf {h} _ {i} ^ {s} W _ {r} ^ {5}\right), \tag {10}
$$
where  $W_r^4 \in \mathbb{R}^{d_s \times d_r}$  and  $W_r^5 \in \mathbb{R}^{d_s \times d_r}$  are model parameters,  $d_s$  is the dimension of context representation, and  $\sigma$  is the activation function which can be tanh or sigmoid function. Note that we use zero matrix to initialize the entity semantics sequence  $U_0 = \{\mathbf{u}_{0,1}, \mathbf{u}_{0,2}, \dots, \mathbf{u}_{0,N}\}$ .
3.3.2 Relation Synchronization Mechanism
Since the relations between opinion entities can provide clues for opinion entity extraction, it's important to encode the relation semantics. For example, if 'overrated' is used to modify 'pizza', this relation could provide guidance to extract the aspect 'pizza' and the opinion expression 'overrated'.
Thus, we design RSM to capture the semantics which reflect the relations and update the hidden representation sequence  $H_{t+1}^{o}$ . Concretely, at the  $t$ -th recurrent step, we can calculate the relation semantics  $\mathbf{r}_{t,i}$  of the  $i$ -th token with the correlated degree  $g_{i,j}^{t}$  from the relation detection unit:
$$
\mathbf {r} _ {t, i} = \sum_ {j = 1} ^ {N} \varphi \left(\phi \left(g _ {i, j} ^ {t}\right)\right) \mathbf {h} _ {j} ^ {s}, \tag {11}
$$
$$
\phi \left(g _ {i, j} ^ {t}\right) = \left\{ \begin{array}{l l l} g _ {i, j} ^ {t}, & i f & g _ {i, j} ^ {t} \geqslant \beta \\ 0, & i f & g _ {i, j} ^ {t} <   \beta \end{array} , \right. \tag {12}
$$
where  $\varphi (\cdot)$  is the same normalization function as Eq.(9). To avoid noise, we utilize  $\phi (\cdot)$  to filter correlated scores below the given threshold  $\beta$
Then, we combine the relation semantics  $\mathbf{r}_{t,i}$  and context representation  $\mathbf{h}_i^s$  to obtain the hidden representation  $\mathbf{h}_{t + 1,i}^o$ :
$$
\mathbf {h} _ {t + 1, i} ^ {o} = \sigma \left(\mathbf {r} _ {t, i} W _ {o} ^ {1} + \mathbf {h} _ {i} ^ {s} W _ {o} ^ {2}\right), \tag {13}
$$
where  $W_{o}^{1} \in \mathbb{R}^{d_{s} \times d_{o}}$  and  $W_{o}^{2} \in \mathbb{R}^{d_{s} \times d_{o}}$  are model parameters. Similar to ESM, the initial relation semantics sequence  $R_{0} = \{\mathbf{r}_{0,1}, \mathbf{r}_{0,2}, \dots, \mathbf{r}_{0,N}\}$  is set to zero.
Particularly, the integration methods used in ESM and RSM can also make the proposed SDRN easy to optimize, which is similar to the shortcut connections (He et al., 2016).
3.4 Joint Learning
To synchronously learn the proposed two channels, we fuse the loss functions from the two channels. For opinion entity extraction unit, given the gold label sequence  $Y$ , we minimize the negative log-likelihood loss function at the last step as follows:
$$
\mathcal {L} _ {E} = \log \sum_ {\widetilde {Y} \in Y _ {X} ^ {T}} \exp (S (X, \widetilde {Y})) - S (X, Y). \tag {14}
$$
For the relation detection unit, we convert the gold annotation to a one-hot matrix, where 0 denotes no relations, and 1 represents the existence of relations between two tokens. Then, we minimize the cross-entropy loss between the predicted distribution  $\hat{p} (z_{i,j}|x_i,x_j)$  at the last step and the gold distribution  $p(z_{i,j}|x_i,x_j)$  as follows:
$$
\mathcal {L} _ {R} = - \sum_ {i = 1} ^ {N} \sum_ {j = 1} ^ {N} p \left(z _ {i, j} \mid x _ {i}, x _ {j}\right) \log [ \hat {p} \left(z _ {i, j} \mid x _ {i}, x _ {j}\right) ]. \tag {15}
$$
Then, the two parts are combined to construct the loss objective of the entire model:
$$
\mathcal {L} (\theta) = \mathcal {L} _ {E} + \mathcal {L} _ {R}. \tag {16}
$$
The optimization problems in Eq. (16) can be solved by using any gradient descent method. In this paper, we adopt the BERTAdam method.
3.5 Inference Layer
Because SDRN synchronously processes opinion entity extraction and relation detection, an inference layer is introduced to generate aspect-opinion pairs based on the results of the two channels.
With the label sequence  $Y^{T}$  predicted by the opinion entity extraction unit at the last recurrent step, we can obtain the aspect set  $A = \{a_{1}, a_{2}, \dots, a_{l_{A}}\}$  with  $l_{A}$  aspects and the opinion set  $O = \{o_{1}, o_{2}, \dots, o_{l_{O}}\}$  with  $l_{O}$  opinion expressions. Then, the relations between aspects and opinion expressions can be calculated according to the weight matrix  $G^{T}$  from the relation detection unit. For instance, given an aspect  $a = \{x_{i_{S}^{a}}, \dots, x_{i_{E}^{a}}\}$  and an opinion expression  $o = \{x_{i_{S}^{o}}, \dots, x_{i_{E}^{o}}\}$ , the correlated degree  $\delta$  between them can be calculated as follows:
$$
\delta = \frac {1}{2} \left(\frac {1}{| a |} \sum_ {k = i _ {S} ^ {a}} ^ {i _ {E} ^ {a}} \sum_ {l = i _ {S} ^ {o}} ^ {i _ {E} ^ {o}} g _ {k, l} + \frac {1}{| o |} \sum_ {l = i _ {S} ^ {o}} ^ {i _ {E} ^ {o}} \sum_ {k = i _ {S} ^ {a}} ^ {i _ {E} ^ {a}} g _ {l, k}\right), \tag {17}
$$
where  $|a|$  and  $|o|$  denote the length of aspect and opinion expression. The pair  $\langle a, o \rangle$  is extracted only if  $\delta$  is higher than a given threshold  $\hat{\delta}$ .
6519
Dataset	#Sent	#A	#O	#R
SemEval-14 Restaurant	Train	3041	3693	3512	2809
Test	800	1134	1014	936
SemEval-14 Laptop	Train	3045	2359	2500	1535
Test	800	653	677	380
SemEval-15 Restaurant	Train	1315	1205	1217	1231
Test	685	542	516	516
JDPA	Camera	3125	6107	4557	4144
Car	6501	8272	11123	8709
MPQA	9471	4676	5849	4823
Table 1: Statistics of datasets. #Sent, #A, #O, and #R represent the number of sentences, aspects, opinion expressions, and relations, respectively.
To evaluate the effectiveness of SDRN, we conduct extensive experiments on five benchmark datasets from SemEval 2014 (Pontiki et al., 2014), SemEval 2015 (Pontiki et al., 2015), MPQA version 2.0 corpus (Wiebe et al., 2005), and J.D. Power and Associates Sentiment Corpora (JDPA) (Kessler et al., 2010). The statistics of these benchmark datasets are shown in Table 1. For SemEval 2014 and 2015 datasets, we manually build relations between aspects and opinion expressions because the original datasets only contain the gold standard annotation for aspects. Note that we follow the annotations for opinion expressions provided by Wang et al. (2016) and Wang et al. (2017).