Question: Which model is used to capture the implicit structure?
Evidence: We propose a design for EI to efficiently learn rich implicit structures for exponentially many combinations of targets to predict. To do so, we explain the process to assign scores to each edge $e$ from our neural architecture. The three yellow boxes in Figure FIGREF14 compute scores for rich implicit structures from the neural architecture consisting of LSTM and self-attention.
Given an input token sequence $\mathbf {x}=\lbrace x_1,x_2,\cdots ,x_{n}\rbrace $ of length $n$, we first compute the concatenated embedding $\mathbf {e}_k=[\mathbf {w}_k;\mathbf {c}_k]$ based on word embedding $\mathbf {w}_k$ and character embedding $\mathbf {c}_k$ at position $k$.
As illustrated on the left part in Figure FIGREF14, we then use a Bi-directional LSTM to encode context features and obtain hidden states $\mathbf {h}_k=\mathrm {BiLSTM}(\mathbf {e_1},\mathbf {e_2}, \cdots , \mathbf {e_n})$. We use two different linear layers $f_t$ and $f_s$ to compute scores for target and sentiment respectively. The linear layer $f_t$ returns a vector of length 4, with each value in the vector indicating the score of the corresponding tag under the BMES tagging scheme. The linear layer $f_s$ returns a vector of length 3, with each value representing the score of a certain polarity of $+,0,-$. We assign such scores to each type of edge as follows:
As illustrated in Figure FIGREF14, we calculate $\mathbf {a}_k$, the output of self-attention at position $k$:
where $\alpha _{k,j}$ is the normalized weight score for $\mathbf {\beta }_{k,j}$, and $\mathbf {\beta }_{k,j}$ is the weight score calculated by target representation at position $k$ and contextual representation at position $j$. In addition, $W$ and $b$ as well as the attention matrix $U$ are the weights to be learned. Such a vector $\mathbf {a}_k$ encodes the implicit structures between the word $x_k$ and each word in the remaining sentence.
FLOAT SELECTED: Figure 4: Neural Architecture
The three yellow boxes in Figure FIGREF14 compute scores for rich implicit structures from the neural architecture consisting of LSTM and self-attention.
Given an input token sequence $\mathbf {x}=\lbrace x_1,x_2,\cdots ,x_{n}\rbrace $ of length $n$, we first compute the concatenated embedding $\mathbf {e}_k=[\mathbf {w}_k;\mathbf {c}_k]$ based on word embedding $\mathbf {w}_k$ and character embedding $\mathbf {c}_k$ at position $k$.
As illustrated on the left part in Figure FIGREF14, we then use a Bi-directional LSTM to encode context features and obtain hidden states $\mathbf {h}_k=\mathrm {BiLSTM}(\mathbf {e_1},\mathbf {e_2}, \cdots , \mathbf {e_n})$. We use two different linear layers $f_t$ and $f_s$ to compute scores for target and sentiment respectively. The linear layer $f_t$ returns a vector of length 4, with each value in the vector indicating the score of the corresponding tag under the BMES tagging scheme. The linear layer $f_s$ returns a vector of length 3, with each value representing the score of a certain polarity of $+,0,-$. We assign such scores to each type of edge as follows:
As illustrated in Figure FIGREF14, we calculate $\mathbf {a}_k$, the output of self-attention at position $k$:

where $\alpha _{k,j}$ is the normalized weight score for $\mathbf {\beta }_{k,j}$, and $\mathbf {\beta }_{k,j}$ is the weight score calculated by target representation at position $k$ and contextual representation at position $j$. In addition, $W$ and $b$ as well as the attention matrix $U$ are the weights to be learned. Such a vector $\mathbf {a}_k$ encodes the implicit structures between the word $x_k$ and each word in the remaining sentence.
FLOAT SELECTED: Figure 4: Neural Architecture
Answer: The answers are shown as follows:
* Bi-directional LSTM
* self-attention