# Visual Enhancement Capsule Network for Aspect-Based Multimodal Sentiment Analysis
## 3 Methodology
### Visual Enhancement Capsule Network Based on Multimodal Fusion
#### 3.2.1 MemCapsNet  
The Adaptive Mask Memory Capsule Network (MemCapsNet) aims to extract text features related to affective entities, as shown in the left of Figure 1, which contains word-embedding layer, encoding layer, N-gram source capsule layer, adaptive mask attention layer, and convolution capsule layer. The following will describe further details about this architecture. **Word-embedding layer. ** All the words in the vocabulary are vectorized through a sharing word embedding \(W_{emb}\in\mathbb{R}^{d\times|V|}\), where \(|V|\) and \(d\) denote the vocabulary size and the dimension of the word vector, respectively. The embedding matrix can be initialized with random initializations [51] or pretrained models [52]. Given a candidate document \(T^{i}=\{w_{1},w_{2},\ldots,w_{n_{1}}\}\) and its aspect phrase sequence \(A_{t}=\{w^{a}_{1},w^{a}_{2},\ldots,w^{a}_{n_{2}}\}\), we can use matrix \(W_{emb}\) to obtain word vector \(v_{t}(v^{a}_{t})\in\mathbb{R}^{d}\) for each word \(w_{t}(w^{a}_{t})\). In order to simplify representations, we uniformly denote the word vector matrix as \(E=\{v_{1},v_{2},\ldots,v_{n}\}\). **Encoding layer. ** The purpose of this layer is to integrate global semantic information into the embedding representation of each word of the given document \(T^{i}\) and its aspect \begin{table}
\begin{tabular}{l l} \hline \hline
**Notation** & **Description** \\ \hline \(w_{t}(w^{a}_{t})\) & The \(t\)th word (aspect word) \\ \(l_{t}\) & The \(t\)th image \\ \(T^{i}(I^{i})\) & Document (image set) included in the \(i\)th review, and \(w_{t}\in T^{i}\), \(l_{t}\in I^{i}\) \\ \(<T^{i},I^{i}>\) & The \(i\)th instance composed of \(T^{i}\) and \(I^{i}\) \\ \(a_{t}\) & The \(t\)th aspect \\ \(A(A_{t})\) & Aspect set (aspect phrase sequence of \(a_{t}\)) and \(a_{t}\in A\), \(w^{a}_{t}\in A_{t}\) \\ \(v_{t}(v^{a}_{t})\) & Word vector of \(w_{t}(w^{a}_{t})\) \\ \(L_{s}\) & A sentimental polarity label \\ \(L\) & A set of the labels composed of \(L_{s}\) \\ \(v_{t}(v^{a}_{t})\) & Word vector of \(w_{t}(w^{a}_{t})\) \\ \(\frac{w_{t}}{h_{t}^{a}}\) & Feature vector of \(w_{t}(w^{a}_{t})\) gained by forward LSTM \\ \(\frac{w_{t}}{h_{t}}\) & Feature vector of \(w_{t}(w^{a}_{t})\) gained by backward LSTM \\ \(h_{t}^{a}\) & Representation vector gained by Bi-LSTM \\ \(h_{t}^{c}(h_{t}^{a})\) & Representation vector of \(w_{t}(w^{a}_{t})\) gained by Bi-LSTM \\ \(H_{t}\) & Representation matrix obtained by Bi-LSTM \\ \(k_{i}\) & Size of the \(i\)th n-gram convolutional kernel \\ \(m_{j}\) & Feature mapping matrix of the \(j\)th image \\ \(p^{pri}(p^{pri}_{j})\) & Capsule pose matrix of a text (the \(j\)th image) from source capsule layer \\ \(a^{pri}(a^{pri}_{j})\) & Capsule active value matrix of a text (the \(j\)th image) from source capsule layer \\ \(p^{att}_{v}\) & Attention matrix from adaptive mask attention layer \\ \(\hat{p}_{j}\) & Aspect mapping capsule vector \\ \(p^{intx}(p^{img}_{j})\) & Capsule feature representation of a text (the \(j\)th image) from capsule \\ \(a^{intx}(a^{img}_{j})\) & Active value capsule matrix of a text (the \(j\)th image) from capsule convolution layer \\ \(p^{intx}\_{class}(p^{img,class})\) & Class capsule matrix of a text (its image set) produced by fully connected \\ \(a^{img,class}(a^{txt,class})\) & Class active capsule matrix of a text (its image set) produced by fully connected routing layer \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Description of the basic notations of model. phrase sequence \(A_{t}^{i}\) in aspect \(a_{t}\). The global semantic information of each word vector is captured with bidirectional LSTM (Bi-LSTM). Give an input word embedding matrix \(E=\{v_{1},v_{2},\ldots,v_{n}\}\), where \(v_{t}\) is the embedding vector of the \(t\)th word. The output can be computed using Equations (1)-(6).