# Aspect-Based Sentiment Analysis Using Aspect Map
## 4 Learning AMEN and AMSAN
### Aspect-Map Extraction Network (AMEN)
Thus, \(\mathcal{D}_{a}\) is a compact representation of \(\mathcal{D}_{s}\) between a text \(\mathbf{t}\) and its aspects \(c\)s, where \(\mathbf{c}^{\mathbf{t}}\) is a binary vector that indicates all \(\mathbf{c}\)s for \(\mathbf{t}\) as \(1\)s but otherwise \(0\)s. Then, \(\theta_{\text{AMEN}}\), a set of all parameters of AMEN, is trained to minimize  
\[\sum_{(\mathbf{t},\mathbf{c}^{\mathbf{t}})\in\mathcal{D}_{a}}\mathcal{L}( \mathbf{c}^{\mathbf{t}},\text{AMEN}(\mathbf{t};\theta_{\text{AMEN}})), \tag{5}\]  
where \(\mathcal{L}\) is the cross-entropy loss. Note that \(\hat{z}^{h_{k}}\) in Equation (3) is the representative of a feature map \(\mathbf{z}^{h_{k}}\) and is directly connected to all aspects. Thus, the aspect map \(\mathcal{A}^{c}\) of a target aspect \(c\) becomes  
\[\mathcal{A}^{c}=\sigma(\mathbf{w}_{y}^{c}\cdot\mathbf{Z}), \tag{6}\]  
where \(\mathbf{w}_{y}^{c}\) is a row vector of \(\mathbf{W}_{y}\) for an aspect \(c\) and \(\sigma(\cdot)\) is the sigmoid function. For example, let us consider Figure 6, where the target aspect is _LOCATION_. AMEN of this figure has \(H=\{1,2,3\}\) and \(K=1\). Then, \(\mathcal{A}^{LOCATION}\) is  
\[\mathcal{A}^{LOCATION}=\sigma(w^{1_{1}}\cdot\mathbf{z}^{1_{1}}+w^{2_{1}}\cdot \mathbf{z}^{2_{1}}+w^{3_{1}}\cdot\mathbf{z}^{3_{1}}).\]  
Figure 4: The architecture of aspect-map extraction network (AMEN). The convolutional layer in this figure has three kernels of size \(h=1,2\) and \(3\). Assume that \(\mathbf{z}^{1_{1}}\) is the feature map that is most relevant to _LOCATION_ and \(\mathbf{z}^{2_{1}}\) is the most irrelevant feature map, as shown by the width of weights \(w^{1_{1}}\), \(w^{2_{1}}\) and \(w^{3_{1}}\). \(\mathcal{A}^{LOCATION}\) indicates \"_view of river_\" as an aspect expression for _LOCATION_ because \"_view_\" and \"_river_\" are activated by the strongly relevant feature map \(\mathbf{z}^{1_{1}}\) and \"_of_\" is activated by the weakly relevant feature map \(\mathbf{z}^{3_{1}}\). Although \"_sushi_\" and \"_rolls_\" are activated strongly by \(\mathbf{z}^{2_{1}}\), they are ignored due to the small value of \(w^{2_{1}}\). One problem of using the aspect map in aspect-level sentiment analysis is that the aspect map could be noisy. In practice, the noises in the aspect map are represented as strong activation of the words irrelevant to a target aspect. As a result, too many words are indicated as an aspect expression. This phenomenon is observed in all applications of CAM, since CAM is obtained by a weakly-supervised manner. The noise in the aspect map could be severer, since AMEN is trained with multi-labeled examples \((\mathbf{t},\mathbf{c}^{\mathbf{t}})\)s. To deal with this problem, the loss function for AMEN in Equation (5) is modified as  
\[\sum_{(\mathbf{t},\mathbf{c}^{\mathbf{t}})\in\mathcal{D}_{a}} \mathcal{L}(\mathbf{c}^{\mathbf{t}},\text{AMEN}(\mathbf{t};\theta_{\text{ AMEN}}))+\lambda\|\mathcal{A}^{\mathbf{t}}\|_{1}. \tag{7}\]  
Here, \(\mathcal{A}^{\mathbf{t}}\in\mathbb{R}^{\mathbb{C}\times N}\) is the matrix \(\mathbf{W}_{y}\cdot\mathbf{Z}^{\mathbf{t}}\) where \(\mathbf{Z}^{\mathbf{t}}\) is the feature matrix \(\mathbf{Z}\) for an input text \(\mathbf{t}\). The added \(L_{1}\) regularizer leads the aspect map \(\mathcal{A}^{c}\) to be sparse. Since AMEN is trained to jointly minimize the cross-entropy loss \(\mathcal{L}\) and the \(L_{1}\) regularizer \(\|\mathcal{A}^{\mathbf{t}}\|_{1}\) by Equation (7), its aspect map can indicate aspect expressions more precisely. That is, the words which actually contribute to classify \(\mathbf{t}\) into \(\mathbf{c}^{\mathbf{t}}\) are activated by the loss, and all other words are forced to be deactivated by the regularizer. Note that excessive regularization on \(\mathcal{A}^{\mathbf{t}}\) may kill the actual aspect expressions in the aspect map, which leads to a poor aspect map and the classification performance degradation. Thus, the impact of the regularizer should be controlled by setting \(\lambda\) to an appropriate value.