# Aspect-Based Sentiment Analysis Using Aspect Map
## 5 Experiments
### Model Training  
Because the datasets are relatively small to train deep learning models, two well-known techniques are employed. One is transfer learning. It is known that pre-training a neural network with a large amount of labeled or unlabeled data from similar tasks is helpful in training it with a small dataset [31; 32; 33]. Thus, for each dataset, a CNN with the same architecture with AMEN is pre-trained. The 300-dimensional GloVe vectors [34] are used to initialize the word embedding layer of the CNN, and then the CNN is trained with the 4M Yelp restaurant reviews ([[https://www.yelp.com/dataset/challenge](https://www.yelp.com/dataset/challenge)]([https://www.yelp.com/dataset/challenge](https://www.yelp.com/dataset/challenge))) for the restaurant dataset and 3M Amazon Electronics reviews (We used only the reviews of the categories related to _Laptop_ and _Computer_ in Amazon Electronics.) [35] for the laptop dataset. Because aspect labels are not available in these reviews, only the weights of the word embedding layer of the pre-trained CNN are used to initialize those of AMEN. For AMSAN, the weights of the word embedding layer and the first convolutional layer are initialized with those of the pre-trained CNN. The other technique used is regularization. Several regularization techniques to prevent deep neural networks from overfitting to training data have been proposed [31]. Among them, the dropout [36] is applied to the output of the convolutional layers of both AMEN and AMSAN as well as the fully connected layer of AMSAN. In addition, \(L_{2}\) regularization is also applied to the output of the classification layers of both AMEN and AMSAN.  
The softmax activation and categorical cross-entropy loss are used for both AMEN and AMSAN, because they show empirically better performance than the sigmoid activation and the binary cross-entropy loss in this task. The filter size \(h\) is (1, 2, 3) for AMEN and (3, 4, 5) for AMSAN, and the number of kernels \(K\) is set to 100 for all \(h\)s. The dimension of the fully connected layer of AMSAN is 100. The AdaGrad [37] is used for both AMEN and AMSAN with a batch size of 16 instances and the default learning rate of \(1e-2\). The maximum number of epochs is 30 for AMEN and 20 for AMSAN. Because training both networks has randomness caused by random initialization and dropout, the experiments are conducted 30 times and then the mean accuracy with standard deviation is reported. All neural networks in this paper are implemented with Keras [38].  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline
**2014-Restaurant** & \multicolumn{3}{c}{**Train**} & \multicolumn{3}{c}{**Test**} \\ \hline AMEN & \multicolumn{3}{c}{3041} & \multicolumn{3}{c}{800} \\  & \multicolumn{3}{c}{(1.22)} & \multicolumn{3}{c}{(1.28)} \\ \hline AMSAN & Positive & Negative & Neutral & Conflict & Positive & Negative & Neutral & Conflict \\  & 2179 & 839 & 500 & 195 & 657 & 222 & 94 & 52 \\ \hline
**2016-Restaurant** & \multicolumn{3}{c}{**Train**} & \multicolumn{3}{c}{**Test**} \\ \hline AMEN & \multicolumn{3}{c}{335} & \multicolumn{3}{c}{90} \\  & \multicolumn{3}{c}{(4.28)} & \multicolumn{3}{c}{(4.49)} \\ \hline AMSAN & Positive & Negative & Neutral & Conflict & Positive & Negative & Neutral & Conflict \\  & 1012 & 327 & 55 & 41 & 286 & 84 & 23 & 11 \\ \hline
**2016-Laptop** & \multicolumn{3}{c}{**Train**} & \multicolumn{3}{c}{**Test**} \\ \hline AMEN & \multicolumn{3}{c}{395} & \multicolumn{3}{c}{80} \\  & \multicolumn{3}{c}{(5.27)} & \multicolumn{3}{c}{(6.68)} \\ \hline AMSAN & Positive & Negative & Neutral & Conflict & Positive & Negative & Neutral & Conflict \\  & 1210 & 708 & 123 & 41 & 330 & 161 & 31 & 13 \\ \hline \end{tabular}
\end{table}
Table 1: Simple statistics on the number of instances in the datasets. Numbers in parentheses are average number of aspects per review.