## 3 Materials and Methods for Manual Expansion
### Evaluation Experiment 4: Improvement of the Auto-Expanded Affect Lexicon  
In this additional evaluation experiment, we manually checked the expressions added by automatic expansion to the ML-Ask affect lexicon database, checked the meaning of each expression, whether it was an emotive expression and in what context it was used, by consulting a general Japanese dictionary, and manually removed expressions other than emotive expressions.  
Table 21 shows the partial and exact match rates for the re-evaluation after manually removing non-emotive expressions, and Table 22 shows the polarity match rate, activity match rate, and incorrect extraction rate.  
Results and Discussion of Experiment 4  
The results of the current evaluation Experiment 4 show a similar decrease in the exact match rate but only by about 1.7% on average, while the other indices are improved.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Partial Match Rate** & **Exact Match Rate** \\ \hline ML-Ask & 66\% & 41\% \\ baseline & & \\ \hline Word Segmentation & 67\% & 40\% \\ +H*idf & (+1\%) * & (\(-\)1\%) \\ \hline Word Segmentation & 67\% & 40\% \\ +BM25 & (+1\%) & (\(-\)1\%) \\ \hline Prototype Processing & 65\% & 39\% \\ +H*idf & (\(-\)2\%) & (\(-\)2\%) \\ \hline Prototype Processing & 67\% & 39\% \\ +BM25 & (+1\%) & (\(-\)2\%) \\ \hline Chunk Processing & 67\% & 38\% \\ +H*idf & (+1\%) & (\(-\)3\%) \\ \hline Chunk Processing & 67\% & 40\% \\ +BM25 & (+1\%) & (\(-\)1\%) \\ \hline \hline \end{tabular}  
* The improvement is shown in red color font and the deterioration is shown in blue color font.  
\end{table}
Table 21: Results of Experiment 4-1.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Polarity Dimension Match Rate** & **Activity Dimension Match Rate** & **Incorrect Extraction Rate** \\ \hline ML-Ask & 65\% & 58\% & 2\% \\ baseline & & & \\ \hline Word Segmentation & 72\% & 65\% & 2\% \\ +H*idf & (+7\%) * & (+7\%) & (+0\%) \\ \hline Word Segmentation & 72\% & 66\% & 2\% \\ +BM25 & (+7\%) & (+8\%) & (+0\%) \\ \hline Prototype Processing & 72\% & 65\% & 2\% \\ +H*idf & (+7\%) & (+7\%) & (+0\%) \\ \hline Prototype Processing & 72\% & 66\% & 2\% \\ +BM25 & (+7\%) & (+7\%) & (+0\%) \\ \hline Chunk Processing & 72\% & 66\% & 2\% \\ +H*idf & (+7\%) & (+8\%) & (+0\%) \\ \hline Chunk Processing & 72\% & 66\% & 2\% \\ +BM25 & (+7\%) & (+8\%) & (+0\%) \\ \hline \hline \end{tabular}  
* The improvement is shown in red color font.  
\end{table}
Table 22: Results of Experiment 4-2.  
The incorrect extraction rate is also unchanged from that of the ML-Ask baseline model, suggesting that the improvement was successful.  
Additionally, the segmentation method that led to the greatest improvement in accuracy is chunking, and the weighting coefficient is OkapiBM25.  
Compared to the baseline system, only the exact match rate decreased slightly, but all other measures improved by an average of 3.3 percentage points.  
We investigated the reason for the decrease in the percentage of exact match rate and found that all of the analyses that did not result in exact matches included emotive expressions (e.g., 'H'' \"Jyou\") consisting of a single character with ambiguous meanings.  
This suggests that emotive expressions consisting of a single character should be removed from the dictionary.  
In addition, to improve the system in this experiment, the emotive expressions had to be removed manually. To solve this problem in the future, it will be necessary to develop a method to automatically remove expressions other than emotive expressions.  
The improvement in results was small, but this was due to the fact that the number of words that could be added to the affect lexicon was very small because only the top 50 candidate emotive expressions were used in this experiment.  
To solve this problem, it is necessary to expand the range of candidate words for emotive expressions (e.g., from the top 50 to 100), increase the data from which the candidate words are extracted, and recheck the results on a larger dataset.