## 5 Discussion and Conclusions  
In this study, we proposed a combination of BERT-based models and a neural topic model using an attention mechanism. Our results demonstrated that this approach outperformed the baseline model. In particular, the proposed model performed better when the amount of data was relatively small. In other words, when the analytical field is challenging to learn and the data size is limited, the proposed method can efficiently increase the performance through fine-tuning. However, for the 20 Newsgroups dataset, performance improvement was not achieved. This suggests that the neural topic model may not significantly affect the cases in which the baseline model performs well. This indicates a need to explore the conditions under which the proposed model provides the greatest benefits. In this study, we focused solely on classification tasks. Future research should include additional experiments on other tasks, such as regression, to further explore the potential \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & \multicolumn{5}{c}{**MRD**} \\ \hline
**Evaluation** & **AUC** & **Precision** & **Recall** & **F1** & **Time** (s) \\ \hline AVM & 0.6604 & 0.6348 & 0.6233 & 0.5940 & 244 \\ \hline ANTM & **0.6862** & **0.6593** & **0.6506** & **0.6213** & 262 \\ \hline ANEM & 0.6288 & 0.6093 & 0.5977 & 0.5684 & 296 \\ \hline EVM & 0.7303 & 0.7005 & 0.6922 & 0.6677 & 249 \\ \hline ENTM & **0.7390** & **0.7125** & **0.7021** & **0.6795** & 266 \\ \hline ENEM & 0.7279 & 0.6927 & 0.6902 & 0.6697 & 302 \\ \hline
**Dataset** & \multicolumn{5}{c}{**20 Newsgroups**} \\ \hline
**Evaluation** & **AUC** & **Precision** & **Recall** & **F1** & **Time** (s)** \\ \hline AVM & 0.8694 & 0.8026 & 0.8016 & 0.7972 & 853 \\ \hline ANTM & **0.8754** & **0.8121** & **0.8114** & **0.8070** & 929 \\ \hline ANEM & 0.8420 & 0.7678 & 0.7686 & 0.7631 & 1043 \\ \hline EVM & **0.8948** & **0.8381** & **0.8372** & **0.8333** & 862 \\ \hline ENTM & 0.8883 & 0.8302 & 0.8295 & 0.8255 & 933 \\ \hline ENEM & 0.8888 & 0.8293 & 0.8287 & 0.8246 & 1049 \\ \hline
**Dataset** & \multicolumn{5}{c}{**YELP**} \\ \hline
**Evaluation** & **AUC** & **Precision** & **Recall** & **F1** & **Time** (s)** \\ \hline AVM & 0.6830 & 0.5656 & 0.5743 & 0.5478 & 1666 \\ \hline ANTM & **0.6871** & **0.5751** & **0.5837** & **0.5567** & 1792 \\ \hline ANEM & 0.6760 & 0.5499 & 0.5626 & 0.5339 & 2039 \\ \hline EVM & 0.7115 & 0.5956 & 0.6047 & 0.5787 & 1694 \\ \hline ENTM & **0.7145** & **0.6050** & **0.6124** & **0.5871** & 1809 \\ \hline ENEM & 0.7114 & 0.5970 & 0.6063 & 0.5802 & 2061 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Ensemble model performance. for performance improvement. Furthermore, conducting comparative studies that include large language models (LLMs) like GPT-4 and LLaMA, as well as other types of models such as multimodal models, would be valuable. Additionally, we used the bag-of-words method to embed the input documents into the neural topic model, which has the disadvantage of ignoring the word order. Therefore, employing embedding methods that consider the order of sentences rather than the bag-of-words method could lead to more effective performance improvements. The experiments conducted in this study revealed that the performance of the proposed model was influenced by the characteristics of the datasets and hyperparameters. Therefore, future research should focus on developing automated methods for determining the optimal number of topics to further enhance model performance. Additionally, understanding the dataset-specific characteristics that influence the effectiveness of neural topic model integration could provide deeper insights into the behavior of the model, potentially leading to more tailored and effective model adaptations. Conceptualization, T.U. and N.K. ; methodology, T.U. and N.K. ; software, T.U. ; validation, T.U. and N.K. ; formal analysis, T.U. ; investigation, T.U. and N.K. ; resources, N.K. ; data curation, T.U. ; writing--original draft preparation, T.U. and N.K.