# An Improved Model for Medical Forum Question Classification Based on CNN and BiLSTM
## 4 Experiments and Analysis
### Result Analysis  
To verify the performance of the proposed model for QC, we first carried out some experiments on the ICHI Dataset. The ICHI dataset is divided into training and test sets. The training set contains 8000 questions while the test set has 3000 questions. Before the training process, the validation set is given 10% of the training set. Here, we compared our proposed model with two baseline models such as CNN and BiLSTM. The results on the test set are shown in Table 3.  
The results in Table 3 show that our proposed model achieved average values for Precision, Recall, and F1-score of 58.71%, 57.29%, and 57.14%, respectively. Comparing our results with the baseline models, our proposed model outperforms the CNN model based on the average values for Precision, Recall, and F1-score with a margin of 0.71%, 0.43%, and 0.14%, respectively; whereas, with the BiLSTM model, our proposed model increased the average values for Precision, Recall and F1-score with a margin of 1.28%, 0.58% and 0.28%, respectively. We can also see that the FAML, GOAL, and PREG categories provide better results compared to the TRMT, SOCL, DEMO, and DISE categories. This may be due to questions in these categories which are very long, which negatively affect the analysis and question-type predictions based on the deep learning methods. Another reason may be that the model failed to understand some technical words used in the medical field. To further test the performance of the proposed model, we conducted another experiment on the MedQuAD dataset. The MedQuAD dataset is divided into training and testing sets at the proportion of 70% and 30%, respectively. Before we start training our model, the validation set is given 10% of the training set. The results on the test set are summarized in Table 4. \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**CNN**} & \multicolumn{2}{c}{**BiLSTM**} & \multicolumn{2}{c}{**Proposed**} \\
**Categories** & **Precision** & **Recall** & **F1-Score** & **Precision** & **Recall** & **F1-Score** & **Precision** & **Recall** & **F1-Score** \\ \hline FAML & 81.00\% & 83.00\% & 82.00\% & 76.00\% & 77.00\% & 77.00\% & 82.00\% & 84.00\% & 83.00\% \\ GOAL & 72.00\% & 73.00\% & 72.00\% & 86.00\% & 70.00\% & 77.00\% & 84.00\% & 65.00\% & 73.00\% \\ SOCL & 47.00\% & 48.00\% & 47.00\% & 56.00\% & 53.00\% & 55.00\% & 51.00\% & 55.00\% & 53.00\% \\ DISE & 42.00\% & 46.00\% & 44.00\% & 48.00\% & 55.00\% & 51.00\% & 44.00\% & 64.00\% & 52.00\% \\ PREG & 69.00\% & 50.00\% & 58.00\% & 49.00\% & 49.00\% & 49.00\% & 58.00\% & 53.00\% & 55.00\% \\ TRMT & 49.00\% & 47.00\% & 48.00\% & 48.00\% & 44.00\% & 46.00\% & 53.00\% & 36.00\% & 43.00\% \\ DEMO & 46.00\% & 51.00\% & 48.00\% & 39.00\% & 49.00\% & 43.00\% & 39.00\% & 44.00\% & 41.00\% \\ Average & 58.00\% & 56.86\% & 57.00\% & 57.43\% & 56.71\% & 56.86\% & 58.71\% & 57.29\% & 57.14\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results of different models on the test set of the ICHI dataset. According to the results from Table 4, our proposed model achieved average values for Precision, Recall, and F1-score of 93.33%, 93.33%, and 93.33%, respectively. Comparing our results with the baseline models, our proposed model outperforms the CNN model based on average values for Precision, Recall, and F1-score with a margin of 3.33%, 3.33%, and 3.33%, respectively; whereas, with the BiLSTM model, our proposed model increased the average values for Precision, Recall and F1-score with a margin of 3.43%, 3.36%, and 3.36%, respectively. In addition, we can see that the results for all categories are very high, except for the \"DOSE\" and \"CAUSES\" categories. This is due to the fact that some questions contain terms that can be classified into multiple question types because they are more related to each other. This affects the classification performance in these categories. In addition, we can see that many categories achieved an accuracy of 100% and a few others obtained 0%.