# Machine Learning-Based Text Classification Comparison: Turkish Language Context
## 5 Findings
### Model Performance Evaluation
#### 5.2.4 Long Term-Short Memory Insights  
It can be argued that we achieved higher accuracy using the model trained using the LSTM technique than when using conventional ML algorithms. When using Keras to train the model, we provided parameters to the LSTM. To set the model's input layer, we included the embedding layer used with text data. Words are densely represented by the embedding layer. The number of inputs, input dimension, and output size are all parameters for the embedding algorithm. Since our dataset contains several classes, we employed the SoftMax function as the activation function in this study. The number of classes affected the output layer. We adjusted the loss function to categorical cross-entropy since our dataset had several classes.  
The size of the batch was provided as 64 while the size of the epoch was 10. Additionally, the 1D layer spatial dropout, which describes the likelihood of adjusting hidden layer outputs to 0, was added, and we assigned it a 0.2 rate. Instead of just deleting a single element, it deletes whole 1D feature maps. To avoid overfitting, we incorporated spatial dropout into the model [45]. The optimizer chosen was Adaptive Moment Estimation (Adam), which has the advantages of requiring little memory, performing well with big datasets, and being simple to use. The labeled data, the input data, the callback parameters, and the validation split were provided when utilizing the fit function. As the training progressed worsened, we abruptly ended it. We have 100 memory units in our LSTM layer. Finally, using the specified parameters, we examined the customer inquiries before and after data preparation.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Precision** & **Accuracy** & **F1-Score** & **Recall** \\ \hline RF & 33\% & 18\% & 7\% & 18\% \\ \hline NB & 72\% & 70\% & 66\% & 70\% \\ \hline LR & 76\% & 76\% & 75\% & 77\% \\ \hline SVM & 77\% & 78\% & 77\% & 78\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Results after data preparation.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Data** & **RF** & **LR** & **NB** & **SVM** & **LSTM** \\ \hline Raw & 36.727 & 5850.307 & 41.663 & 46,943.523 & 60,214.325 \\ \hline Prepared & 7.445 & 39.614 & 0.227 & 8865.088 & 10,426.658 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Training time comparison (seconds).  
The parameter for Keras was set to 10 epochs; however, after the 7 th epoch, the training was over. Early halting, indicated by the parameter, recognized that the model had not improved and halted training. As a result, rather than after the 10 th epoch, the training was finished after the 7 th epoch. Figure 7 shows that the accuracy value of the LSTM, which was applied to the original data, was 78% in the most recent epoch. There was no significant overfitting issue in the model training, as seen by the decreasing validation loss and loss values and the rising values of accuracy and validation accuracy with more epochs. The model trained using the prepared data had an accuracy rating of 84%. When compared to 1.55 before preparation, the data loss was around 0.67 after data preparation, as shown in Figure 8.  
Our results are consistent with [PERSON] et al. (2018), [PERSON] and [PERSON] (2018), and [PERSON] (2018)'s findings in that DL techniques, such as the LSTM, are more accurate compared to traditional ML techniques. Our results are also consistent with ([PERSON] et al., 2018; 2018), while opposed to the findings of ([PERSON] et al., 2018) in that the data preparation phase is critical in text classification.  
Figure 8: Data loss: (**a**) loss plot before data preparation; (**b**) loss plot after data preparation.  
Figure 7: Accuracy value of the LSTM: (**a**) accuracy plot before data preparation; (**b**) accuracy plot after data preparation.