## 7 Discussion  
Several studies have addressed the topic of automated bug classification, indicating that the manual classification of bugs is cumbersome and time-consuming. Moreover, since the size and complexity of software keep increasing, manual bug classification is not scalable and tractable anymore for human experts. As a result, the chance of making more errors in the bug classification process can increase, and herewith also the process for handling and resolving the bugs. Although many studies in the literature investigated the automated bug classification problem applying machine learning techniques; our approach distinguishes from these studies in focus on the combination of various aspects: including the usage of commercial software, use of bilingual bug reports (Turkish and English), use of unstructured bug reports, and use of black-box bug reports. Each of these aspects by themselves implies an additional complexity in the bug classification process. Together, these aspects even further impede the manual bug classification process and the automated bug classification process. The problem is thus inherently complicated. Therefore, the first research question focused on searching the answer for whether the adoption of an automated bug classification approach and, in particular, the approach that we have presented would be effective. Despite the inherent difficulties, we can indeed state that the presented approach is effective. The presented approach adopts and integrates machine learning, text mining, and natural language processing techniques to support the classification of software bugs. \begin{table}
\begin{tabular}{l c} \hline \hline
**Classifier** & **F1-Measure (\%)** \\ \hline FastText with Naive Bayes & 65.35 \\ FastText with SVM-linear kernel & 67.33 \\ FastText with SVM-poly kernel & 70.30 \\ FastText with SVM-RBF kernel & 69.31 \\ FastText with SVM-sigmoid kernel & 65.35 \\ FastText with KNN & 58.42 \\ FastText with logistic regression & 63.37 \\ FastText with decision tree & 50.00 \\ FastText with random forest & **71.19** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Effect of bilingual preprocessing using stop-word elimination (SW) and lemmatizing (LEM) for the case study. \begin{table}
\begin{tabular}{l c c} \hline \hline  & **Accuracy (\%)** & **Weighted F1 (\%)** \\ \hline ESE1 & 60.32 & 58.48 \\ ESE2 & 56.75 & 55.36 \\ Automated classification & **73.70** & **71.66** \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Accuracy and weighted F1 measures for manual and automatic classification. All of these techniques were necessary and had to be adequately integrated. Hence, we have also discussed a well-defined and preplanned process in this paper. We have adopted proven methods in text mining to analyze the bug reports lexically by tokenization, derive the semantic meaning by lemmatization, and extract features using TF-IDF and word embeddings. Then, we mapped these to vectors (vectorization) which machine learning algorithms can process. With this, we took into account and also processed words in the two different languages. We have not limited ourselves to a few machine learning algorithms but adopted Naive Bayes (NB), support vector machine (SVM), k-nearest neighbor (KNN), logistic regression (LR), decision tree (DT), and random forest (RF) classifiers in our experiments. We have also used all four kernels of the SVM classifier: linear, poly, RBF, and sigmoid kernels. It should be stated that the process required some iterations to fine-tune the hyperparameters of the ML algorithms. In addition to the BOW approach, we have also adopted the more recent approach in feature extraction using word embeddings. In general, word embeddings might be more efficient than the BOW models in classification tasks. However, in our case study, we obtained a 71.19 % F1 measure using FastText with random forest. This value is slightly less than we obtained with the BOW approach and SVM with linear kernel.