# Developing Data-Conscious Deep Learning Models for Product Classification
## 4 Experiment  
### Performance Results  
Based on the data-conscious method proposed in this paper, we compared and evaluated the performance of each model trained by plugging in the CNN and Bi-LSTM models presented in the previous section. The NB model was considered as the baseline. We measured the accuracy as the performance metric. All the experiments were conducted using a GPU (Tesla P100-PCIE-16 GB RAM) provided by Google Colab Pro (Google Colaboratory, [[https://colab.research.google.com/](https://colab.research.google.com/)]([https://colab.research.google.com/](https://colab.research.google.com/))).  
We performed the experiment to determine the performance of each model for the selection, transformation, and filtering processes. Table 3 shows the accuracy for each combination method and the training time required for each method. For example, the second row of Table 3 shows that the accuracy of the model trained using the CNN deep learning model up to the selection phase is 48.67% in the worst case, 74.25% in the best  
Figure 5: Architectures of our CNN model and Bi-LSTM model. Each model is plugged in as a deep learning model in Figure 2.  
case, and 67.74% on an average. If this CNN deep learning model is used in the selection and transformation phases, the accuracy is 58.92% in the worst case, 78.69% in the best case, and 73.14% on an average. These results demonstrate that if the CNN deep learning model is used indiscriminately without considering the data as a product classification model, its accuracy can differ by approximately 67.74% from that of the CNN model that effectively considers the data in both selection and transformation phases. The average accuracy increases by 7.4% after the selection-transformation phases compared with that after only the selection phase. Similar results are obtained when the Bi-LSTM deep learning model is used: the maximum and average differences are 29% and 7.3%, respectively.  
The final average accuracy is 52.28% in the case wherein the NB model is used. However, it is 73.45% and 72.32% in the cases where the proposed CNN and Bi-LSTM models, respectively, are used. Furthermore, when the baseline and deep learning models are compared, the minimum and maximum differences in accuracy are 10.79% and 31.09%, respectively. These results verify that the method of training the deep learning models is effective for increasing the classification accuracy. Furthermore, it was observed that the two models are highly suitable for the methodology and product data used in this study. A comparison of the accuracy of the deep learning models reveals that the CNN model shows an accuracy that is marginally higher (an average difference of 1.12%) than that of the Bi-LSTM model. A performance comparison between the selection-transformation phases and selection-transformation-filtering phases for each model reveals that the Bi-LSTM shows an increase, although it is marginal. For example, the average accuracy increases from 71.91% to 72.32. However, in the case of the CNN model, the increase is smaller than that for the Bi-LSTM and negligible. These results indicate that attention should be paid to the selection of stopwords for removal in the filtering phase. The next section presents a case study of this.  
The deep learning models were also examined in terms of time because the training time was required. The training time for each case is shown in the cell below that with the relevant accuracy performance in Table 3. For example, the training time required for the CNN model that showed the best accuracy after the filtering phase was approximately 6 min shorter than that of the Bi-LSTM for the same case. Although we could not present all the cases in the table, the time required for the training after completing the filtering phase was shorter in the CNN model than in the Bi-LSTM model. The difference in the training time is likely to be larger if the data size increases from that in this experiment.