# Developing Data-Conscious Deep Learning Models for Product Classification
## 3 Proposed Methods
### Models for Baseline  
In this study, we used NB, CNN, and Bi-LSTM as models for product classification.  
**Naive Bayes (NB).** We applied NB as a baseline model for comparison with the deep learning models proposed in this paper. We chose it as the baseline non-deep learning model, similar to the approach described in [13; 15]. In the experiments, NB was applied after training TF-IDF (term frequency-inverse document frequency) [22].  
**Convolutional Neural Network (CNN).** The model was designed primarily to effectively process images. A convolutional layer that extracts feature values and a pooling layer that reduces the layer size are configured repeatedly. All the nodes with the resulting data processed until the present are connected and represented in a one-dimensional array. This is called a fully-connected layer. It is added at the end of model construction. Figure 5a shows the architecture of the CNN model employed in this paper.  
\begin{table}
\begin{tabular}{c c} \hline \hline  & **An Example of Data Transformation (for the Product Description Attribute)** \\ \hline Before transformation & hand-made candle wt na-tu–ral soybean pre-mium \\  & fragrant oils \\ \hline After transformation & handmade candle soy wax premium fragrance oil \\ \hline \hline \end{tabular}
\end{table}
Table 2: Example of data according to the transformation phase.  
Figure 4: Words and their frequency distributions. Each word is sorted in descending order based on its appearance frequency in all the attributes (**a**) or in individual attributes (**b**).  
After the filtering method is applied to an original image, the classification operation is performed for the filtered image. In this study, the channel size was set to one for it to be suitable for text, among various CNN models. The width size was 300, which is equal to that for embedding. The embedding layer was applied, and the dropout activation function was executed using a value of 0.3. The inputs based on the attribute values set up were combined into a single layer. After transforming a three-dimensional tensor into a four-dimensional tensor, the dropout function was executed. After the development of convolution layers with the kernel size list of the input attributes, the tensor was reduced to two dimensions. After inserting max pooling layers, the tensor was manipulated into four dimensions, and these were combined into a single layer. The linear layers were processed repeatedly. Finally, the Softmax function was applied for normalization and output.  
**Bidirectional Long-Short Term Memory (Bi-LSTM).** LSTM is a type of recurrent neural network (RNN) [23]. It was designed to solve the long-term dependency problem, which occurs when the time gap for obtaining necessary information is large. It has an architecture wherein a cell-state is added to the hidden state in an RNN. The limitation of LSTM is that it is executed based on the immediately preceding pattern. Bi-LSTM was designed to overcome this limitation. In Bi-LSTM, forward training and backward training are performed through an RNN. One of its advantages is that the performance does not decrease even when the length of data is large. Figure 5b shows the architecture of the Bi-LSTM model employed in this paper.  
In this study, the proposed Bi-LSTM is set to facilitate this model in both directions. The hidden size is set to 64, and the embedding is performed. The inputs based on the attribute values that are set are combined into a single layer. The combined layer is entered into the LSTM layer and processed. Two pooling layers are passed through, and each layer outputs the mean and maximum. Then, these are combined back into one layer. This linear layer is processed repeatedly. Subsequently, the dropout function is executed with a value of 0.3 [5]. Finally, the linear layer is passed through one more time, which completes the process.