# Classification of Retail Products: From Probabilistic Ranking to Neural Networks
## 10 Conclusions  
In this paper, we propose a solution for automatic classification of groceries in a digital transformation scenario, where the availability of quality data are key for retailers and costumers. In addition, the food market is a very dynamic context where new products emerge daily, so product catalogs are composed of a huge amount of data that need constant updates. Within this context, the Spanish company _Midiadia_ provides retailers with quality data about their product catalogs. _Midiadia_ processes the information from the products packaging and labeling to offer relevant knowledge to retailers that eventually support commercial processes (offers, customization, etc.). Our main objective is to collaborate with _Midiadia_ to improve the efficiency of one of their internal processes: classification of new products into the ontology the company uses. In particular, we focused on providing an automatic classification mechanism that replaces the manual task of deciding to which _Variety_ (one of the taxonomy levels that is composed of 159 values) a new product belongs. With this aim, we worked with three different alternatives in order to decide the most appropriate: score-based algorithms, machine learning approaches, and deep neural networks. After intensive work to preprocess the dataset (MDD-DS, composed by 20,888 products), we applied the following algorithms: first, we define a score-based algorithm based on the probabilistic model BM25, but adapting the formulation to our specific problem of groceries. Then, we applied the KNN, FKNN, and XGBoost algorithms, after reducing the data dimensionality with PCA. Finally, we also applied an MLP algorithm using the same PCA mechanism for reducing data dimensionality. The main objective is providing a totally automatic classification tool that directly offers one result: the most appropriate _Variety_ for a new product (_Top_1_ option). However, and considering the company benefits, we also studied other two alternatives: (i) _Top_2_  
Figure 8: Comparative study between all algorithms. Figure 7: Accuracy & F1-score of KNN, FKNN, XGBoost, MLP. that obtains the two most appropriate _Varieties_ for a new product and (ii) _Top_3_ that obtains the three most appropriate _Varieties_ for a new product. Both of them would also help the classification process by turning the problem of classifying a new product within 159 categories into a problem of classifying a new product within two or three categories. According to our results, the best in _Top_1_ is FKNN, closely followed by KNN. However, in _Top_2_ and _Top_3_, the score-based algorithm performs better, closely followed by FKNN. We are currently working on the creation of a dictionary of synonyms for ingredients. This need arises because the current regulations allow manufacturers to include different terms for the same element (vitamin C or ascorbic acid, for instance). Additionally, we are working on extending our analysis to other approaches, such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Deep Bidirectional LSTMs (BILSTM), Bidirectional Encoder Representations from Transformers (BERT), and Support Vector Machine (SVM). Regarding the course of dimensionality, applying DNN with abstract features is a natural next step provided that the issues related with memory consumption during training can be mitigated. After that, we expect to perform a comparison experiment with deep neural networks using additional hidden layers. In addition, the research work already done for the classification problem is the basis of a recommender algorithm to provide product alternatives to the customer when the desired product is not available. With this aim, we are currently working on a multidimensional recommender that takes into account the list of ingredients and other relevant data, such as calories, allergens, healthy aspects, etc. Conceptualization, M.M.H., A.F.V., R.P.D.R. and H.O.P. ; methodology, M.M.H., A.F.V., R.P.D.R. and H.O.P. ; software, M.M.H. and H.O.P. ; validation, M.M.H.