# The Chef's Choice: System for Allergen and Style Classification in Recipes
## 4 Data Acquisition
### Dataset Preprocessing  
The Kaggle dataset has already been preprocessed beforehand by the creators of the dataset. However, in order to streamline the process, both datasets, Kaggle and Open Food Facts, will both be processed in the same way. The Kaggle dataset has no outlier or undesired words but can benefit from Lemmatization. The Open Food Facts on the other hand has an abundance of content provided by the community and thus needs preprocessing. Since allergen and cuisine classification is based on ingredients of a recipe, other unwanted characters have to be filtered. A recipe that is available on a website will be shown as an example of why preprocessing is needed.  
Example of a recipe for \"Whole Wheat and Honey Pizza Dough\" ([[https://www.allrecipes.com/recipe/24372/whole-wheat-and-honey-pizza-dough/print](https://www.allrecipes.com/recipe/24372/whole-wheat-and-honey-pizza-dough/print)]([https://www.allrecipes.com/recipe/24372/whole-wheat-and-honey-pizza-dough/print](https://www.allrecipes.com/recipe/24372/whole-wheat-and-honey-pizza-dough/print)), accessed on 27 February 2022): _1 (.25 ounce) package active dry yeast, 1 cup warm water, 2 cups whole wheat flour, 1/4 cup wheat germ, 1 teapoon salt, 1 tablespoon honey_.  
Preprocessing for the dataset includes several steps:  
* Removal of non-alphanumeric characters
* Stopword removal
* POS tagging
* Lemmatization of ingredients
* Filtering
* One-Hot-Encoding
* Up- and down-sampling
* Feature transformation  
Due to a large amount of data and the nature of the preprocessing steps, processing the Open Food Facts dataset with all steps can take up to an hour or even more. Since training a classifier is also quite time-consuming, a few steps are done only once beforehand, while other steps are completed as part of training the classifier.  
Up- and Down-Sampling  
Up-and down-sampling in the multi-label case is more complicated. Adding and removing a sample from one class also inevitably removes an entry from another class as one sample can belong to more than one class.  
After the under-sampling has been applied, the majority classes are reduced, which can be seen in Figure 4. Under-sampling reduces data, which means there is information loss. This can potentially be a problem for machine learning and the end result. Given the big difference between the majority and minority classes, under-sampling has been applied to reduce the skewedness of the data.  
For oversampling, a label powerset transformation with random oversampling is applied. The original code from [PERSON] (2018) is used and changed to operate with the Open Food Facts dataset. First, an extra column is added to the dataframe for the label powerset transformation of the original labels. Then for each power label, the distance or gap between the highest powerset label and the current one is calculated. Using the distance, random samples are selected for each power label and added to the dataframe. The overall dataset is not perfectly balanced even after up- and down-sampling. Figure 5 shows that after random oversampling, the minority classes have more samples and the overall distribution looks better than before. It also shows that the class with the lowest amount of samples is \"lupin\". This raises concerns for model training later on, where the number of samples for this class might influence the accuracy of the trained classifiers.