## 3 Industrial Case Study
### Machine Learning Model Development  
In this chapter, the implementation of the method, described in Chapter 3, upon the industrial pilot case, is presented. Figure 5: Visualization components of the dashboard: (**a**) Bin fill level over time, (**b**) Current bin-fill level, and (**c**) Scrap generation over time. * _Define bin fill levels_: For the purpose of the case study in this work, twelve (12) fill-levels were defined as follows: 0% (empty bin), [1-10%], [11-20%], [21-30%], [21-30%], [31-40%], [41-50%], [51-60%], [61-70%], [71-80%], [81-90%], [91-100%], and greater than 100% (\"Overfilled\"), and a class which signify the absence of a bin (\"No Bin\"). * _Image data acquisition and pre-processing_: The first step is using the camera to capture an image of the bin. During the time of the study, more than twenty thousand images have been captured (Table 2). A sample picture which is captured from the camera is shown in Figure 3. To build the training dataset, each image needs to be pre-processed. Image pre-processing starts by cropping the top and bottom parts and the next step is to rotate the image clockwise by 90 degrees. After the rotation, the picture is then cut down the middle into two parts creating two images, one for each bin (see Figure 6). The bins are always placed inside a red rectangle drawn on the floor. This means that the placement of the bins is fixed but can deviate a few centimetres in all directions. Since one camera was used to capture the images for both bins, due to the camera position not all details of Bin 2 can be captured. Due to the position and field of view of the camera, there is a blind spot on the left side of Bin 2 (see Figure 6b) which can make the fill level recognition more challenging for this particular bin. The placement of the camera was based on the needs of the manufacturer for giving priority to monitoring Bin 1. * _Image classification (labelling)_: After discussion with users and operators, the classification/labelling of the captured images took place. Images were classified according to the fill level of the bin shown. Labelling was performed by examining the captured images one by one and placing them in the appropriate class inside the dataset. This was done for both Bin 1 and Bin 2. In Figure 7 sample images for both Bin1 and Bin2 are provided. Figure 6: (**a**) Image of Bin1 after pre-processing, (**b**) Image of Bin2 after pre-processing; the blind spot is indicated. * _Training and evaluation of CNN model_: The VGG16 [21] image classifier CNN model, pre-trained on the ImageNet benchmark dataset, was selected for executing the image classification task. By using a pre-trained model with initial weight values, less images can be used for training, rather than starting from scratch. This reduces the amount of time needed for image acquisition and labelling as well as the required computational power to train the model. A customized model for each bin based on the initial model was developed using the Tensorflow Keras [22]. The following training setup was used:
* The annotated dataset of images (Table 2) was utilized for training. * For both Bin 1 and Bin 2, the split for training, validation, and testing of the dataset in Table 2 was 70%, 20%, and 10% respectively. * The images were normalized, and their input shape set to 224 \(\times\) 224 \(\times\) 3. * The batch size for the training was set to 32. * The number of training epochs used was 12 for the model for Bin 1 and 29 for the model for Bin 2. The number of epochs was not statically determined at the beginning of the training process. Instead, an early stopping function was used which monitored the validation loss. The function was implemented with a patience of 3, which restored the best weights. For Bin 1, the validation loss threshold after which training stopped was 0.19. For Bin 2, the validation loss threshold after which training stopped was 0.14. * The values of the network weights were initialized to ImageNet ones (pre-trained to ImageNet dataset).