## 3 Industrial Case Study
### Machine Learning Model Development
* Pooling was set to \"avg\", meaning that Global Average Pooling was applied to the output of the last convolutional block. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Bin**} & \multicolumn{8}{c}{**Fill Levels (\%)**} \\ \cline{2-13}  & **0** & **1–10** & **11–20** & **21–30** & **31–40** & **41–50** & **51–60** & **61–70** & **71–80** & **81–90** & **91–100** & **No Bin** & **Overfilled** \\ \hline
1 & 236 & 466 & 245 & 441 & 846 & 1101 & 600 & 1514 & 838 & 1803 & 885 & 12 & 784 \\
2 & 2087 & 2087 & 932 & 632 & 729 & 1058 & 1009 & 963 & 719 & 440 & 422 & 13 & 142 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Labelled data per bin and fill-level. Figure 7: Samples of different fill levels for Bin 1 and Bin 2. * All network layers were un-frozen and free to be retrained on the new data. * The top, dense, layer available by default in Tensorflow Keras was replaced with a custom one that works for classifying 13 classes. The new top layer consisted of the following consecutive layers and the model's architecture can be seen in Figure 8: 1. A Flatten Layer 2. A Dense Layer with 4096 units and its activation function set to \"relu\" 3. A Dropout layer with a rate of 0.4 A Dense Layer with 1024 units and its activation function set to \"relu\" 4. A last Dense Layer with 13 units and its activation function set to \"softmax\"
* To optimize the model parameters, the Adam [23] optimizer was used in all CNN models. The learning rate was set at 0.00001. After training was completed, the customized VGG16 model achieved the following performance results:  
* For Bin 1, the model achieved 96% training accuracy, 93% validation accuracy, and 89% accuracy on the testing set. The model's loss was 0.1. * For Bin 2, the model achieved 96% accuracy on the training set, 94% validation accuracy, and 95% accuracy on the testing dataset. The model's loss was 0.14. In Figure 9, the graphs for the accuracy and loss performance of VGG16 model in Bin 1 are shown. Figure 8: Architecture of customized VGG16 model. Figure 9: Performance of customized VGG16 model for Bin 1: (**a**) model accuracy and (**b**) model loss. In Figures 10 and 11, the graphs for the accuracy and loss performance of the VGG16 model in Bin 2 are presented. In order to assess the performance of the customized VGG16 model, three additional state of the art, off-the-shelf image classifiers were tested upon the developed dataset. These models include the ResNet50 model [24], the InceptionV3 [25], and the MobileNetV2 model [26]. In Table 3, the results of the training, evaluation, and testing of each model are presented. Figure 11: Loss of customized VGG16 model for Bin 2. Figure 10: Accuracy of customized VGG16 model for Bin 2. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **VGG16** & **ResNet50** & **InceptionV3** & **MobileNetV2** \\ \hline \multirow{4}{*}{Bin 1} & Train. Accuracy: 96\% & Train. Accuracy: 99\% & Train. Accuracy: 98\% & Train. Accuracy: 75\% \\  & Val. Accuracy: & Val. Accuracy: 97\% & Val. Accuracy: 95\% & Val. Accuracy: 71\% \\  & Loss: 0.1 & Loss: 0.2 & Loss: 0.22 & Loss: 0.63 \\  & Test. Accuracy: & 82\% & Test. Accuracy: 81\% & Test. Accuracy: 72\% \\  & 89\% & Overfitting & Overfitting & Worst Performing Model \\ \hline \multirow{4}{*}{Bin 2} & Train. Accuracy: 96\% & Train. Accuracy: 95\% & Train. Accuracy: 88\% & Train. Accuracy: 69\% \\  & Val. Accuracy: 94\% & Val. Accuracy: 93\% & Val. Accuracy: 91\% & Val. Accuracy: 73\% \\ \cline{1-1}  & Loss: 0.14 & Loss: 0.2 & Loss: 0.28 & Test. Accuracy: 71\% \\ \cline{1-1}  & Test. Accuracy: & 92\% & Test. Accuracy: 88\% & Test. Accuracy: 71\% \\ \cline{1-1}  & Best Performing Model & Overfitting & Underfitting & Worst Performing Model—Underfitting \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison between customized VGG16, ResNet50, InceptionV3, and MobileNetV2 CNN models when tested for Bin 1 and Bin 2.