# Last-Mile Optimization Using Neural Networks
## 4 Results and Discussion  
In this section, we provide the outcomes of our experiments with various MLP neural network models utilizing the same dataset. We conduct a comparative analysis of these MLP models to identify the optimal one by assessing their performance on the same dataset. Our evaluation of experimental results encompasses six metrics: accuracy, error rate, precision, recall, F1 score, and total training time. Noteworthy distinctions among the models include variations in the number of epochs and batch size. To enhance accuracy or diminish the error rate, we considered increasing the number of epochs, indicating additional training periods for improvement. However, careful attention must be given to another variable, namely, batch size. Neglecting this variable may result in increased training time; decreased accuracy; and, consequently, elevated error rates. Drawing insights from existing literature, we opted for categorical cross-entropy loss, RMSprop optimizer, and ReLu activation. The experimental determination of the number of hidden layers and neurons was executed empirically, with a fixed choice of two hidden layers due to the dataset's limited attributes. Detailed experimental results are outlined in Table 3. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Metrics** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **Error Rate** & **Total Training Time (Second)** \\ \hline MLP1 & 97.56\% & 97.63\% & 97.56\% & 97.17\% & 13.83\% & 10.07 \\ MLP2 & 96.34\% & 96.52\% & 94.34\% & 95.29\% & 13.78\% & 2.73 \\ MLP3 & 95.12\% & 90.61\% & 95.12\% & 92.78\% & 16.41\% & 2.99 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance comparisons among different MLP models. Figure 3: The Multilayer perceptron. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Item** & **MLP1** & **MLP2** & **MLP3** \\ \hline Layer & 2 & 2 & 2 \\ Neuron & 128, 64 & 32, 32 & 64, 64 \\ Activation function & ReLu, ReLu & ReLu, ReLu & ReLu, ReLu \\ Loss function & Categorical Cross Entropy & Categorical Cross Entropy & Categorical Cross Entropy \\ Optimizer & RMSprop & RMSprop & RMSprop \\ Batch size & 64 & 32 & 64 \\ Epochs & 42 & 62 & 52 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparative analysis of MLP models. Our analysis indicates that all four MLP neural network models demonstrate comparable performance in the experimental results. Notably, Model 1 achieves the highest accuracy, precision, recall, F1 score, and the lowest error rate. Model 1 exhibits an outstanding accuracy of 97.56%, and its F1 score, representing a balance of precision and recall, is also notably high at 97.17%. Precision, highlighting the impact of false positives against true positives, and recall, reflecting false negatives, both excel in Model 1, with precision being relatively higher than recall. The model's training time on a system with 8 GB of RAM and a clock speed of 1.80 GHz is 10.07 s. While MLP 2 and MLP 3 exhibit shorter build times, their classification accuracy falls short of that achieved by MLP 1. Consequently, Model 1 stands out as the superior MLP neural network in this evaluation. Now, we present the performance of the model via the evolution of the classification accuracy and the loss function of this model during training. These results are plotted throughout the training and testing phases of the neural network, which are shown in Figure 4 below. Figure 4 above shows that the curves follow each other and reach an accuracy of 97.56% in the test phase and at the end of the training phase. Thus, the function of the loss function decreases relatively during the epochs until it almost reaches the value 13.83%. Figure 5 below presents the confusion matrix for the four classes on the test database. It presents a breakdown of instances that were classified correctly and those that were not, categorizing them into true positives, true negatives, false positives, and false negatives. The aim is to minimize the occurrences of false positives and false negatives.