# Attention-Based Light Weight Deep Learning Models for Early Potato Disease Detection
## 3 Experimental Results and Analysis
### MobileNet  
The attention-based MobileNet architecture consists of three dense layers: the first with 128 units and ReLU activation, and an \(L_{2}\) regularizer of 0.001; the second with 64 units, also using ReLU activation and the same regularizer; and the final layer with 3 units and softmax activation. Meanwhile, the traditional MobileNet architecture consists of four dense layers: the first three layers each have 512 units with ReLU activation and an \(L_{2}\) regularizer of 0.002, while the fourth layer has 3 units with softmax activation. The MobileNet model uses 1,056,771 trainable parameters to achieve an accuracy of 95.96%. In contrast, the attention-based MobileNet model with 405,477 trainable parameters achieves an accuracy of 96.09%. This attention-based model reduces the number of trainable parameters by approximately 62%.  
Table 8 shows the accuracy distribution for the MobileNet model implemented with and without attention.  
Figure 7 shows the decrease in the loss when implementing the MobileNet module with and without attention. Figure 7a shows the decrease in loss when the attention module combined with MobileNet is implemented and Figure 7b shows the decrease in loss when the MobileNet model is implemented without any form of attention. Clearly, we can observe that after 10 epochs, the decrease in loss is much lower in case of the attention-based technique; using attention, we were able to reduce loss below the level of 0.3, while the model without attention was still struggling around the value of 0.5. Table 9 shows the loss values after training for 10 epochs.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**With Attention**} & \multicolumn{2}{c}{**Without Attention**} \\ \hline
**Training** & **Validation** & **Training** & **Validation** \\ \hline Loss & 0.1329 & 0.1551 & 0.3273 & 0.3078 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Attention vs. without attention comparison of loss in DenseNet169 architecture.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **With Attention** & **Without Attention** \\ \hline Precision & 95.24 & 96.44 \\ \hline Recall & 95.93 & 96.55 \\ \hline \(F_{1}\) Score & 95.56 & 96.49 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Summary table for DenseNet169.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**With Attention**} & \multicolumn{2}{c}{**Without Attention**} \\ \hline  & **Training** & **Validation** & **Testing** & **Training** & **Validation** & **Testing** \\ \hline Accuracy & 96.09 & 95.25 & 97.25 & 95.96 & 94.14 & 94.67 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Attention vs. without attention comparison of accuracy in MobileNet architecture.  
The results can be further extended to other metrics also like precision, recall, and \(F_{1}\) score. The attention-based MobileNet model achieved a precision score of around 96.84% while the without-attention MobileNet model achieved 94.82%. The attention module combined with MobileNet was able to achieve a recall score of 97.15% while the without-attention model achieved 92.90%. The attention-based MobileNet model achieved an \(F_{1}\) score of 96.99%, while the without-attention model achieved 93.70%. The precision, recall, and \(F_{1}\) score are calculated by taking the average value of each individual class. Table 10 shows the distribution of the mentioned metrics.