# CNN-Based Pill Image Recognition for Retrieval Systems
## 5 Discussion  
The experimental results in Section 4 suggest that our proposed _CNN+kNN_ model architecture outperformed the closely-related _ResNet-50_ model and the _CNN+SVM_ model. Although all three classification techniques (k-NN, SVM, and ResNet-50) performed well, the proposed _CNN+kNN_ model was able to achieve the highest accuracy.  
Table 2 compares the mAP values of _ResNet-50, CNN+SVM_, and _CNN+kNN_ using the NLM dataset. With regard to the mAP values, applying the _CNN+kNN_ successfully increased the prediction Precision from 80% to 91%. This finding implies that, if the _CNN+kNN_ model predicts a target class (i.e., finds a pill type of a query pill image), then there is a 91% chance that the prediction will be correct.  
In Figure 9, we compared between the three models to evaluate the \"goodness\" of the prediction model. Each line in the figure represents the model's Precision-Recall curve. A larger area under the curve implies better Precision and Recall. That is, the more the curve pushes to the top-right corner of the plot, the better the model is. Out of all the three models, _CNN+kNN_ had the largest area, thus implying a higher prediction quality.  
Figure 10 provides us with an idea of how well (or bad) the model understands the different features of a pill in order to make a decision about the pill type. Figure 10a suggests that the model was successfully able to differentiate between all distinct shapes. However, Figure 10b,c suggest that the model struggled in making a decision when the pill color was _Turquoise_ and imprint was _SOMA_ and _M6_, respectively. The low Precision values for these target classes (e.g., 57.1% for predicting _Turquoise_) may be due to the low number of training images with pills having these target classes.  
Table 4 suggests that using a Convolutional Neural Network with the k-Nearest Neighbor classifier improves pill identification accuracy by about 10%, which is considered a significant improvement. For sensitive applications or diseases, we suggest considering Top-1 Accuracy. If Top-5 Accuracy is to be considered, albeit at 96% accuracy, we suggest consulting an expert for confirmation.  
The runtime experiment, as reported in Table 5, suggests that using the k-NN classifier does not compromise the efficiency of the overall image retrieval system. This finding implies that a pill image retrieval system's accuracy can be improved without compromising its runtime.  
Although our experiments suggest higher pill image retrieval success than comparable methods, we encountered some challenges pertaining to consumer image quality. Conflicting light conditions, placement of the pills, and the distance from the camera used in the consumer images negatively impacted the shape extraction process. Thus, the presence of high noise in images may incur high classification error if the image retrieval model is not equipped with adequate filters to account for such noise.  
Lastly, we would like to mention that, for the performance evaluation of our proposed model, we used the NLM dataset [46]. This dataset was published by the National Institutes of Health for an open research competition, and it has since been widely used by various seminal exiting works in the area of pill identification from images. For the sake of performance comparison with existing studies in the same area (see Table 3, we adopted  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**ResNet-50** & **CNN+SVM** & **CNN+kNN** \\
**(Best Performing (Table 3)** & & **(Our Proposed Model)** \\ \hline
1.25 ms & 1.05 ms & 1.02 ms \\ \hline \hline \end{tabular}
\end{table}
Table 5: Runtime Comparison (ms).  
the NLM dataset in our evaluations. That said, we plan on using more datasets in our future work that builds on this study to further validate our findings.