# CNN-Based Pill Image Recognition for Retrieval Systems
## 4 Results
### Performance Analysis
Given the pill images in the NLM dataset, we calculated the _mAP_ performance metric for _ResNet-50_, _CNN+SVM_, and _CNN+kNN_ (our proposed model), for a 0.1 \(\leq\) threshold \(\leq\) 0.9. The mAP metric comparison is illustrated in Table 2. Moreover, we plotted the Precision-Recall curves of the three models. A Precision-Recall curve of a prediction model visualizes the accuracy of the model. The larger the area under the curve is, the better the prediction quality (reflecting both good Prediction and Recall). Figure 9 shows three Prediction-Recall curves for the above-mentioned three models, respectively. The Precision in Equation (2) is computed upon determining the TP and FP values of a target class in a prediction task. These values can be determined after running the prediction model against a dataset of images. For example, if a query image contains a round pill shape, then is the model able to predict that the shape of the query image is in fact round? In other words, we would like to know how many round-shaped pills the model successfully predicted as round-shaped (as opposed to any other shape). In the above example, the target class was _Round_, which belongs to the \"shape\" property of a pill. Our model extracts three features from any query pill image: shape, color, and imprint (see Figure 6). To help us evaluate the performance of our model in terms of correctly identifying target classes, we constructed a confusion matrix for each pill feature. Figure 10 represents three confusion matrices for shape, color, and imprint, respectively. The x-axis and y-axis arbitrarily list values (or target classes) of a specific feature (e.g., _Round_, _Capsule_ and _Oval_ are values of the shape property). The x-axis represents the known target class, whereas the y-axis represents the predicted class. An intersection between any pair of values \((c_{x},c_{y})\) \begin{table}
\begin{tabular}{c c} \hline \hline
**Models** & **Mean Average Precision (mAP)** \\ \hline _ResNet-50_ & 80\% \\ _CNN+SVM_ & 86.3\% \\ _CNN+kNN_ (our proposed model) & 90.8\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: mAP Comparison Models. Figure 9: Precision–Recall curves of three pill identification models: _ResNet-50_, _CNN+SVM_, and _CNN+kNN_. on the x-axis and y-axis, respectively, represents the number of times (or percentage) that the model predicted \(c_{y}\) given a pill of a target class of \(c_{x}\). Figure 10: Confusion matrices reporting our model accuracies w.r.t. predicting pill: (**a**) shape, (**b**) colors, and (**c**) imprints. For example, Figure 10 demonstrates the ability of the proposed model to understand the shape of the query pill. Looking at the _Round_ target class on the x-axis, the model predicted _Round_ as _Round_ 98.9% of the times, but predicted _Round_ as _Oval_ 0.6% of the times. The Top-k Accuracy metric considers a model's prediction to be correct if the target class exists among the top k predictions. For example, given a query pill image \(I_{q}\) from the NLM dataset and k = 5, if the model matches \(I_{q}\) with an ordered set of \(n\) images \(\langle I_{1},I_{2},I_{3},\ldots,I_{n}\rangle\), where \(I_{q}\in\{I_{1},I_{2},I_{3},I_{4},I_{5}\}\), then the Top-k Accuracy metric considers this match to be a correct prediction. It stands to reason that, as k increases, the Top-k Accuracy is expected to increase, because the metric considers a larger pool of potential matches. For k = 1 and 5, Table 3 reports the results of the Top-k Accuracy of the four related models. Based on the evaluations performed by [PERSON] [9], _ResNet-50_[31] performed the best among all the other models. Next, we performed further comparisons between _ResNet-50_, _CNN+SVM_, and _CNN+kNN_ (our proposed model). Table 4 varies k in the Top-k Accuracy measure between \(1\leq\) k \(\leq\) 15, and reports the finding for _ResNet-50_, _CNN+SVM_, and _CNN+kNN_. We noticed that, as k increased, the Top-k Accuracy result in Table 4 also increased, since the set of potential matched images expanded.