# CNN-Based Pill Image Recognition for Retrieval Systems
## 4 Results
### Performance Analysis  
Given an input pill image taken by a consumer, we wished to evaluate how accurate our model was at identifying the corresponding reference pill image based on pill shape, color, and imprint. Figure 8 visually showcases the result of applying our proposed pill image recognition model using the pill images in the NLM dataset. Each object in Figure 8 is a pill. Matched pills (consumer image and its corresponding reference image) were put next to each other. The objective of this figure is to visually demonstrate the overall accuracy of the proposed model. In the remainder of this section, we will use widely adopted accuracy metrics, namely, mean Average Precision (mAP), confusion matrices to measure True Positives, and Top-k Accuracy. Moreover, we compared our model and labeled _CNN+kNN_ with _CNN+SVM_ and _ResNet-50_[31]. The above-mentioned three accuracy metrics are based on the notions of Precision and Recall. Precision measures the fraction of correct identifications among all positive identifications. Recall measures the fraction of correct identifications among all the dataset's actual positives. The term \"positive\" refers to a target class; in this case, a pill. Below are the equations for Precision, Recall, and Accuracy:  
\[Precision=\frac{TP}{TP+FP} \tag{2}\]  
\[Recall=\frac{TP}{TP+FN} \tag{3}\]  
\[Accuracy=\frac{TP+TN}{TP+FN+FP+TN}, \tag{4}\]  
where _TP_ (True Positive) is the number of correct predictions of a target class, _FN_ (False Negative) is the number of wrong predictions of a target class, _FP_ (False Positive) is the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Features** & **Reference Image** & **Consumer Image** \\ \hline Format & jpeg & jpeg \\ Width & 2400 & 4416 \\ Height & 1600 & 3312 \\ XResolution & 72 & 180 \\ YResolution & 72 & 180 \\ ColorType & TrueColor & TrueColor \\ BitDepth & 24 & 24 \\ YCDCrPositioning & Centered & Co-Sited \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Metadata of reference and consumer images. Figure 7: Sample pill image: (**a**) Reference version. (**b**) Consumer version. number of wrong predictions of a non-target class, and \(TN\) (True Negative) is the number of correct predictions of a target class. For a classifier to correctly predict a target class, the classifier must find an \"acceptable\" match between a query image (consumer pill image) and a target image (reference pill image). This match is numerically defined by a _threshold_ that measures the fraction of the overlapping area between the query image and the target image. Based on this threshold, the values of Precision and Recall vary. Another performance metric that is commonly used in the evaluation of information retrieval and object detection systems is the mean Average Precision (mAP). The mAP metric measures the average precision values of a classifier across different Recall values. A higher mAP score indicates better performance of the model in retrieving relevant information or detecting objects accurately. The mAP incorporates the trade-off between Precision and Recall, and it considers both false positives (FP) and false negatives (FN). This measurement provides a broader understanding of the classifier accuracy in identifying pills. The mAP is calculated as follows:  
* For each object class, calculate the Average Precision (\(AP\)) as: \[AP=\frac{1}{n}*\sum_{k=1}^{n}\text{{Precision\_at\_each\_k\_object}}\] (5) where (\(n\)) is the total number of relevant items in the dataset for the given object class, and the Precision at each relevant k-object is the Precision calculated at the position of the relevant item in the ranked list of predicted items. * Calculate the \(mAP\) as the mean of the \(AP\) scores for all object classes: \[mAP=\frac{1}{N}*\sum_{k=1}^{N}AP_{k}\] (6)  
Figure 8: Matching the NLM dataset consumer pill images with reference pill images using the proposed model. where (_N_) is the total number of object classes in the dataset.