# Smart Diet Diary: Real-Time Mobile Application for Food Recognition
## 3 Materials and Methods
### Volume Estimation
The following setups had the most success in classification: a decaying learning rate of 0.0002 to 0.00002, achieved a mean average precision (mAP) of 0.7907, an average recall (AR) of 0.8132, and a decaying learning rate of 0.002 to 0.0002 led to a mean average precision (mAP) of 0.8211 and an average recall (AR) of 0.8352. These two methods have remarkably similar accuracies, although the latter slightly outperforms the former, as shown in Figure 7.

Real-time testing of the application was carried out in the locations where they are most likely to exist or where food is consumed, such as convenience stores located inside the premises of the university, food courts, and areas reserved for students to relax. These areas have different lighting conditions. As expected, during real-time validation, the accuracy of the model was affected (increase in misclassifications and decrease in confidence levels) by factors such as lighting, angle, and the image being out of focus. A model with 0.8211 mAP was used for testing in a real-time setup. There were 100 tests for the three classes (coke, apples, and oranges). As seen in Table 2, apples and oranges were correctly identified approximately 90% of the time, while coke cans were correctly identified 60% of the time. Calorie estimation was tested with the 0.8211 mAP model. It was used to estimate the calories of a Coca-Cola can, apples, and oranges 100 times; when the classification was correct, it returned an average calorie count. The average calorie count of these three classes was within a 10% error of the actual calories. The predictions can vary from the ground truth, as listed in Table 2, and this deviation can be very large at times. Coca-Cola can recognition underperformed compared with apples and oranges. This may be due to variations within the Coca-Cola can dataset, as a subset of the data was obtained through web scraping. This may limit the usability of the application because of its misclassification; however, the user is still able to use the food correction screen to choose the correct item manually. This application may have better usability if object detection can be performed on the device. For example, the model classifies food in real-time while simultaneously capturing a photo. In this way, the user can take time to find the perfect position to obtain a correct classification, and the application will be shown on the screen if the item is identified. In the current implementation, we used image augmentation according to which dataset is limited and which may not contain the image under different conditions. For the volume estimation, the average calories measured for a particular food were within a 10% range of actual calorie count (20 calories lower or higher for the mentioned \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Item** & **Accuracy** & **Average Calories** & **Actual Calories** \\ \hline Coca-Cola can & 60\% & 153 & 142 \\ \hline Apple & 92\% & 77 & 78 \\ \hline Orange & 90\% & 53 & 58 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results from the real environment test. Figure 7: Mean average precision for both setups. cases). The overestimation was likely due to the GrabCut algorithm not segmenting the image cleanly; thus, there were additional background pixels left in the image. In addition, the volume varied depending on the location of the coin. The user must hold the coin at the same height as the food item, as shown in Figure 8. If the coin is placed higher than the food, the calorie estimate is lower; if it is placed lower, the calorie estimate is higher. However, it is not difficult to obtain a consistent estimate if the user knows how high it is to hold the coin. Additionally, for the orange and apple estimates, the fruit was not always perfectly spherical. Depending on which side of the fruit faces the camera, calorie estimation is affected. Calorie estimation is limited by the fact that only a single image is used to determine the size of the food.