# Smart Diet Diary: Real-Time Mobile Application for Food Recognition
## 3 Materials and Methods
### Volume Estimation
There were no restrictions on the number of meal entries per day. Users can also see the history of food intake. #### 3.7.3 Food Classifier Component

The user can take a picture or select one from their phone's storage and transmit it to the back-end server for prediction using the food classifier component. The food item's most distinguishing characteristics must be visible in the photograph. Additionally, as shown in Figure 6c, a coin must be held parallel to the food object at the same height. This is required to obtain a more accurate estimate of calories. When the server responds, a

Figure 6: Different components of front-end: (**a**) user profile, (**b**) diary component and database, (**c**) food classifier component, and (**d**) food correction component. summary of the detected foods and their calorie counts are shown on the screen. The user now has two options: either turn to the food correction panel or put the items on show to their diary. #### 3.7.4 Food Correction Component

Food prediction is not precise all the time as it is dependent on the angle or brightness of the picture. We have incorporated a food correction capability as a remedy to this problem. The food correction element grants the user the ability to fix the results in case of misclassification or no classification. Users may add, remove, or edit items using this component, as shown in Figure 6d. There are two means to insert or change items. The former allows the user to select the correct food item with the correct calorie density from the database and only require supplying the mass (in grams) of the meal. In the latter case, users can enter custom items along with the calorie count. ## 4 Results and Discussion

The model developed was trained and evaluated using the TensorFlow Object Detection API designed by Google and pretrained model, the fast-er_rcnn_inception_v2_coco, as a starting point. The pretrained model was acquired from the TensorFlow detection model zoo used in [17] which had a collection of pre-trained models with distinct precisions and speeds. The trained model was assessed using 20% of the images in the dataset. The training was performed with numerous changes to the decaying learning rate and was left for five epochs of training. ### Evaluation Metrics

We used mean average precision (mAP) to gauge the performance of the system as this is the most frequently applied evaluation metric for object detection and image retrieval. It measures the average precision of a model across multiple classes or queries. The formal definitions for accuracy, precision, and recall are provided in Equations (2)-(4)

\[\text{Accuracy}=\frac{\text{TP}}{\text{TP}+\text{FP}+\text{FN}} \tag{2}\]

\[\text{Prescision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{3}\]

\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{4}\]

AP is the area under the precision-recall curve, which plots the precision against the recall. Once the AP score for each class or query is computed, the mAP score is calculated as the mean of the AP scores across all classes or queries. In other words, the mAP score provides an overall performance measure of the model, considering its ability to retrieve relevant instances across multiple classes or queries, as given by Equation (5). \[\text{mAP}=\frac{\sum_{n}^{\text{N}}\text{AP}}{\text{N}} \tag{5}\]

where AP is average precision for a query; n is the set N of queries. ### Evaluation Results

The accuracy of the model was evaluated offline and in real-time. The real-time scenario involves identifying certain foods 100 times and determining the probability of correct classification. To evaluate the performance of the calorie estimation, the average calorie count over 100 different images was determined and compared to the ground truth. We used mean average precision (mAP) to evaluate the performance of our system.