# Smart Diet Diary: Real-Time Mobile Application for Food Recognition
## 1 Introduction
Semi-automatic methods have been introduced to address under-reporting by utilizing different wearable technologies to automate parts of the meal recording process [10], but many application features still require input from the user. In addition, they do not provide feedback on the user's diet [11]. Fully automated systems come in the form of wearable devices that aim to minimize user interaction in food intake tracking [12]. The advantages of recording diet intake have been researched extensively [13]. In recent years, a considerable amount of the literature has been published on mobile calorie-tracking applications that utilize built-in cameras [14]. These studies often involve two key components: (1) food recognition and (2) volume estimation. Lately, researchers have suggested a deep learning model called a convolutional neural network (CNN), which eases the complexity and the number of weights due to a shared-weight network arrangement. It is widely applied in the domains of object recognition [15] and image segmentation [16, 17]. The research community has investigated the deployment of multitask CNNs for food identification and categorization at the same time [18, 19]. The knowledge acquired from an individual CNN can be shared to enhance the accuracy of these functions. It is particularly beneficial for identifying dishes that have similar appearances or for creating a classifier with a wider reach, as food can be recognized by retrieving recipes from the database. The volume is estimated by building a 3D model of the food by extracting salient features from the picture [20] or by reconstructing a voxel description from a similar depth map [21]. An alternative method is to employ a reference article with a known size to match the food items. An example of the reference item can be chopsticks due to their universal size [22]. Other references include checkboard [23] and credit cards [24]. Extracting 3D information from a single image is a complex problem and a hindrance to fully automated solutions which requires attention. Additionally, there is no single food recognition system that could recognize all kinds of foods. Therefore, researchers have focused on developing a solution which can solve their endemic problems. Other works have been dedicated to developing a food recognition system with greater precision by deploying CNN, but has not been successful in the delivery of processed information to the consumer and a healthcare staff in a quick and flexible way. We propose a user-friendly mobile application called Smart Diet Diary that overcomes the above shortcomings and improves semi-automatic calorie estimation using deep neural networks(DNNs) for object classification and detection; we use this information to calculate the volume and nutritional value. The current version of this application focuses on recognizing some of the most widely consumed food items by university students and their estimated nutritional value. The major contributions of this study are as follows:  
* We developed a semi-automated diet tracking smartphone application based on a food recognition engine trained using faster R-CNN. * We developed our food recognition model for mobile application and extended it with a mechanism to calculate the volume and approximate calorie value. * We generated a customized food image dataset composed of over 16,000 images from fourteen classes to train and test the system. * We achieved a combined accuracy of approximately 80.1%, with a maximum of 90.2% for some of the classes. The remainder of this paper is organized as follows. We present a detailed literature review of the techniques for food recognition and the calculation of nutritional values in Section 2. The details of our methodology and the structure of the food detection system are presented in Section 3, followed by Section 4, where features of mobile applications are presented. In Section 5, we evaluate our system using different images from the dataset and present the results of our experiments.