A Cloud-Assisted Mobile Food Recognition System
Creators: name: Pouladzadeh, Parisa
Publisher: name: Université d'Ottawa / University of Ottawa
Year: 2017

description: Accurate instruments and methods for measuring food and energy intake are crucial in the battle against obesity. Providing users and patients with convenient, intelligent solutions that help them measure their food intake and collect dietary information is valuable for long-term prevention and successful treatment programs. In this thesis, we propose an assistive calorie measurement system to help patients and doctors succeed in their fight against diet-related health conditions. Our proposed system runs as an application on smartphones that automatically measures the calorie intake based on a picture of the food taken by the user. The key functions of our application involve image segmentation, image processing, and food classification and recognition. Client-side devices (e.g., smartphones, tablets) do have some limitations in their ability to handle the time-sensitive and computationally intensive algorithms required for our application. The image processing and classification algorithms used for food recognition and calorie measurement consume device batteries quickly, which is inconvenient for the user. It is also very challenging for client-side devices to scale the large amount of data and images that is needed for food recognition. The entire process is time consuming, inefficient, and frustrating for the users, and may deter them from using the application. In this thesis, we address these challenges by proposing a cloud-assisted mobile food recognition system. Our results show that the accuracy of the recognition step within this cloud-assisted application, compared to Support Vector Machine (SVM), is increased in single food portions, non-mixed plates, and mixed plates of food. In addition, by applying a deep neural network, the accuracy of food recognition in single food portions is increased to 100%. However, in real-life scenarios, it is common for a food image to contain more than one food item, which reduces recognition accuracy. To solve this problem, we propose a novel method that detects both food item combinations and their locations in the image with minimal supervision. In our dataset, images with only one food item are used as the training dataset, while images with multiple food items are used as the test dataset. In the training stage, we began by using region proposal algorithms to generate candidate regions and extract the convolutional neural network (CNN) features of all regions. Second, we performed region mining to select positive regions for each food category using maximum cover in our proposed submodular optimization method. In the testing stage, we began by generating a set of candidate regions. For each region, a classification score was computed based on its extracted CNN features and the application predicted food names of the selected regions. Our experiments, conducted with the FooDD dataset, show an average recall rate of 90.98%, a precision rate of 93.05%, and an accuracy rate of 94.11%, compared to 50.8%–88% accuracy in other existing food recognition systems. The analysis and implementation of the proposed system are described in the following chapters.