Measuring accurate dietary intake is considered to be an open research problem in the nutrition and health fields. Food portions estimation is a challenging problem as food preparation and consumption process pose large variations on food shapes and appearances. We use geometric model based technique to estimate food portions and further improve estimation accuracy using co-occurrence patterns. We estimate the food portion co-occurrence patterns from food images we collected from dietary studies using the mobile Food Record (mFR) system we developed. Co-occurrence patterns is used as prior knowledge to refine portion estimation results. We show that the portion estimation accuracy has been improved when incorporating the co-occurrence patterns as contextual information. Index Terms-Dietary Assessment, Food Portion Size Estimation, Geometric Model, Food Portion Co-Occurrence Pattern. 1. INTRODUCTION Six of the ten leading causes of death in the United States, including cancer, diabetes, and heart heart disease can be directly linked to diet. Due to the growing concern of chronic diseases and other health problems related to diet, there is a need to develop accurate methods to estimate individual's food and energy intake. Dietary assessment, the process of determining what someone eats during the course of the day, provides valuable insights for mounting intervention programs for prevention of many of the above chronic diseases. Measuring accurate dietary intake is considered to be an open research in the nutrition and health fields. Developing methods for dietary assessment and evaluation has continued to be a challenging task. [10] to automatically determine the food types and energy consumed by a user using image analysis techniques. In the mobile dietary assessment system that we developed, we focus on the use of image analysis techniques to automatically analyze and determine the food types and energy consumed by a user [11, 5, 12 ] from a single food image. Food volume estimation (also known as portion size estimation or portion estimation) is a challenging problem as food preparation and consumption process can pose large variations in food shape and appearance. To date several images analysis based techniques have been developed such as those based on single view image [13, 14, 15, 12, 10, 16, 17] , multiple images [18, 8, 19], video [20], 3D range finding [21] and RGB-D image [22] . We feel that either modifying the mobile device or acquiring multiple images/videos of the eating scene is not desirable for users. We focus on food volume estimation using a single-view food image as shown in [23] which reduces user burden. Estimating the volume of an object from a single-view image is an ill posed problem. Most of the 3D information has been lost during the projection process from 3D world coordinates onto the 2D camera sensor plane. The use of the priori information is required to estimate the food portions. In [15] food portion estimation was converted into pre-determined serving size classification hence the technique could not be generalized. In [14, 24] pre-defined 3D template matching was used however it required manual tuning hence scaling with many foods became a problem. Food portion estimation using geometric models [12] and the approach based on predicted depth map using a Convolutional Neural Network (CNN) [10, 25] overcame the scaling issue with many foods. We compared the food portion size estimation accuracy using both geometric models and depth images [26] . We showed that geometric model based approach achieved 462 978-1-5090-5990-4/17/$31.00
Similarly, Fang et al. REF use special fiducial markers placed in the scenes to estimate the food portion size.
The use of co-occurrence patterns in single image based food portion estimation
2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)
2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)
Computer Science Medicine