Testing the Validity of an Artificial Intelligence-based Program to Identify Foods and Estimate Food Portion Size Among Adults, a Pilot Study
Phase: NA
Conditions: Nutrition Assessment
Sponsor: Pennington Biomedical Research Center

Official title: Validity of an AI-based Program to Identify Foods and Estimate Food Portion Size 
NCT number: NCT05343585
Document date: 04/08/2022

1
Title: Testing the validity of an artificial intelligence-based program to identify foods and estimate food 
portion size among adults, a pilot study.   
Principal Investigator: Chloe Lozano, Ph.D. 
Co-Investigator: John W. Apolzan, Ph.D.; Sanjoy Saha, Ph.D.; Stephanie Broyles, Ph.D; Robbie Beyl, Ph.D; 
Corby K. Martin, Ph.D.
Protocol Version Date: April 4th, 2022
Background:
Accurately quantifying food intake is vital to promoting health and reducing chronic disease risk. Food 
intake encompasses energy intake, nutrient intake (macronutrients, micronutrients, vitamins, minerals), 
and intake of various food groups (e.g., fruits, vegetables); thus, it reflects the nutritional status of 
individuals. Nutrition affects disease risk, including risk of developing obesity,1 diabetes,2 and cancer,3 all 
of which negatively affect the United States (US). Nonetheless, accurate assessment of food and 
nutrient intake has remained challenging, despite methodological improvements. Self-report methods, 
namely food records, are a mainstay of nutritional research,4 with food recall being another popular 
method.5 These methods rely on the participant to accurately estimate portion size and, for food recall, 
remember what was consumed. The accuracy of these methods has been questioned6-9 and the 
problems with human recall have been comprehensively outlined.10 As a result, there remains a 
significant need for methods that are sufficiently accurate to provide users, clinicians, and researchers 
with data to guide health promotion efforts. 
The PortionSize™ app was designed by our laboratory to overcome the limitations outlined above and to 
guide users to follow specific diets. PortionSize relies on users capturing images of their food selection 
and waste. Food intake data are immediately provided since the user relies on built-in tools, including 
templates, to estimate portion size within the app. Although the PortionSize app provides users (and 
clinicians and researchers) with real-time information about food intake, it comes at the cost of 
increased user burden since the user must identify foods in their images and estimate portion size. This 
differs from other food photography-based methods that our laboratory and other groups have 
developed, including the Remote Food Photography Method© (RFPM) and SmartIntake® app. For 
example, when using the RFPM and SmartIntake app, users only need to capture images of their foods 
with the app; identify any foods that are not identifiable from a wrapper, container, or logo; and the app 
sends the images to the researchers for analysis. Indeed, based on our research, participants view 
SmartIntake as less burdensome and more preferred compared to PortionSize 11, and we expect that 
this could negatively affect peoples’ use of PortionSize over time.  
To reduce PortionSize’s user burden, we aim to utilize machine learning, image recognition, and artificial 
intelligence for nutrition tracking. However, before we can incorporate these technologies into 
PortionSize, they first need to be validated. Therefore, we aim to test the validity of Nutrition AI 
technology, currently embedded in the Openfit® app, to identify foods and to estimate portion size 
(food volume) using automated methods. When using the Nutrition AI in Openfit, if the user believes the 
automated identification of the food and/or portion estimate needs correcting, adjustments can be 
made manually. However, this user burden is likely to be low and will also be explored in the current 
study. Therefore, the primary aims of the current pilot study are to: 1a) test the validity of Nutrition AI at 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022

2
identifying foods, 1b) test the validity of Nutrition AI at quantifying the amount of food provided, and 
1c) assess user satisfaction and usability of Nutrition AI in Openfit among adults in Baton Rouge, 
Louisiana. Openfit is not currently designed to assess plate waste, though Nutrition AI has the potential 
to assess plate waste and ultimately food intake. Therefore, the secondary aims of this pilot study are 
to: 2a) test the validity of Nutrition AI to identify plate waste and, 2b) test the validity of Nutrition AI to 
quantify the amount plate waste, and 2c) to assess user satisfaction and usability of the Nutrition AI in 
Openfit to assess plate waste, among adults in Baton Rouge, Louisiana. From this data we can calculate 
food intake (food plated – plate waste). 
Primary Hypotheses:
1a) Nutrition AI will correctly identify ≥70% of foods plated with automated methods, and 
≥80% after user correction. 
1b) Compared to weighed values, energy estimates (kcal) of foods plated, from Nutrition AI, will 
be: 
i.
Within error margins of 20% without user correction (automated)
ii.
Within error margins of 15% with user correction (semi-automated)
Secondary Hypotheses:
2a) Nutrition AI will correctly identify ≥60% of plate waste with automated methods, and ≥80% 
after user correction.
2b) Compared to weighed values, energy estimates (kcal) of plate waste, from Nutrition AI, will 
be:
i.
Within error margins of 20% without user correction (automated)
ii.
Within error margins of 15% with user correction (semi-automated)
Methods
Study Design
For this pilot study, using a convenience sample, we will recruit up to 25 adults to use the Nutrition AI 
technology in Openfit to identify and estimate portion size of foods provided and simulated plate waste, 
and food intake in a laboratory setting at Pennington Biomedical Research Center (PBRC) and/or 
Louisiana State University (LSU). Laboratory members within the Ingestive Behavioral Laboratory will 
also test the ability of Nutrition AI to identify foods and to quantify foods provided, plate waste and food 
intake, in the laboratory. Meals will be simulated, and participants will not consume the foods provided. 
Participants and Recruitment
We will recruit up to 25 adults 
-
Inclusion criteria include:
o
Male or female, age 18-62 years
o
Self-reported body mass index (BMI) 18.5-50 kg/m2 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022

3
-
Exclusion criteria include:
o
Any condition or circumstance that could impede study completion
o
Unfamiliar with or not able to use an iPhone
NOTE: PBRC and LSU employees (vulnerable populations) will be targeted and invited to participate. The 
consent form will explicitly specify that if a PBRC or LSU employee chooses not to participate, there will 
be no impact on their employment.
Procedures
Written informed consent will initially be obtained. We will then obtain:
-
Demographics (self-reported): DOB, race, ethnicity, sex, employment, income, marital status,
education level (Lifestyle, Demographics, and Health Questionnaire)
-
Anthropometrics (self-reported): height, weight, BMI
On being deemed eligible, participants will receive training on how to use the Openfit app housing the 
Nutrition AI technology. Subsequently, participants will be randomly assigned to one of five study 
menus (5 participants per menu) of differing calorie levels (see Table 1). Participants will use a provided 
smartphone to identify foods and quantify the amount of food plated (e.g., an apple) and simulated 
plate waste (e.g., half an apple) with Nutrition AI. Within each study menu, participants will be served 
two meals of a similar calorie level. These two meals will be served during the same study visit. To 
ensure we are testing the validity of the Nutrition AI across differing portion sizes, each meal (e.g., the 
chicken meal), will be tested across two different study menus where the amount of food provided and 
plate waste differs. The amount of plate waste will be assigned randomly using an exponential 
distribution, with the minimal being 0% remaining (i.e., no waste) and the maximum being 74%12. On 
average we will aim for ~5% plate waste across tests to reflect average plate waste in free-living 
conditions13. 
These tests will be conducted via Openfit in a laboratory-based setting at PBRC or LSU. Simulated food 
provision and plate waste will be covertly identified and weighed, and participants will not eat the food 
provided. During the study visit, tasks completed by each participant on the iPhone will be recorded 
using the record function on the iPhone and the recording will be saved to the iPhone camera roll. Data 
will be accessed from the iPhone at study completion. Consent, training, and testing will span up to ~2 
hours.
Near the end of the visit, participants will complete the Computer System Usability Questionnaire 
(CSUQ) 14. The CSUQ is a standardized, reliable, and valid questionnaire originally designed to evaluate 
computer programs that has been used to quantify the usability of mobile phone apps. Participants will 
also complete a user satisfaction survey to quantify satisfaction and ease of use of Openfit for tracking 
dietary intake. 
Outcome measures
Primary outcomes: 
1a) Agreement surrounding identification of food and beverages provided compared with known 
identification, at the item level (e.g., for burger n=10), and across all items where identification is 
determined by:
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022

4
I.
Nutrition AI without user correction (automated)
II.
Nutrition AI with user correction (semi-automated)
1b) Error between mean estimates of portion weights (g), energy (kcal) and macronutrients (fat, 
carbohydrates, protein) and known portion weights, energy, and macronutrients of provided food and 
beverages, where estimates are determined by:
I.
Nutrition AI without user correction (automated)
II.
Nutrition AI with user correction (semi-automated)
1c) User satisfaction and usability of Nutrition AI for identification of food and beverages and for 
estimating portion size. 
Secondary outcomes: 
2a) Agreement surrounding identification of plate waste compared with known identification, at the 
item level (e.g., for burger n=10), and across all items, where identification is determined by:
I.
Nutrition AI without user correction (automated)
II.
Nutrition AI with user correction (semi-automated)
2b) Error between mean estimates of portion weights (g), energy (kcal) and macronutrients (fat, 
carbohydrates, protein) and known portion weights, energy, and macronutrients of plate waste, where 
estimates are determined by:
I.
Nutrition AI without user correction (automated)
II.
Nutrition AI with user correction (semi-automated)
2c) User satisfaction and usability of Nutrition AI for identification of food and beverages and for 
estimating portion size of plate waste. 
2d)  Error between mean estimates of portion weights (g), energy (kcal) and macronutrients (fat, 
carbohydrates, protein) and known portion weights, energy, and macronutrients of food intake (food 
plated – plate waste), where estimates are determined by:
I.
Nutrition AI without user correction (automated)
II.
Nutrition AI with user correction (semi-automated)
Statistical analyses
For a food identified through the Nutrition AI to be considered an exact food match, the name of the 
food identified must match or be a close match to the food served. For example, a fruit cocktail 
identified as a fruit salad is an acceptable match15. Proportions will be used to assess whether the 
percentage of food items plated that were correctly identified by Nutrition AI is different to the 
percentage of foods correctly identified by a criterion method (human rater). Descriptive data will also 
be used to describe the frequency at which food plated and plate waste were correctly identified. 
Mean error and Bland-Altman analysis16 will be performed to determine errors in estimation of energy 
intake and food group servings from the Nutrition AI compared to estimations from the criterion 
measure17. 
 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022
 

5
Responses to the user satisfaction and usability questionnaires will be assessed using frequencies and 
percentages for Likert scale data. Open responses will be evaluated using qualitative methods to identify 
common themes. 
Provisions to Monitor the Data to Ensure the Safety of Subjects
Adverse events will be monitored. 
Withdrawal of Subjects
Subjects may be withdrawn from the study at any time. If a subject voluntarily withdraws from the 
study, no additional data will be collected, and they will be considered dropouts in the study.
Risks to Subjects
This study involves no greater than minimal risk. The main risk is breach of confidentiality, and the PBRC 
team will work to minimize this during data collection, handling, and analysis.
Potential Benefits to Subjects
Participants may benefit by increased awareness of energy and nutrients in food served. 
Sharing of Results with Subjects
Participants will have access to the study results once published. 
Setting
The Ingestive Behavior, Weight Management, and Health Promotion Laboratory at PBRC, and 162 Knapp 
Hall or 252 Knapp Hall at LSU. 
Compensation
Participants will receive $50 for the successful completion of the study.
Provisions to Protect the Privacy Interests of Subjects
All attempts will be made to maintain a subject’s privacy. Safeguards such as password protected 
computers and networks have been put in place to limit access to subject data. Subjects will be given 
ample time to read over the consent, ask questions, and agree to participate in the research study. 
Subjects may decline to answer questions with which they are not comfortable. Each procedure will be 
explained to the subject before it is performed. We will always ensure the privacy of the subjects.
Compensation for Research-Related Injury
No compensation will be provided for research-related injury.
Economic Burden to Subjects
All study-related tests and procedures will be at no cost to the subject. 
Consent Process 
 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022
 

6
A designated and trained staff member will obtain informed consent during the visit. Ample opportunity 
will be given for the subject to review the consent form and ask any questions prior to signing the form. 
If subjects wish, they can take the form home and return at a different visit. We are aware that consent 
is an ongoing process.
 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022
 

7
Table 1. PortionSize AI Groups
Menu 1- 400 kcal each meal 
Burger
Chicken
Tomato
Ready Rice
Onion
Oranges
Burger bun
Carrots
Cheese slice
Butter
Ketchup
Sweet Tea
Cookies
Coke 
Menu 2- 800 kcal each meal 
Chicken
Pizza
Ready Rice
Lettuce
Oranges
Tomato
Carrots
Dressing
Butter
Cookies
Sweet Tea
Coke 
Menu 3- 500 kcal each meal 
Pizza
Pork Chop
Lettuce
Broccoli
Tomato
Butter
Dressing
Apple Slices
Cookies
Sweet Tea
Coke 
 
Menu 4- 700 kcal each meal 
Pork Chop
Lettuce
Broccoli
Carrot
Butter
Tomato
Apple Slices
Pre-cooked frozen chicken
Sweet Tea
Dressing
 
Milk
Menu 5- 600 kcal each meal
Lettuce
Burger
Carrot
Tomato
Tomato
Onion
Pre-cooked frozen chicken
Burger bun
Dressing
Cheese slice
Milk
Ketchup
Cookies
 
Coke 
 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022
 

8
References 
1. 
Swinburn BA, Caterson I, Seidell JC, James WP. Diet, nutrition, and the prevention of
excess weight gain and obesity. Public Health Nutr. 2004;7(1A):123-146.
2. 
Ley SH, Hamdy O, Mohan V, Hu FB. Prevention and management of type 2 diabetes:
dietary components and nutritional strategies. Lancet. 2014;383(9933):1999-2007.
3. 
Huang T, Yang B, Zheng J, Li G, Wahlqvist ML, Li D. Cardiovascular disease mortality and
cancer incidence in vegetarians: a meta-analysis and systematic review. Ann Nutr
Metab. 2012;60(4):233-240.
4. 
Willett WC. Nutritional epidemiology. New York: Oxford University Press; 1998.
5.
Lytle LA, Nicastro HL, Roberts SB, et al. Accumulating Data to Optimally Predict Obesity
Treatment (ADOPT) core measures: behavioral domain. Obesity (Silver Spring). 2018;26
Suppl 2:S16-S24.
6. 
Beaton GH, Burema J, Ritenbaugh C. Errors in the interpretation of dietary assessments.
Am J Clin Nutr. 1997;65(4 Suppl):1100S-1107S.
7. 
Tran KM, Johnson RK, Soultanakis RP, Matthews DE. In-person vs telephone administered 
multiple-pass 24-hour recalls in women: validation with doubly labeled water. J Am Diet Assoc. 
2000;100(7):777-783.
8. 
Schoeller DA, Bandini LG, Dietz WH. Inaccuracies in self-reported intake identified by
comparison with the doubly labelled water method. Can J Physiol Pharmacol.
1990;68(7):941-949.
9. 
Bandini LG, Schoeller DA, Cyr HN, Dietz WH. Validity of reported energy intake in obese
and nonobese adolescents. Am J Clin Nutr. 1990;52(3):421-425.
10. 
Archer E, Pavela G, Lavie CJ. The inadmissibility of What We Eat in America and NHANES
dietary data in nutrition and obesity research and the scientific formulation of
national dietary guidelines. Mayo Clin Proc. 2015.
11.
Höchsmann C, Fearnbach N, Dorling JL, et al. Preference, expected burden, and willingness to 
use digital and traditional methods to assess food and alcohol intake. Nutrients. 2021; 13, 3340. 
12. 
Roe BE, Qi D, Beyl RA, et al. The validity, time burden, and user satisfaction of the FoodImage™ 
smartphone app for food waste measurement versus diaries: a randomized crossover trial. 
Resour Conserv Recycl. 2020; 160:104858. 
 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022
 

9
13.
Roe BE, Apolzan JW, Qi D, et al. Plate waste of adults in the United States measured in free-living 
conditions. PLoS One. 2018; 13(2).
14. 
Lewis, J. R. IBM Computer Usability Satisfaction Questionnaires: psychometric evaluation and 
instructions for use. International Journal of Human-Computer Interaction. 1995; 7(1): 57-78
15. 
Schap TRE, Six BL, Delp EJ, et al. Adolescents in the United States can identify familiar foods at 
the time of consumption and when prompted with an image 14 h postprandial, but poorly 
estimate portions. Public Health Nutrition. 2010; 14(7): 1184-1191.
16. 
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of 
clinical measurement. Lancet. 1986;1(8476):307-310
17. 
Martin CK, Han H, Coulon SM, et al. A novel method to remotely measure food intake of free-
living individuals in real time: the remote food photography method. The British journal of 
nutrition. 2009;101(3):446-456.
 
Pennington Biomedical IRB FWA 00006218
Approved April 8, 2022