Pivot Breath Sensor Human Factors and Usability Study
Phase: NA
Conditions: Smoking Cessation, Smoking, Tobacco
Sponsor: Jennifer Marler, MD

Cam,t Inc. 
DocwncntNo.-
PIVOT BREATH SENSOR HUMAN FACTORS AND USABILITY STUDY 
PROTOCOL­
PROTOCOL NUMBER: 
Senior Clinical Research Associate, Clinical Affairs, Carrot lnc. 
Approved By: 
Director, Clinical Affairs, Carrot Inc. 
Sr. Director, Clinical and Medical Affairs, Canot Inc. 
Director, Quality Assurance, Carrot Inc. 
Chief Operating Officer, Carrot Inc. 
Principal, UserWise, Inc. 
CONPIDENTIAL 
DATE: February 11, 2019

Carrot Inc. 
Document No. 
CONFIDENTIAL 
2
Synopsis 
Device Names 
Pivot Breath Sensor 
Protocol Title 
Pivot Breath Sensor Human Factors and Usability Study 
Principal Investigators 
Co-Investigators 
Protocol Number 
Protocol Version 
Sponsor 
Carrot Inc.,
Study Design 
Prospective, observational open label, single center study enrolling up to 15 
subjects to evaluate human factors and usability of the Pivot Breath Sensor. No 
medical decisions will be made related to test results. 
Primary Objective 
Human Factors and Usability:  

Ensure that intended users are able to operate the Pivot Breath
Sensor independently

Validate appropriate mitigations of use related hazards identified in
risk management documentation. 

Uncover previously unforeseen use errors.
Potential Subjects 

Up to 15 subjects who self-report smoking 2 or more cigarettes each
day. 
Inclusion Criteria 

18-80 years of age

English speaking

Owns and uses a smartphone

Willing to sign the Informed Consent Form
Exclusion Criteria 

Prior experience with a study sponsored by Carrot Inc

Pregnancy
Recruitment 
Eligible subjects will be identified via a clinical research recruiter and/or 
advertisement at outdoor locations (i.e. flea markets, outside of movie theaters, 
shopping malls, train stations, stores that sell cigarettes), print media (i.e., local 
newspapers, billboards), and web media (i.e., Craigslist, Facebook).   
Study Session 
Single day study including: observation of user set up and use of the Pivot 
Breath Sensor, user documentation assessment, subjective feedback and rating 
scales. 
Human Factors and 
Usability Outcomes 
Assessment of intended user performance in use scenarios, user documentation 
assessment, subjective feedback and rating scales.  

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
3
Contents 
 
List of Abbreviations ........................................................................................................... 4 
 
Introduction .......................................................................................................................... 5 
2.1 
Background ....................................................................................................................... 5 
2.2 
Study Rationale ................................................................................................................. 6 
2.3 
Potential Risks and Benefits .............................................................................................. 6 
 
Objectives ............................................................................................................................ 7 
 
References ............................................................................................................................ 7 
 
Definitions............................................................................................................................ 7 
 
Pivot Breath Sensor, Study Device ...................................................................................... 8 
6.1 
Indications for Use ............................................................................................................ 8 
6.2 
Labeling and Packaging .................................................................................................... 8 
6.3 
Submitting a Breath Sample (Specific Steps) ................................................................... 9 
6.4 
CO Log .............................................................................................................................. 9 
 
Task Analysis and Identification of Critical Tasks ............................................................ 10 
 
Study Design and Overview .............................................................................................. 12 
8.1 
Study Design ................................................................................................................... 12 
8.2 
Study Endpoints .............................................................................................................. 12 
8.3 
Eligibility Criteria ............................................................................................................ 12 
8.4 
Subject Recruitment ........................................................................................................ 13 
8.5 
Subject Stipend ................................................................................................................ 13 
8.6 
Sample Size and justification .......................................................................................... 13 
8.7 
Study articles ................................................................................................................... 14 
8.8 
Equipment and Materials ................................................................................................. 14 
8.9 
User Manual and Training ............................................................................................... 14 
8.10  Test Environment ............................................................................................................ 15 
8.11  Study Personnel ............................................................................................................... 15 
8.12  IRB Oversight .................................................................................................................. 15 
 
Study Procedure ................................................................................................................. 15 
 
Data Collection .................................................................................................................. 16 
 
Technical Approach and Study Design .............................................................................. 17 
11.1  Human Factors and Usability Acceptance Criteria ......................................................... 17 
11.2  Study Session Format ...................................................................................................... 18 
11.3  Use Scenarios .................................................................................................................. 19 
11.4  User Documentation Assessment .................................................................................... 24 
11.5  Subjective Feedback and Rating Scales .......................................................................... 30 
11.6  Methods for Capturing Use Errors .................................................................................. 31 
 
Study Results ..................................................................................................................... 31 
12.1  Data Collection and Confidentiality ................................................................................ 31 
12.2  Analysis ........................................................................................................................... 32 
12.3  Report .............................................................................................................................. 32 
12.4  Deviations From Protocol ............................................................................................... 33 
12.5  Record Retention ............................................................................................................. 33 
 
Conflicts of Interest............................................................................................................ 33 
 
 ............................................................................................................ 34 
 
Literature References ......................................................................................................... 34 
 
Document Revision History: .............................................................................................. 34 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
4
 
 
 LIST OF ABBREVIATIONS 
CO 
Carbon Monoxide 
CRF 
Case Report Form 
USB 
Universal Serial Bus 
IRB 
Institutional Review Board 
ICF 
Informed Consent Form 
ppm 
parts per million 
QSG 
Quick Start Guide 
UM 
User Manual 
 
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
5
 INTRODUCTION 
2.1 
BACKGROUND 
Smoking tobacco products is a leading cause of preventable morbidity and mortality, 
excess health care expenditure, and lost work productivity.  
 
 
 
   
 
 
 
 
 
 
 
 
   
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Since our 2018 study, Carrot Inc. (Redwood City, CA, USA) has made improvements to 
the CO breath sensor (the Pivot Breath Sensor). The Pivot Breath Sensor consists of a 
stand-alone handheld breath sensor to measure CO in exhaled breath with a display screen 
to show results. This breath sensor can optionally be used with a smartphone app (Pivot 
Application). The Pivot Breath Sensor is intended to be used as a personal and mobile 
product and to be available ‘over the counter’.  
This study will assess the human factors and the usability of the Pivot Breath Sensor. 
2.2 
STUDY RATIONALE 
We aim to assess whether an untrained lay user group (representative of intended users) 
can operate the device and interpret the results correctly using only the instructions that 
will be provided with the marketed device.  
2.3 
POTENTIAL RISKS AND BENEFITS 
 Known Potential Risks 
There are no anticipated risks or harms to the subject.  No medical decisions are made 
based on study data. 
 Known Potential Benefits 
While we do not anticipate that subjects will receive any benefit from their 
participation in this study, it is possible that some subjects may experience an 
increased awareness of their smoking behavior. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
7
 Risk Benefit Assessment 
Given the non-invasive nature of the breath sampling and the data collection, there are 
no anticipated risks to subject safety.  Study staff will be with subjects and monitoring 
their use of the study devices for the duration of the study. Subjects are not asked to 
change their smoking behavior.  No medical decisions are made based on study data. 
 OBJECTIVES 
The objectives of the study are to: 
 Ensure that representative intended users are able to operate the Pivot Breath Sensor 
independently, focusing on the following: 
o Proper breath sample collection 
o User comprehension of how to navigate the Pivot Breath Sensor screen 
o User interpretation and comprehension of their results 
 Validate appropriate mitigations of use related hazards identified in risk management 
documentation 
 Uncover previously unforeseen use errors 
Overall, this human factors and usability testing is conducted to demonstrate that the Pivot 
Breath Sensor can be used by the intended users without serious use errors or problems, for 
the intended uses and under the expected use conditions.  
 REFERENCES 
 FDA Guidance Document Entitled, “Guidance for Industry and Food and Drug 
Administration Staff: Applying Human Factors and Usability Engineering to Medical 
Devices,” issued on February 3, 2016 
 FDA Guidance Document Entitled, “Design Considerations for Devices Intended for 
Home Use, Guidance for Industry and Food and Drug Administration Staff” issued on 
November 24, 2014 
 ISO/IEC 62366-1:2015, Medical devices – Application of usability engineering to 
medical devices. Geneva, International Electro technical Commission. 
 AAMI/ANSI HE75:2009, Human Factors Engineering – Design of Medical Devices. 
 https://www.bluetooth.com/what-is-bluetooth-technology/bluetooth-technology-basics 
 DEFINITIONS 
Close Call (a.k.a. “Near-Miss”): Instances in which a user has difficulty or makes a use error 
that could result in harm, but the user takes an action to “recover” and prevents the harm 
from occurring (FDA Guidance 2016). 
Critical Task: A user task which, if performed incorrectly or not performed at all, would or 
could cause serious harm to the patient, user, or instrument, where harm is defined to include 
compromised medical care. 
Objective Data: Data collected through direct observation. 
Pivot Breath Sensor:  The study device.  It is a personal mobile breath sensor that is capable 
of measuring the level of carbon monoxide (CO) in exhaled breath.  It is portable, battery-

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
8
powered, and small enough to be carried by the user throughout the day (pocket, purse, 
backpack).  The sensor is intended for single-user use by cigarette smokers to inform the user 
about how breath carbon monoxide levels are affected by smoking behavior.  The user 
submits a breath sample by exhaling (blowing) directly into the mouthpiece connected to the 
sensor.   
Subjective Data: Data collected through user feedback via comments, ratings, and 
assessments provided by participants. 
Task: Action or set of actions performed by a user to achieve a specific goal. 
Use Error: User action or lack of action that was different from that expected by the 
manufacturer and caused a result that (1) was different from the result expected by the user 
and (2) was not caused solely by device failure and (3) did or could result in harm. (Section 
3.9, FDA Guidance 2016). 
User: Person who interacts with (i.e., operates or handles) the Pivot Breath Sensor. 
 PIVOT BREATH SENSOR, STUDY DEVICE 
The Pivot Breath Sensor comprises a personal mobile breath sensor that measures the level of 
CO in exhaled breath and displays the exhaled breath CO value to the user directly on the 
device. The range of CO measurement is 0-100 ppm.  
The Pivot Breath Sensor (Figure 1) is portable, battery-powered, and small enough to be 
conveniently carried by the user throughout the day (pocket, purse, backpack).  The Pivot 
Breath Sensor is rechargeable using a micro-USB cable.  The user submits a breath sample 
by exhaling (blowing) into the mouthpiece of the Pivot Breath Sensor.  
6.1 
INDICATIONS FOR USE 
The Pivot Breath Sensor is a breath carbon monoxide monitor intended for single-user use 
by cigarette smokers to inform the user about how breath carbon monoxide levels are 
affected by smoking behavior.  The device is not intended to be used with other inhaled 
products. 
 
Figure 1. Pivot Breath Sensor 
 
6.2 
LABELING AND PACKAGING 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
9
The Pivot Breath Sensor package includes: 
 1 Pivot Breath Sensor 
 1 USB charging cable 
 1 Replaceable Mouthpiece 
 1 Quick Start Guide
 
The Pivot Breath Sensor packaging is shown in 
  
6.3 
SUBMITTING A BREATH SAMPLE (SPECIFIC STEPS) 
Submitting a breath sample involves the following steps. Instructions can be accessed in 
multiple locations including the Pivot Sensor’s display screen, the Quick Start Guide, on 
the packaging or also in the User Manual that will be available online.  
1. User presses any button to turn on the Pivot Breath Sensor. 
2. User presses and holds the center button on the Pivot Breath Sensor until the sensor 
beeps once. 
3. User takes a deep breath in and holds their breath for 10 seconds. 
4. After about 10 seconds, the device beeps 3 times, the user then will exhale slowly 
until the sensor vibrates at 12 seconds. 
5. The user’s CO level will display on the sensor’s screen. 
A properly submitted breath sample is defined as one that initiates after the third beep 
prompt and is of at least 6 seconds duration.  Breath sensor hardware and firmware is 
capable of detecting when a breath sample submission starts and stops. 
The CO Log screen in the Pivot Breath Sensor provides the CO (ppm) value for properly 
submitted breath samples and provides a descriptive error message for improperly 
submitted breath samples (i.e. “Error: breath sample too late”).  
The computation of concentration of CO (ppm) is performed within the Pivot Breath 
Sensor firmware and the data (CO ppm concentration, time and date) are stored in the 
Pivot Breath Sensor memory.   
6.4 
CO LOG 
The CO Log can be accessed from the Pivot Breath Sensor’s display screen and shows the 
most recent exhaled breath CO value in ppm at the top of the screen.  The user can view 
previous values by scrolling within the log (Figure 2).   
Values are color coded and can be interpreted by the user with help from the labeling 
(Figure 3).  
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
10
Figure 2. CO log on Pivot Breath Sensor  
 
Figure 3. Package labeling to interpret results  
 TASK ANALYSIS AND IDENTIFICATION OF CRITICAL TASKS  
For the human factors and usability portion of the study, a task analysis was performed to 
identify the tasks associated with the use of the Pivot Breath Sensor. The task analysis is 
depicted in Figure 4. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
11
Figure 4. Task Analysis of Pivot Breath Sensor 
(Note: Sub-work flow is shown in lighter blue.)  
 
  

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
12
As part of the product development process, a comprehensive use related failure mode and 
effect analysis (Use FMEA) was performed to identify and analyze the potential use-related 
failure modes (i.e., use errors) that might occur when the intended users interact with the 
Pivot Breath Sensor.  
The task analysis and the Pivot Breath Sensor Use FMEA were assessed for critical tasks. 
Based on the definition of a critical task provided in FDA “Guidance for Industry and Food 
and Drug Administration Staff: Applying Human Factors and Usability Engineering to 
Medical Devices,” issued on February 3, 2016 ("A user task which, if performed incorrectly 
or not performed at all, would or could cause serious harm to the patient or user, where harm 
is defined to include compromised medical care"), it was identified that the Pivot Breath 
Sensor does not have use related tasks that may result in serious harm or compromised 
medical care if the device is not operated properly by the user.  
 
Carrot Inc. chose to perform usability testing to assess user tasks and 
comprehension items that are essential to the effective operation of the device and user 
interpretation of breath sample results, and, to monitor for unforeseen use errors that could 
lead to serious harm.  
Table 1. List of “Severity 2” Tasks 
 User Tasks and Comprehension Items 
Severity 
 STUDY DESIGN AND OVERVIEW 
8.1 
STUDY DESIGN 
This is a prospective open-label human factors and usability study, conducted with IRB 
approval, enrolling up to 15 subjects who report daily smoking.  The study will be 
conducted at our research center in Redwood City, CA or an equivalent location. Subjects 
will be asked to set up the Pivot Breath Sensor and provide a breath sample using the 
device during a 60-minute session. No follow-up visit is required. 
8.2 
STUDY ENDPOINTS 
Assessment of intended user performance in use scenarios, user documentation 
assessment, subjective feedback, and rating scales. 
8.3 
ELIGIBILITY CRITERIA 
The intended user population (user group) for the Pivot Breath Sensor System are lay 
users who are smokers, ages 18-80, capable of using a smartphone to track/monitor health-
related data. The intended users range in age, degree of smoking, mental and physical 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
13
capabilities, health literacy, experience using other CO analyzers or home use medical 
devices, and experience using smartphones.  
Study participants will consist of individuals who represent the intended user profile, 
current smokers. Participant criteria include all of the following: 
Inclusion Criteria 
 18-80 years of age 
 Current daily cigarette smokers (at least 2 cigarettes per day with target distribution 
of individuals who smoke: 2-5 cigarettes per day, 6-10 cigarettes per day, 11-15 
cigarettes per day and 16+ cigarettes per day)  
 Resident of the United States 
 Able to read and comprehend English 
 Currently own and use an Apple, Inc. iPhone iOS smartphone or an Android 
smartphone for at least one month (such that they are familiar with basic operations 
of the mobile platform) 
 Willing to sign the Informed Consent Form 
Exclusion Criteria 
 Prior experience with a study sponsored by Carrot Inc. 
 Pregnancy  
8.4 
SUBJECT RECRUITMENT 
Subjects will be recruited in the United State by a clinical research recruiter and/or 
through advertisement at outdoor locations (i.e. flea markets, outside of movie theaters, 
shopping malls, train stations, stores that sell cigarettes), print media (i.e., local 
newspapers, billboards), and web media (e.g. Craigslist, Facebook, Task Rabbit, Google 
Ads) 
. Potential subjects will convey their interest in participating in the 
study by calling or emailing study personnel using the contact information provided in the 
recruitment materials, or responding to outreach from the clinical recruiter. 
All study participants will undergo a screening call where they will be asked questions to 
confirm study eligibility. During this call, study personnel will inform the potential subject 
of the study details and will answer any questions. Eligible participants who meet the 
inclusion/exclusion criteria will be offered the opportunity to participate in the study. 
Potential subjects will then determine if they would like to proceed with scheduling, and if 
so, study personnel will schedule the subject for a test session.  At the test session, an 
Informed Consent Form 
 will be signed prior to participation. 
8.5 
SUBJECT STIPEND 
Participants will be compensated $100 in the form of a Visa Gift card for the test session 
lasting approximately 60 minutes. Participants will be compensated at the end of their 
study session. 
8.6 
SAMPLE SIZE AND JUSTIFICATION 
Per FDA Guidance (US Food and Drug Administration (FDA), 2016) and HE75 – Human 
factors engineering – Design of medical devices (ANSI/AAMI, 2013), a Usability 
Validation Study should include at least fifteen (15) participants of each user group. Since 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
14
the Pivot Breath Sensor has one (1) user group and no critical tasks were identified, this 
study will include up to 15 subjects. This sample size is expected to provide adequate data 
to assess the human factors and usability of the Pivot Breath Sensor.1 
8.7 
STUDY ARTICLES 
Pivot Breath Sensor in representative packaging 
 that includes: 
 Pivot Breath Sensor  
 USB charging Cable 
 Replaceable Mouthpiece  
 Quick Start Guide 
 
 User Manual (available on an iPad) 
In addition, the testing room will be equipped with electrical outlets and a charging cube.  
8.8 
EQUIPMENT AND MATERIALS 
 A pre-set Pivot Breath Sensor with example CO breath samples, or a CO Log Print 
Out 
 Printed prompt cards for the participant to reference during each task 
 Timer or clock 
 Video recording equipment 
 Camera 
 Post-it notes 
 Quick Start Guide for Observer/ Moderator 
 User Manual (printed) for Observer/ Moderator 
 Electronic Surveys 
 Case Report Forms 
A full list of equipment and materials used during the study will be recorded in the report. 
8.9 
USER MANUAL AND TRAINING 
The Pivot Breath Sensor was designed to be as intuitive as possible, such that users are 
able to understand how to use the product on their own. No in-person training is formally 
offered to Pivot Breath Sensor users. A Quick Start Guide and package labeling is 
included with the device and an online User Manual is also available to aid the user. All of 
these items will be accessible to the study participant plus a phone and a phone number 
within the labeling material to contact technical support. Since in-person training will not 
be formally offered to all users who purchase the Pivot Breath Sensor, in-person training 
will not be included in this study. Study participants will have access to the 
aforementioned materials.  Any instance of participants referring to the User Manual, 
                                                 
1 In previous usability testing, two subgroups of users used a similar, predecessor breath sensor, with one subgroup 
including 18 to 49-year-olds, and with the other subgroup including 50 to 80-year-olds. User performance was 
compared between the two subgroups, and no statistical difference was identified between the two cohorts for any of 
the tasks, except for accessing the CO Breath Sensor Frequently Asked Questions (FAQs). Based on previous 
testing, it was concluded that age is not significant enough of a contributing factor to require two separate user 
profiles in usability validation testing for this product. As such, a single user group consisting of participants with a 
range of ages will be tested as part of this study. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
15
Quick Start Guide or packaging during the study will be recorded.  
While participants will have the User Manual, Quick Start Guide and packaging available 
to them at any point during the study, they will not be asked to read instructions (except 
during the knowledge questions covered in the User Documentation Assessment – see 
Section 11.2) or use any of the forms of training available.  
8.10 TEST ENVIRONMENT 
The study will be performed in an office building conference room or office. 
 Location testing will be performed:  
The location of testing will either be at the Carrot Inc. office 
 
 or equivalent. For example, office space may be rented 
to be nearer to test subjects.  
8.11 STUDY PERSONNEL 
The study team members will be trained by reading the study protocol and reviewing the 
case report forms. The study team members will have full training on the operation of the 
study devices. 
The following roles will be fulfilled by Carrot Inc. employees and consultants during the 
study: 
 Study Moderator – Responsible for moderating the study, delivering instructions (as 
needed) and ensuring that participants progress from one use scenario to the next, 
and asking questions to probe about observed use errors and close calls. 
 Study Facilitator – Responsible for organizing the simulated environment, study 
articles, equipment, and required documentation (e.g. informed consent form, non-
disclosure agreements).  
 Study Observer(s) – Responsible for taking notes during the study sessions and 
documenting key events of the study as they occur.  
 Technical Support – Responsible for troubleshooting system issues and recording 
bugs in the case there is a system issue during the study session. 
One person may fulfill more than one role.  
8.12 IRB OVERSIGHT 
This study will be conducted under IRB oversight.  
 STUDY PROCEDURE 
At the beginning of the test session, subjects will have ample time to read the Informed 
Consent Form (ICF) and the opportunity to ask questions. The subject will sign the ICF 
before participating in this study. Once signing the ICF, the subject will be given a unique 
identifier to protect subject privacy.  Subjects may choose to exit the study at any time.  They 
are not required to complete the study if they choose to stop. 
Study participants will self-train on how to use the study device using only the available 
study Pivot Breath Sensor device labeling, including product packaging and Quick Start 
Guide. A technical support line provided in packaging/Quick Start Guide and the User 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
16
Manual will be available should a participant request them.  These labeling materials will be 
available to study participants at any point during the study. Study participants will not be 
asked to read instructions or use any of the forms of training available except during the 
knowledge questions covered in the User Documentation Assessment – see Section 11.4. 
Study personnel will refrain from answering questions on how to use the study device during 
the study.  
Once the study participant has completed the self-training on the Pivot Breath Sensor, they 
will be asked if they feel ready to proceed with submitting a breath sample.  If a participant 
indicates they do not feel ready, they will be asked to do what they would normally do at 
home to prepare to use the product and to let the test administrator know when they are ready 
to submit another sample. 
When the participant indicates readiness to provide a breath sample, he/she will be instructed 
to proceed to provide one breath sample using the study device. After providing this breath 
sample, the subject will be asked to interpret his/her breath sample result followed by 
interpretation of other possible breath sample results and the first set of subjective questions. 
The subjective questions will address the subject’s feedback on how giving a breath sample 
went for them. At which point, the participant will be asked to change the mouthpiece. An 
optional breath sample may be requested by the moderator followed by interpretation of 
his/her breath sample result.   
Upon completion of these breath sample(s), participants will be asked for subjective 
feedback, if they experienced difficulty, concerns, or moments of hesitation while using the 
product, and if they have safety concerns regarding the product or its labeling. Next, they will 
be assessed for their newly learned knowledge of the device (focused on if CO is “good or 
bad for you”) and their ability to find and understand information in the product labeling. 
This is followed by obtaining the participants additional subjective feedback and rating 
scales. 
 
 DATA COLLECTION 
Study personnel will collect information about the participant including 
 Age 
 Gender  
 Race/ethnicity  
 Vision 
o Wear glasses? 
o Color blind? 
 Hearing 
o Hearing issues? 
o Use a hearing aid? 
 Type of smartphone participant uses (iPhone or Android) 
 Number of years smoking  
 Number of quit attempts over the past 12 months 
 Number of cigarettes currently smoked per day 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
17
 Smoking history in the past 24 hours  
o Number of cigarettes smoked 
o Time of last cigarette 
o Use of any combustible, inhaled tobacco product other than cigarettes (pipes, 
cigars, cigarillos, etc.), any electronic cigarette product (e-cigs, vape, vapor), and 
any other inhaled combustible material 
 Attitudes towards smoking 
o Desire to quit 
o Readiness to quit 
o Confidence to quit 
o Perceived difficulty of staying quit 
 Current smoking cessation medications 
 Participation in a smoking cessation program 
o Past 
o Present 
 Breath sensor use (past) 
o Use of CO breath sensors in the past 
 Breath sensor use during the study session 
o Model#/Lot# of breath sensor 
o CO measurements 
 TECHNICAL APPROACH AND STUDY DESIGN 
11.1 HUMAN FACTORS AND USABILITY ACCEPTANCE CRITERIA 
Each use scenario and user documentation assessment test case will be attributed one of 
the following assessments for task success: 
 Successful: User is able to complete the task safely and effectively. If the user does 
not complete the task on first try but self-corrects to complete the task without 
moderator direction or assistance, the task completion may be considered 
Successful. 
 Difficulty: User is able to complete the task safely and effectively, but has 
significant hesitation or challenges while completing the task. If the user does not 
complete the task on first try but self-corrects to complete the task without 
moderator direction or assistance, the task completion may be considered 
Difficulty. 
 Close Call: User is able to complete the task safely and effectively, but performs it 
in a way that presents the potential for user harm (i.e. near-miss). 
 Unsuccessful (Use Error): User is unable to independently complete the task or 
does not complete the task safely and effectively. Includes instances in which users 
are unable to complete the task and as a result require study moderator assistance. 
Instances in which the user self-corrects without requiring moderator assistance do 
not qualify as unsuccessful. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
18
 Did Not Perform: User does not complete the task for safe and effective use of the 
device.  
At any time, the moderator may remind the participant of the scenario of the testing and 
the frame of mind they should strive for. A possible reminder may be “Continue to do 
your best…as a friendly reminder imagine that you are home, that we are not here, and 
that you just bought this product from the store.” Any instance of study moderator 
assistance will be noted and assessed depending upon the extent of the assistance 
provided.  
The Pivot Breath Sensor Use FMEA governs Acceptance Criteria for the Pivot Breath 
Sensor Usability Validation Study. Each hazard-related use scenario observed during 
usability validation testing will be documented and assessed for risk acceptability as 
defined in the Use FMEA.  
In the case that a hazard-related use scenario is concluded to have an acceptable risk profile per 
the Use FMEA, its associated test case is considered to “pass.” In the case that a hazard-related 
use scenario is concluded to have an unacceptable risk profile per the Use FMEA, its associated 
test case is considered to “fail.” In the case of a failed test case, further risk mitigation and 
usability evaluation would be required. In the case that further risk mitigation is not possible, a 
risk-benefit analysis would be documented.   
 
Since there are no critical tasks associated with this product, there is no pre-identified task-
specific acceptance criteria related to this study; no use errors were identified that could lead to 
serious harm. Overall, user performance on each task will be monitored for unforeseen use errors 
that could lead to serious harm. 
 
11.2 STUDY SESSION FORMAT 
 
Each study session will have the following format: 
1. Introduction  
2. Simulated Use 
3. Subjective Feedback 
4. Knowledge Tasks 
5. User Documentation Assessment 
6. Subjective Feedback and Rating Scales  
Following a brief introduction each participant will undergo a study session lasting about 60 
minutes. The study session will comprise use scenarios, knowledge tasks, user 
documentation (User Manual and Quick Start Guide) assessment, subjective feedback and 
rating scales. In addition to the subjective feedback obtained during the defined sections 
below, subjective feedback may be collected during the use scenarios, knowledge tasks, and 
User Documentation assessment to inform root cause analysis. 
The use scenarios included in the study consider inputs from risk management 
documentation (Pivot Breath Sensor Use FMEA) to test mitigations to device use (operation) 
related hazards. Each use scenario is expected to take up to 5 minutes (most will take 2-3 
minutes). The use scenarios are listed in Table 2. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
19
The study session will be conducted as a simulated use study where each participant will 
have access to the study article and will be monitored while performing the use scenarios. 
Each use scenario is monitored for difficulties, close calls and use errors. The use scenarios 
will allow for a detailed and focused assessment of the different components of the Pivot 
Breath Sensor set-up and use.  
Table 2. Use Scenarios for Usability Validation 
ID 
Use Scenarios 
1 
 
2 
  
3 
 
4 
 
5 
ple 
6 
 
 
11.3 USE SCENARIOS 
 Use Scenario 1: Self-Selection 
 Configuration: 
The study moderator presents the participant with the Pivot Breath Sensor device 
package. 
 Prompt:  
Take a moment to examine the package. Now I will ask you some questions about 
whether this product is appropriate for different types of people. (Note:
 
) 
1. 
 
 
2. 
 
 
 
 
 
 
3. 
 
 
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
20
 Use Scenario 2: Start-up device  
 Configuration: 
The study moderator presents the participant with the Pivot Breath Sensor device 
package. 
 Prompt: 
Here is a new Pivot Breath Sensor that has never been used. Please imagine you just 
bought this product from the store. Given that you just purchased it, you can take as 
much time as you need to familiarize yourself with the product. Pretend that we are 
not here. Please start using the product as you normally would at home when you are 
ready. Feel free to take as much time as you need. 
(If user does not begin using device after familiarization period) Please proceed with 
using the product. 
(If the user states they would not use or purchase this product) Let’s assume you were 
given this product to use as a requirement of a smoking cessation program. Take as 
much time as you need to familiarize yourself with the product. Please start using the 
product when you are ready.  
 Tasks: 
o Open the Pivot Breath Sensor packaging  
o Complete Breath Sample 
 Prompt: 
How did that go for you? 
 Use Scenario 3: Interpret your results – Part 1 
 Prompt: 
Please walk me through your test results. What does each result mean?  
 
 
 Tasks: 
o Access the CO Log  
o Understand results from the breath tests 
Note: This Use Scenario may be continued for interpreting results for Use Scenario 5. 
(See section 11.3.5 below.) 
Expected Response: 
For green value:  
 
 
   
Or,  
 
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
21
  
Or,  
 
 
 
 
 
For orange value:  
 
 
    Or,  
 
 
  
Or, 
 
 
 
 
 
 
For red value:  
 
  
    Or,  
 
  
Or, 
 
  
 
 
 Use Scenario 4: Interpret more results 
 Configuration: 
The study moderator provides Device with at least three (3) past CO readings having 
all of the colored CO ppm ranges for the participant to interpret (red, green orange) so 
participant can interpret colored ppm ranges he/she did not obtain from his/her own 
breath sample(s). 
 Study moderator and/or observer to monitor participant 
 
 Prompt 1: 
What does it mean if the result of a person’s breath test looks like this? (Moderator 
points to a green, orange, or red result. An attempt will be made to evenly distribute 
the order the colors are presented across all groups by counterbalancing. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
22
Counterbalancing scheme will be provided in the final report. Moderator will only ask 
about results that were not previously covered in Scenario 3) 
Expected response: See 11.3.3 response 
 Prompt 2: 
What does it mean if the result of a person’s breath test looks like this? (Moderator 
points to a green, orange, red or gray result, counterbalanced across participants. 
Moderator will only ask about results that were not previously covered in Scenario 3) 
Expected response: See 11.3.3 response 
 Prompt 3:  
What does it mean if the result of a person’s breath test looks like this? (Moderator 
points to a green, orange, red or gray result, counterbalanced across participants. 
Moderator will only ask about results that were not previously covered in Scenario 3) 
Note that Prompt 3 will be asked unless all result colors were previously covered. 
Expected response: See 11.3.3 response 
 Prompt 4: 
 
 
 
 
 
 Use Scenario 5: Use a new mouthpiece and optionally submit a breath sample 
 Prompt:  
Please replace the mouthpiece on the Pivot Breath Sensor. 
  Prompt (Optional) 
Please replace the mouthpiece on the Pivot Breath Sensor and submit a breath sample.  
 Study moderator and/or observer to monitor for lightheadedness while providing a 
breath sample. 
 Study moderator and/or observer to monitor and record the number of seconds of 
participant’s breath hold starting from inhalation. 
 Tasks: 
o Assemble the Pivot Breath Sensor with the new mouthpiece  
o Submit a breath sample (Optional) 
 Follow up prompt (Optional):  
(If participant does not switch out mouth piece before submitting the new sample): 
Please use a new mouthpiece. 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
23
 Study observer confirms breath sample taken 3 to 5 minutes from last successful 
breath sample and records the time of the sample submission. 
 Prompt: 
How did that go for you? 
(Next prompt is a Continuation from Use Scenario 3 above) 
 Prompt: 
Please walk me through your test result. What does this result mean?  
 
 
 Tasks: 
o Access the CO Log  
o Understand result from the breath test 
Expected response: See 11.3.3 response 
 
 Use Scenario 6: Charge the Pivot Breath Sensor  
 Prompt: 
Please proceed to charge the Pivot Breath Sensor. 
 Tasks: 
o Connect the USB charging cable 
o Charge the device 
 
 Subjective Feedback Part 1 
 Prompt: 
Were there any points during this portion of the study that you experienced difficulty 
or had concerns about the product? 
 Prompt: 
Did you experience any moments of hesitation while using the product? 
 Prompt: 
Do you have any safety concerns regarding the product or labeling? 
 Prompt (Optional): 
Did you hold your breath when you submitted the recent samples? 
 
  Knowledge Tasks 
 Prompt:  

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
24
 
 
 
 
 
 
   
 
Upon completion of the use scenarios and knowledge tasks (described above), a user 
documentation assessment session (described below) will be performed followed by a 
final subjective feedback session to better inform the safety and efficacy of the Pivot 
Breath Sensor. 
11.4 USER DOCUMENTATION ASSESSMENT 
Participants will never be asked to refer to the labeling during simulated use scenarios. 
Any instance of participants referring to the labeling during the simulated use scenarios 
will be recorded and assessed. 
After completion of the simulated use scenarios, the user documentation (Quick Start 
Guide, Packaging, and User Manual) will be tested for ease of use and clarity of content. 
Each user documentation assessment session will have test cases to assess the findability 
and the interpretation of knowledge task data found in the User Documentation. 
Participants will be asked to find and/or interpret pieces of essential information. 
Participants will demonstrate understanding through verbal explanation. The questions 
and expected answers for the user documentation assessment are found in Table 3. 
Table 3. User Documentation Assessment 
User 
Documentation 
Task ID 
User 
Documentation 
Assessment 
Tasks 
Question 
Participant Response 
 
 
1 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
25
   
2 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
  
 
 
  
 

Carrot Inc. 
 
 
Document No.
  
 
CONFIDENTIAL 
26
 
 
 
 
 
3 
  
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
27
  
 
 
 
 
 
4 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
  
 
 
 
 
  
 
 
  
 
 
 
 
 
 
5 
  
 
 
 
 
 
 
. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
28
6 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
  
7 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
  
 
  
 
 
 
  
8 
 
 
 
 
  
 
 
 
 
 
  
 
 
  
 
 
 
 
  
 
  
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
29
 
 
. 
9 
 
 
 
 
 
 
 
 
 
 
 
  
  
 
 
10 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
  
 
 
 
11 
 
 
 
 
  
 
 
 
 
 
 
  
 
 
 
 
12 
 
 
 
  
 
 
 
 
 
  
 
 
 
 
 
 
 
 
  
 
 
. 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
30
11.5 SUBJECTIVE FEEDBACK AND RATING SCALES 
In addition to the subjective feedback obtained periodically throughout the study session, 
13 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
  
  
14 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15 
(Ask only if 
participant did 
not state both 
expected 
answers) 
 
 
 
 
 
 
 
 
 
 
 
 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
31
additional subjective feedback may be collected during the use scenarios and user 
documentation assessment to inform root cause analysis.  
Following the user documentation assessment, the participants will be subjected to rating 
scale questions to rate the ease of use of the device, the clarity of the user documentation 
and their ability to understand and interpret the test results.  
11.6 METHODS FOR CAPTURING USE ERRORS 
During each study session, observer(s), and moderator(s) will observe for difficulties, use 
errors and close calls that could inform modifications to the Pivot Breath Sensor Use 
FMEA. The study moderator will also ask probing questions, at the end of the session, to 
reveal close calls that were not conveyed through observation. All use errors and 
difficulties will be followed with questions to identify how and why the participant 
believes they occurred. New, unforeseen use-related hazards will be assessed for risk 
acceptability and added to the Pivot Breath Sensor Use FMEA.  
The study team will include subjective assessments by participants for any observed use-
related issues that could potentially result in harm to the user.  
Users will perform tasks independently and in a natural manner, without guidance, 
coaching, praise, or critique from the test facilitator or moderator. Users will not be 
allowed a “second chance” to perform a task correctly after a failure. That is, if an 
irreversible safety or efficacy issue is observed in which a user would not normally have a 
“second chance” to resolve the issue, then any subsequent attempts to complete the task 
will serve as input only for continuous usability improvement purposes. In instances in 
which an irreversible safety or efficacy issue is not observed and the user would normally 
have additional chances to resolve this issue, any subsequent attempts by the user to 
complete the task will be taken into consideration when attributing an assessment (i.e. 
successful, difficulty, unsuccessful, close call, did not perform) to the task. 
The study team will ensure participant feedback is documented throughout the session, as 
well as any protocol deviations. Prior to the end of the test session, the study team will 
review all data sheets for completeness and pursue additional information or clarification 
as needed. The study team will then inform participants when the test session is complete. 
The study team will scan the data sheets and save them to a secure study file. All members 
of the study team will be familiar with the device operation to be qualified to administer 
the test and assess participant performance with the device. The study team will write the 
summary report. 
 STUDY RESULTS 
12.1 DATA COLLECTION AND CONFIDENTIALITY 
Subjects will be assigned a unique subject ID which will be used for data collection. Data 
collection will take place on paper or electronic case report forms (CRFs) completed by 
the study team and study subjects. CRFs will be reviewed by the study team prior to the 
end of the study session to ensure completeness and then stored in a secure filing cabinet 
at the end of the day with access limited to necessary study personnel. Study data will be 
kept in a secure database by the investigator. This database will be accessible only by 
necessary study personnel. The database will not contain personal identifying information; 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
32
all subject data in the database will be associated with study identification numbers. The 
data will be kept for 5 years. 
12.2 ANALYSIS 
The study will utilize paper-based data entry procedures, subject to audit per Carrot Inc. 
standard operating procedures.  
 Human Factors and Usability  
The data sheets will be printed and used to facilitate note taking and the capture of 
close calls and use errors that are observed during each session. Data collection and 
analysis will include both objective data and subjective data collected via test cases 
that are provided to the user.   
Objective data will include:  
 Participant answers to use scenarios and user documentation assessment test case 
questions assessing user comprehension. 
 Study team assessment by use scenario, based on pre-determined acceptance criteria 
(see sections 11.1 and 11.4). 
Subjective data will include: 
 Participant feedback on ease of use, understandability, and possible improvements. 
 Study team feedback on any issues with participant comprehension or ability to 
complete user tasks. 
In the case that a problem or a use error is observed, the study moderator will probe 
for root cause and record the participant’s assessment of safety and risk acceptability. 
This subjective input of risk acceptability will inform the final assessment and 
conclusion of risk acceptability made by the study personnel during data analysis. 
Both observational data and subjective input will facilitate identification and 
understanding of the root causes of use errors or difficulties that occur. 
As needed, photo and video recordings may be implemented during the study to 
facilitate post-study analysis of user-feedback, use errors, and close calls. 
More than one study observer may record data using the data sheets. As a result, data 
may differ between note-takers. After the session is complete, the observer(s) and 
moderator(s) will meet to discuss any difficulties, use errors, or close calls. Data will 
be compiled in a spreadsheet, and study team may re-assess ratings based on further 
analysis, video review, or discussion (e.g. a “Difficulty” may be re-assessed as a 
“Close Call” after the data sheets are complete). Hand-written data will not require 
amendment based on the re-assessment. In any instances where ratings are modified, a 
comment will be made in the cell in the spreadsheet or a log will be otherwise 
recorded to indicate that each change was agreed upon by the study personnel. 
12.3 REPORT 
A final report will be issued upon completion of testing, allowable in PowerPoint format, 
and will include the following: 
 Report must include observations of task performance and occurrences of use 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
33
errors, close calls, and use problems. 
 Report must include feedback from interviews with test participants regarding 
device use, use errors and problems (as applicable). 
 Report must include description and analysis of all use errors and difficulties that 
could cause harm, root causes of the problems, and implications for additional risk 
elimination or reduction. 
 Report must include testing details such as description of products tested and 
identification numbers. 
 Report must identify staff members who tested and evaluated/inspected samples. 
Test date(s) must be identified. 
 Report must identify any deviation from the protocol, and any anomalies in sample 
testing. Justification and approval of deviation by Carrot Inc. must be documented. 
 Report must include test results section and must identify any changes to the testing 
samples. 
 Report must reference completed data sheets.  All data sheets will be signed and 
dated. 
 Report must be signed by the primary investigator or study team designee. It must 
also be reviewed and approved by the study team. 
12.4 DEVIATIONS FROM PROTOCOL 
To be determined by the study team during usability study. Deviations will be assessed 
through a consensus review of the study team. All deviations will be identified, recorded 
and analyzed in the final report.  
12.5 RECORD RETENTION 
Record keeping is performed in accordance with the SOP Control of Quality Records, 
 
 CONFLICTS OF INTEREST 
The following conflicts of interest are noted: 
 The study sponsor is Carrot Inc.,
 
 The investigator owns equity in Carrot Inc., which is the company that invented and 
owns two of the breath sensors being evaluated in this study.   
 The company (Carrot Inc.) holds patent rights to two of the devices being evaluated in 
the study (the Pivot Breath Sensor and the CO Breath Sensor). 
The investigators hold the following positions: 
 
 Sr. Director of Clinical and Medical Affairs (senior 
management officer) 
 
 Director, Clinical Affairs 
 
 Senior Clinical Research Associate, Clinical Affairs 
These conflicts of interest are mitigated by the following: 
 The risk profile of the study, specifically, that there are no anticipated risks or harms to 
the subject.  No medical decisions are made based on study data. The process of 

Carrot Inc. 
 
 
Document No. 
  
 
CONFIDENTIAL 
34
setting up the Pivot Breath Sensor and providing breath samples is non-invasive, and 
done under the guidance of trained study personnel. Study participants will be able to 
indicate how they are feeling and if they would like to continue or discontinue giving 
breath samples.   
 The role of Carrot Inc. in the study is outlined in the study Informed Consent Form; 
participants will be aware of this role prior to providing informed consent.  
 The aim of the study is to assess the usability of the Pivot Breath Sensor in intended 
users and then use this data to optimize the product as needed. This study will inform 
resultant improvements made to the Carrot Inc. products (the Pivot Breath Sensor). As 
such, it is imperative, and the goal, that the study sponsor uses this data to ensure its 
products are as user-friendly and accurate as possible. 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 LITERATURE REFERENCES 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 DOCUMENT REVISION HISTORY: 
Rev.  A     Original Protocol (Pivot Breath Sensor Usability Study)