Carbon Monoxide Breath Sensor System (COBSS) Performance, Human Factors, and Usability Assessment in a Single Visit
Phase: NA
Conditions: Smoking, Tobacco
Sponsor: David Utley, MD



Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
2
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
3
Device Names 
1) Carbon Monoxide Breath Sensor System (COBSS) 
2) Bedfont Scientific Ltd. Micro+ Smokerlyzer 
Protocol Title 
Carbon Monoxide Breath Sensor System (COBSS) Performance, Human 
Factors, and Usability Study Protocol (
) 
Principal Investigators 
 
Protocol Number 
 
Protocol Version 
 
Protocol Shorten Name 
Performance, Human Factors, and Usability Study Protocol 
Study Design 
Prospective, open label, single center clinical study enrolling 70 subjects. No 
medical decisions will be made related to test results. 
Objective 
Performance:  
 
Non-invasively measure carbon monoxide (CO) levels in the exhaled 
breath of study subjects using two different types of measurement 
devices.   
 
We will assess the correlation between the measured CO levels (in 
parts per million, or ppm) of the study CO breath sensor (Carrot Sense) 
and the predicate CO breath sensor (Bedfont). 
Human Factors and Usability:  
 
Ensure that representative intended users are able to operate the 
COBSS independently. 
 
Validate appropriate mitigations of use related hazards identified in 
risk management documentation. 
 
Uncover previously unforeseen use errors. 
 
Potential Subjects 
70 subjects who self-report smoking 2 or more cigarettes each day. 
 
Inclusion Criteria 
18-80 years of age 
English speaking 
Owns and uses a smartphone 
Willing to sign the Informed Consent Form 
Exclusion Criteria 
Pregnancy 
Recruitment 
Eligible subjects will be identified via clinical research recruiter and 
advertisement at outdoor locations (i.e. flea markets, outside of movie theaters, 
shopping malls, train stations, stores that sell cigarettes), print media (i.e., local 
newspapers, billboards), and web media (i.e., Craigslist, Facebook).   
Study Session 
Single day study collecting 1 paired breath sample from each subject using two 
non-invasive breath sampling devices (COBSS and Bedfont; Performance), as 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
4
well as conducting user tasks, user documentation assessment, subjective 
feedback and rating scales (Human Factors and Usability). 
Performance Variable 
CO concentrations in parts per million (ppm) in exhaled breath from subjects. 
Performance Endpoint 
and Analysis 
CO concentrations from each sensor type will be plotted for comparison.  
Human Factors and 
Usability 
Assessment of intended user performance in use scenarios, user documentation 
assessment, subjective feedback and rating scales.  
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
5
Contents 
1.0 
List of Abbreviations ........................................................................................................... 7 
2.0 
Introduction .......................................................................................................................... 8 
2.1 
Background ....................................................................................................................... 8 
2.2 
Study Rationale ................................................................................................................. 9 
2.3 
Potential Risks and Benefits .............................................................................................. 9 
3.0 
Objectives .......................................................................................................................... 10 
3.1 
Performance ..................................................................................................................... 10 
3.2 
Human Factors and Usability .......................................................................................... 10 
4.0 
References .......................................................................................................................... 10 
5.0 
Definitions.......................................................................................................................... 10 
6.0 
Device Description............................................................................................................. 11 
6.1 
Carbon Monoxide Breath Sensor System (COBSS) ....................................................... 11 
6.2 
MICRO+™ SMOKERLYZER®, predicate device ........................................................ 12 
7.0 
Carbon Monoxide Breath Sensor System (COBSS) .......................................................... 12 
7.1 
Labeling and Packaging .................................................................................................. 12 
7.2 
CO Breath Sensor ............................................................................................................ 13 
7.3 
Breath Sensor Application (Pairing) ............................................................................... 13 
7.4 
Breath Sensor Application (Getting Started) ................................................................... 13 
7.5 
Submitting a Breath Sample (Specific Steps) ................................................................. 13 
7.6 
Breath Sensor Application (CO LOG) ............................................................................ 15 
7.7 
Breath Sensor Application (Syncing with CO Breath Sensor) ........................................ 16 
7.8 
Breath Sensor Application (CO Breath Sensor Settings) ................................................ 16 
7.9 
Breath Sensor Application (Guided Breath Test, or GBT) ............................................. 16 
8.0 
Task Analysis and Identification of Critical Tasks ............................................................ 16 
9.0 
Study Design and Overview .............................................................................................. 18 
9.1 
Study Design ................................................................................................................... 18 
9.2 
Study Endpoints .............................................................................................................. 19 
9.3 
Eligibility Criteria ............................................................................................................ 19 
9.4 
Sample Size and justification .......................................................................................... 20 
9.5 
Performance Study Endpoint Acceptance Criteria .......................................................... 22 
9.6 
Study articles ................................................................................................................... 22 
9.7 
Equipment and Materials ................................................................................................. 22 
9.8 
User Manual and Training ............................................................................................... 23 
9.9 
Test Environment Description ......................................................................................... 23 
9.10 Study Personnel ............................................................................................................... 24 
9.11 IRB Oversight .................................................................................................................. 24 
10.0 Study Procedure and Schedule........................................................................................... 24 
11.0 Technical Approach and Study Design.............................................................................. 28 
11.1 Use Scenarios .................................................................................................................. 29 
11.2 User Documentation Assessment .................................................................................... 35 
11.3 Subjective Feedback and Rating Scales .......................................................................... 41 
11.4 Human Factors and Usability Acceptance Criteria ......................................................... 41 
11.5 Methods for Capturing Use Errors .................................................................................. 42 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
6
12.0 Study Results ..................................................................................................................... 43 
12.1 Data Collection and Confidentiality ................................................................................ 43 
12.2 Analysis ........................................................................................................................... 43 
12.3 Report .............................................................................................................................. 44 
12.4 Deviations From Protocol ............................................................................................... 44 
13.0 
 ............................................................................................................ 45 
14.0 Record Retention ............................................................................................................... 45 
15.0 Literature References ......................................................................................................... 45 
16.0 Document Revision History:.............................................................................................. 46 
 
 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
7
 
1.0 
LIST OF ABBREVIATIONS 
CO 
Carbon Monoxide 
COBSS 
Carbon Monoxide Breath Sensor System 
CRF 
Case Report Form 
BSA 
Breath Sensor Application 
USB 
Universal Serial Bus 
IRB 
Institutional Review Board 
ICF 
Informed Consent Form 
HCP 
Health Care Provider 
ppm 
parts per million 
 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
8
2.0 
INTRODUCTION 
2.1 
BACKGROUND 
The smoking of tobacco products is a leading cause of preventable morbidity and mortality, 
excess health care expenditure, and lost work productivity.  
 
 
 
   
 
 
 
 
 
  
   
 
 
 
  
 
 
 
   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
9
 
 
 
 
 
 
 
We have developed the Carbon Monoxide Breath 
Sensor System (COBSS, Carrot Sense, Redwood City, CA, USA), which consists of a handheld 
breath sensor to measure CO in exhaled breath, with results displayed in a smartphone app. 
COBSS is intended to be used as a personal and mobile product and available ‘over the counter’. 
This study will compare the performance of the COBSS to that of a clinic-based CO breath 
sensor (Micro+Smokelyzer; Bedfont; Kent, UK) in the measurement of CO in exhaled breath, 
and, will assess the human factors and usability of the COBSS. 
2.2 
STUDY RATIONALE 
Performance: We aim to non-invasively measure carbon monoxide (CO) levels in the exhaled 
breath of study subjects using two different types of measurement devices.  We will assess the 
correlation between the measured CO levels (in ppm) of the study CO breath sensor (COBSS; 
Carrot Sense) completed by a lay user with no personal instruction (using labeling only) and the 
predicate CO breath sensor (Micro+Smokelyzer; Bedfont) administered by a person trained to 
instruct users on submitting a sample with the device. 
Human Factors and Usability: We aim to assess whether an untrained lay user group 
(representative of intended users) can operate the device and interpret the results correctly using 
only the instructions that will be provided with the marketed device. 
 
2.3 
POTENTIAL RISKS AND BENEFITS 
2.3.1 Known Potential Risks 
There are no anticipated risks or harms to the subject.  No medical decisions are made based on 
study data. 
2.3.2 Known Potential Benefits 
While we do not anticipate that subjects will receive any benefit from their participation in this 
study, it is possible that some subjects may experience an increased awareness of their smoking 
behavior. 
2.3.3 Risk Benefit Assessment 
Given the non-invasive nature of the breath sampling and the data collection, there are no 
anticipated risks to subject safety.  Study staff will be with subjects and monitoring their use of 
the study devices for the duration of the study. Subjects are not asked to change their smoking 
behavior or lack thereof.  No medical decisions are made based on study data. 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
10
3.0 
OBJECTIVES 
3.1 
PERFORMANCE 
We aim to non-invasively measure carbon monoxide (CO) levels in the exhaled breath of study 
subjects using two different types of measurement devices.  We will assess the correlation 
between the measured CO levels (in ppm) of the study CO breath sensor (COBSS, Carrot Sense) 
and the predicate CO breath sensor (EC50 Micro+™ Smokerlyzer® – Bedfont (K082315)). 
3.2 
HUMAN FACTORS AND USABILITY 
The objectives of the human factors and usability portion of the study are to: 
 Ensure that representative intended users are able to operate the COBSS 
independently, focusing on the following: 
o Proper breath sample collection using the CO Breath Sensor by a user  
o User comprehension of how to navigate the Breath Sensor Application (BSA) 
o User interpretation and comprehension of their results 
 Validate appropriate mitigations of use related hazards identified in risk 
management documentation 
 Uncover previously unforeseen use errors 
 
Overall, this human factors and usability validation testing is conducted to demonstrate that the 
COBSS can be used by the intended users without serious use errors or problems, for the 
intended uses and under the expected use conditions. 
4.0 
REFERENCES 
 FDA Guidance Document Entitled, “Guidance for Industry and Food and Drug 
Administration Staff: Applying Human Factors and Usability Engineering to Medical 
Devices,” issued on February 3, 2016 
 FDA Guidance Document Entitled, “Design Considerations for Devices Intended for 
Home Use, Guidance for Industry and Food and Drug Administration Staff” issued on 
November 24, 2014 
 ISO/IEC 62366-1:2015, Medical devices – Application of usability engineering to 
medical devices. Geneva, International Electro technical Commission. 
 AAMI/ANSI HE75:2009, Human Factors Engineering – Design of Medical Devices. 
 https://www.bluetooth.com/what-is-bluetooth-technology/bluetooth-technology-basics 
5.0 
DEFINITIONS 
Bluetooth Low Energy (BLE): a global wireless communication standard that connects devices 
together over a certain distance.  
Breath Sensor Application (BSA): The smart phone application (“app”) that is part of the subject 
system. The BSA displays exhaled breath carbon monoxide value to the user. The BSA is paired 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
11
to the CO Breath Sensor via Bluetooth Low Energy. 
Carbon Monoxide Breath Sensor System (COBSS): the name of the subject system. This system 
comprises the CO Breath Sensor and the Breath Sensor Application (BSA), and is intended for 
single-user use in smoking cessation programs as an educational and motivational tool showing 
how breath carbon monoxide levels are affected by smoking behavior. 
Close Call (a.k.a. “Near-Miss”): Instances in which a user has difficulty or makes a use error that 
could result in harm, but the user takes an action to “recover” and prevents the harm from 
occurring (FDA Guidance 2016). 
CO Breath Sensor: a component of the subject system. It is a personal mobile breath sensor that 
is capable of measuring the level of carbon monoxide (CO) in exhaled breath. The CO Breath 
Sensor is a portable, battery-powered, rechargeable device slightly smaller than the palm of 
one’s hand. This device pairs to the Breath Sensor Application (BSA) on a smartphone via 
Bluetooth Low Energy. The user submits a breath sample by exhaling (blowing) into the 
personal mobile breath sensor via a provided straw. 
Critical Task: A user task which, if performed incorrectly or not performed at all, would or could 
cause serious harm to the patient, user, or instrument, where harm is defined to include 
compromised medical care. 
Guided Breath Test (GBT): a function in the BSA wherein the BSA prompts the user through the 
steps of submitting a breath test. Based on the user’s adherence to the proper technique, the user 
is provided with a pass or fail.  They may repeat this GBT as often as they like if additional 
education and reinforcement of the technique is desired. 
Objective Data: Data collected through direct observation. 
Subjective Data: Data collected through user feedback via comments, ratings, and assessments 
provided by participants. 
Task: Action or set of actions performed by a user to achieve a specific goal. 
Use Error: User action or lack of action that was different from that expected by the 
manufacturer and caused a result that (1) was different from the result expected by the user and 
(2) was not caused solely by device failure and (3) did or could result in harm. (Section 3.9, FDA 
Guidance 2016). 
User: Person who interacts with (i.e., operates or handles) the COBSS. 
6.0 
DEVICE DESCRIPTION 
6.1 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
The Carbon Monoxide (CO) Breath Sensor System (COBSS) comprises a personal mobile breath 
sensor (CO Breath Sensor) capable of measuring the level of CO in exhaled breath and a Breath 
Sensor App (BSA) for smartphones which displays the exhaled breath CO value to the user.  The 
BSA is compatible with iOS based devices (iPhone 5 and above, operating system iOS 9.0 and 
above) and Android based devices that support Bluetooth Low Energy (Android 4.4 and above, 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
13
7.2 
CO BREATH SENSOR 
The CO Breath Sensor (Figure 1) is portable, battery-powered, and small enough to be 
conveniently carried by the user throughout the day (pocket, purse, backpack).  The CO Breath 
Sensor is rechargeable using a micro-USB cable.  The LED pulses orange while charging and 
then solid green when fully charged. 
The CO Breath Sensor pairs to the BSA on the smartphone via low-energy Bluetooth.  The user 
submits a breath sample by exhaling (blowing) into the CO Breath Sensor via a straw. The CO 
Breath Sensor comes packaged with a straw already inserted in the inlet. The straw, made of 
polypropylene, is reusable, and can be cleaned with soap and water or replaced as necessary. In 
addition, there is a pack of 7 straws (for 8 straws in total) included in the CO Breath Sensor 
package.  
7.3 
BREATH SENSOR APPLICATION (PAIRING) 
The BSA, Quick Start Guide and User Manual guide the first-time user through the process of 
pairing their smartphone to the CO Breath Sensor.  Once paired, the CO Breath Sensor 
communicates exclusively with the user’s phone and is invisible to other smartphones and other 
Bluetooth Low Energy wireless devices.  
7.4 
BREATH SENSOR APPLICATION (GETTING STARTED) 
After pairing, the BSA displays a Getting Started screen containing a brief video tutorial 
on how to submit a breath sample. This video depicts the user steps to 
submit a breath sample, as described in the User Manual. The BSA software automatically 
advances to and requires the user to complete the video tutorial after pairing. The video tutorial 
must be completed to advance further in the set-up and use of the BSA (system requirement). 
The video tutorial can be paused and played again. It is also accessible to the user at any time 
through the Settings screen in the BSA. 
Next, a Guided Breath Test (GBT) function is used by the user wherein the BSA prompts the 
user through the steps of submitting a breath test (interactive version of the user steps disclosed 
in the User Manual).  Based on the user’s adherence to the proper technique, the GBT provides 
the user with a pass or fail.  Pass or fail status is communicated to the user by the CO Breath 
Sensor LED (flashing green indicates pass and flashing red indicates fail), as well as the screen 
on the BSA, which includes a smiley face, flashing green dot and descriptive text for pass, or, a 
sad face, flashing red dot and descriptive text for fail 
 
 The BSA software requires the user to complete three GBTs before advancing further in 
the set-up and use of the BSA.  
The user may repeat the GBT as often as they like if additional education and reinforcement of 
the technique is desired. The GBT is also accessible to the user at any time through the device 
button in the upper left corner of the CO Log screen in the BSA. 
7.5 
SUBMITTING A BREATH SAMPLE (SPECIFIC STEPS) 
Submitting a breath sample involves the following steps (instructions provided in the User 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
14
Manual, Reference Card, video tutorial, Guided Breath Test and Quick Start Guide).  
1. If not already in place, user places provided straw in the CO Breath Sensor inlet 
2. User presses and holds (~2 seconds) the (only) button on the CO Breath Sensor until the 
device beeps once and the LED flashes blue 
3. User takes a deep breath in and holds their breath  
4. After about 10 seconds, the device beeps 3 times and the flashing blue LED converts to 
solid blue LED 
5. The user then gently blows into the straw for at least 6 seconds (goal is 12 seconds) or 
until they exhaust their entire breath (Figure 2) 
6. The user then removes straw from their lips 
7. After a few seconds, the solid blue LED switches to: 
a. 
Blinking green LED indicates adequate breath sample, data stored, or, 
b. 
Blinking red LED indicates inadequate breath sample, please repeat 
A properly submitted breath sample is defined, by the system, as one that initiates after the third 
beep prompt and is of at least 6 seconds duration.  Breath sensor hardware and firmware is 
capable of detecting when a breath sample submission starts and stops.  
    
              
 
      
   Figure 1. CO Breath Sensor  
       
      Figure 2.  Submitting a Breath Sample 
 
A properly submitted breath sample is indicated to the user through the CO Breath Sensor, with 
the LED flashing green for a proper submission and red for an improper submission. In addition, 
the CO Log screen in the BSA provides the CO (ppm) value for properly submitted breath 
samples, and provides a descriptive error message for improperly submitted breath samples (i.e. 
“Error: breath sample too late”). 
The computation of concentration of CO (ppm) is performed within the CO Breath Sensor 
firmware and the data (CO ppm concentration, time and date) are stored in the CO Breath Sensor 
memory.  The measured values are transmitted via Bluetooth to the smartphone (when paired, in-
range, and after synchronization) and are displayed on the primary screen of the Breath Sensor 
Application (CO Log) (Figure 3).  The data is stored locally on the smartphone. 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
15
7.6 
BREATH SENSOR APPLICATION (CO LOG) 
The primary screen for the CO Breath Sensor is the CO Log screen in the Breath Sensor 
Application.  In the CO Log, the most recent exhaled breath CO value in ppm is displayed at the 
top of the screen.  The user can view previous values by scrolling within the log.   
Values are color coded to align with the predicate device and scientific literature. 
 
 
Figure 3. – Color coded bars according to CO breath levels (from User Manual) 
Furthermore, the length of the bar is associated with the ppm value to graphically show the user 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
16
their relative levels of exhaled breath CO throughout the day and between days. 
Maximum bar width = 50 ppm 
Values above 50 ppm are annotated with a white arrow. 
7.7 
BREATH SENSOR APPLICATION (SYNCING WITH CO BREATH SENSOR) 
Breath sample CO values, date and time are stored on the CO Breath Sensor until a sync occurs 
with the Breath Sensor App (BSA).  Syncing occurs automatically when the app is open in the 
foreground or background.  When a user submits a breath sample on the CO Breath Sensor, they 
will be able to view the results in the CO Log within one minute.  If the most recent values are 
not in the CO Log, the time of last sync can be viewed in Settings and a manual sync can be 
performed in Utilities. 
7.8 
BREATH SENSOR APPLICATION (CO BREATH SENSOR SETTINGS) 
In the BSA, when the user navigates to the Settings page by tapping on the gear icon in the upper 
right corner, the user sees device status (connected, not connected, syncing), last sync, and 
battery status. 
A user may choose to use the Reminders function to help them remember to submit breath 
samples throughout their day.  When reminders are toggled from off to on, the user may choose 
the time of the first and last reminder of the day, reminder time interval, type of reminder 
(vibration and/or beep), and the intensity (volume) of the reminder where applicable.  
Additionally, the user may set the volume of the breath test beeps described in the User Manual. 
A utilities menu includes options to manually synchronize the CO Breath Sensor with the 
smartphone (in the event that auto sync is not working), ring the CO Breath Sensor (if 
misplaced), and forget the sensor (unpair). 
Settings also provides access to the video tutorial on how to submit a breath sample, for those 
users who wish to obtain additional reinforcement and learning regarding breath sample 
submission. 
7.9 
BREATH SENSOR APPLICATION (GUIDED BREATH TEST, OR GBT) 
In the BSA, the user may access the Guided Breath Test at any time using the device icon in the 
upper left corner. After completing device set-up and training (required 3 GBTs), the user may 
use the GBT for any and all subsequent breath samples. 
 
8.0 
TASK ANALYSIS AND IDENTIFICATION OF CRITICAL TASKS  
For the human factors and usability portion of the study, a task analysis was performed to 
identify the tasks associated with the use of the COBSS. The task analysis is depicted in Figure 
4.  

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
19
9.2 
STUDY ENDPOINTS 
9.2.1 Human Factors 
Assessment of intended user performance in use scenarios, user documentation assessment, 
subjective feedback and rating scales. 
9.2.2 Performance 
Comparison of paired breath sample values from the study and predicate devices. 
9.3 
ELIGIBILITY CRITERIA 
The intended user population for the CO Breath Sensor System are lay users who are smokers, 
ages 18 and above, capable of using a smartphone to track/monitor health-related data. The 
intended users range in age, degree of smoking, mental and physical capabilities, health literacy, 
experience using other CO analyzers or home use medical devices, and experience using 
smartphones.  
The user population may be further classified into two age-based subgroups, 18-49yr and the 50-
80yr age groups. The 18-49yr olds, in general, are believed to be more tech savvy and have 
better physical capabilities compared to the 50-80yr olds.  
Study participants will consist of individuals who represent the intended COBSS user profile, 
current smokers. Participant inclusion criteria include all of the following: 
 18+ years of age 
 Current daily cigarette smokers (at least 2 cigarettes per day with target distribution of 
individuals who smoke: 2-5 cigarettes per day, 6-10 cigarettes per day, 11-15 cigarettes 
per day and 16+ cigarettes per day)  
 Resident of the United States 
 Able to read and comprehend English 
 Target a mix of male and female participants 
 Target 35 participants in each of the two subgroups, the 18-49yr and the 50-80yr age 
groups, with at least 15 individuals in each group  
 Currently own and use an Apple, Inc. iPhone iOS smartphone or an Android smartphone 
for at least one month (such that they are familiar with basic operations of the mobile 
platform) 
 Willing to sign the Informed Consent Form 
Exclusion Criteria 
 Prior experience using the COBSS 
 Not interested in quitting smoking 
 Pregnancy  
Study participants will be recruited in the United States. Each potential participant will be asked 
a standard series of questions to confirm study eligibility and to capture demographic data. 
Eligible participants who meet the inclusion/exclusion criteria will be entered into the study. An 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
20
Informed Consent Form 
 will be signed prior to participation. 
9.4 
SAMPLE SIZE AND JUSTIFICATION 
This study will include 70 subjects. This sample size is expected to provide adequate data to 
compare CO measurements between the devices. Sample size needed was determined as 
described below. 
Data from a formative study was used to develop the statistical methodology for the present 
study.  We conducted a feasibility study titled 
, with approval by the Institutional Review 
Board 
  The study was conducted with signed informed 
consent.  
 
  
 
  
 
 
 
 
  
 
 
Figure 5. Plot of paired CO (ppm) levels for Carrot Sense (CS) vs. Bedfont (Predicate) 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
22
In a secondary analysis of this pilot data, 
 
 
This resulted in no change to our power calculation to follow. 
9.5 
PERFORMANCE STUDY ENDPOINT ACCEPTANCE CRITERIA 
Based on the null hypothesis that the Pearson correlation coefficient of Bedfont and CS 
 
passing criterion is refuting the null 
hypothesis (confirming the alternative hypothesis) with a power of 
 or higher assuming a 
 alpha level.  
9.6 
STUDY ARTICLES 
 Carbon Monoxide Breath Sensor System (COBSS) in representative packaging that 
includes: 
 CO Breath Sensor  
 Micro-USB charger  
 Pack of 7 straws for use with the CO Breath Sensor  
 User Manual 
 
 Quick Start Guide 
 
 Reference Card 
 
 The BSA, installed on an Apple, Inc. iOS and an Android smartphone  
 Predicate device (Micro+™ Smokerlyzer®, Bedfont Scientific Ltd; Harrietsham, UK; 
510K number: K082315] 
In addition, the testing room will be equipped with electrical outlets and a charging cube.  
9.7 
EQUIPMENT AND MATERIALS 
 Apple, Inc. iPhone with the BSA installed on it 
 An Android smartphone with the BSA installed on it 
 CO Log Print Out 
 Printed prompt cards for the participant to reference during each task 
 Hydrogen breath sensor (Gastro+ Gastrolyzer ®, Bedfont Scientific Ltd.) 
 Timer or clock 
 Video recording equipment 
 Camera 
 Post-it notes 
 Quick Start Guide for Observer/ Moderator 
 User Manual for Observer/ Moderator 
 Reference Card for Observer/Moderator 
A full list of equipment and materials used during the study will be recorded in the report. 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
23
9.8 
USER MANUAL AND TRAINING 
The COBSS was designed to be as intuitive as possible, such that users are able to understand 
how to use the product on their own. No in-person training is formally offered to COBSS users. 
However, the User Manual, Quick Start Guide and Reference Card are included with the 
purchase of the device. Additionally, users are required by the COBSS software design to 
complete the COBSS video tutorial and 3 Guided Breath Tests. The video tutorial and Guided 
Breath Test provide in-app user training and reinforce the procedure of submitting a CO breath 
sample using the device. 
Since in-person training will not be formally offered to all users who purchase the COBSS, in-
person training will not be included in this study. Study participants will have access to the User 
Manual, Quick Start Guide and Reference Card in the device package, as will be the case for the 
marketed product. Any instance of participants referring to the User Manual, Quick Start Guide 
or Reference Card during the study will be recorded.  
Additionally, the video tutorial and Guided Breath Test, offered by the COBSS user interface, as 
part of BSA start up, will help participants familiarize themselves with the device and understand 
how to use it. The BSA software requires completion of both the video tutorial and 3 Guided 
Breath Tests to advance through the COBSS set-up and start using the device.  
During the usability validation, the participants will have the User Manual, Quick Start Guide, 
Reference Card, video tutorial and Guided Breath Test available to them at any point during the 
study. This mimics real-world use. However, they will not be asked to read instructions (except 
during the knowledge questions covered in the User Documentation Assessment – see Section 
11.2) or use any of the forms of training available. At the beginning of the study, participants 
will have the opportunity to familiarize themselves with the product and instructions. 
9.9 
TEST ENVIRONMENT DESCRIPTION 
9.9.1 Intended Use Environment 
The COBSS has been designed as an over-the-counter device. The intended use environment is 
primarily the user’s home, work, school, or similar environment.  
The home environment can be characterized as a familiar environment with relatively dim (~150 
lux) to relatively bright (~1000 lux) lighting; low to medium noise, medium level of activities, 
and some distraction (family, TV, radio).  
 The work/school environment can be characterized as a familiar environment with low to 
medium noise, well illuminated, low level of activity, low level of traffic, and limited 
people interaction. 
The user submits breath samples at their discretion, and will be able to factor environmental 
conditions (such as noise, light) into the decision of when to use the CO Breath Sensor. 
9.9.2 Test Environment 
The conditions under which simulated use testing is to be conducted will be sufficiently realistic 
to the expected use environment to enable the results of the testing to be generalized to anticipate 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
24
actual use. Environmental factors believed to affect user performance, such as lighting, noise, 
and space constraints, will be simulated. 
The study will be performed indoors to assess use in the anticipated most common use 
environment. The study will be conducted in a conference room or office space, representative of 
the intended use environment, where the user is able to complete the assessment.  
9.9.3 Dates testing will be performed:  
Testing dates are anticipated to be in 
 Actual testing dates will be 
recorded in the study report. 
9.9.4 Location testing will be performed:  
The location of testing will 
 
. For example, space may be rented to be nearer to test 
subjects.  
9.10 
STUDY PERSONNEL 
The following roles will be fulfilled by Carrot Sense employees and consultants during the study: 
 Study Moderator – Responsible for moderating the study, delivering instructions (as 
needed) and ensuring that participants progress from one use scenario to the next, and 
asking questions to probe about observed use errors and close calls. 
 Study Facilitator – Responsible for organizing the simulated environment, study articles, 
equipment, and required documentation (e.g. informed consent form, non-disclosure 
agreements).  
 Study Observer(s) – Responsible for taking notes during the study sessions and 
documenting key events of the study as they occur.  
 Technical Support – Responsible for troubleshooting system issues and recording bugs in 
the case there is a system issue during the study session. 
Individuals fulfilling the roles will be recorded in the final Usability Validation Study Report. 
9.11 
IRB OVERSIGHT 
This study will be conducted under IRB oversight.  
10.0 
STUDY PROCEDURE AND SCHEDULE 
For an overview of study session flow, see Figure 7. At the beginning of the study session, each 
subject will have ample time to read the Informed Consent Form (ICF) and the opportunity to 
ask questions. The subject will sign the ICF before participating in this study. Once signing the 
ICF, the subject will be given a unique identifier to protect subject privacy. Subjects may choose 
to exit the study at any time.  They are not required to complete the study if they choose to stop. 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
25
Study personnel will collect information about the patient including 
 Age 
 Gender  
 Eye sight 
 Ethnicity  
 Prior experience with Carbon Monoxide breath sensors  
 Desire to quit smoking 
 Smartphone experience 
 When did you start smoking, how old were you? 
 Number of cigarettes typically smoked per day 
 Smoking history in the past 24 hours  
o Number of cigarettes smoked 
o Time of last cigarette 
o Use of any combustible, inhaled tobacco product other than cigarettes 
(pipes, cigars, cigarillos, etc.), any electronic cigarette product (e-cigs, 
vape, vapor), and any other inhaled combustible material.  
Subject will self-train on how to use the study device using only the available study CO breath 
sensor device labeling, including product packaging, User Manual, Quick Start Guide, Reference 
Card, a technical support line provided in User Manual, and Breath Sensor App resources (video 
tutorial, guided breath test (GBT), Settings).  These labeling materials will all be included in the 
future marketed product. To mimic real-world use, these labeling materials will be available to 
study subject at any point during the study. However, the study subject will not be asked to read 
instructions or use any of the forms of training available.  Since in-person training will not be 
formally offered to all users who purchase the study device, in-person training on the study 
device will not be included in this study. Study personnel will refrain from answering questions 
on how to use the study device during the study.  
As part of the process of setting up and learning to use the study device, the device software 
requires the user to complete the video tutorial and the GBT (the latter must be successfully 
passed three times); this is the case for the marketed device and therefore will be the case in this 
study as well.  
Once the study subject has completed the system-required set-up and self-training on the study 
device, they will be asked if they feel ready to proceed with submitting another breath sample 
(continuing to the paired breath samples).  If a subject indicates they do not feel ready, they will 
be asked to do what they would normally do at home to prepare to use the product and to let the 
test administrator know when they are ready to submit another sample. 
When the subject indicates readiness to provide a breath sample, he/she will be instructed to 
proceed to provide one breath sample using the study device. After providing this breath sample, 
the subject will be asked to interpret his/her breath sample result followed by the first set of 
subjective questions. 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
26
Within 3-5-minutes of the first breath sample and after completing the questions addressing 
interpretation of breath sample results and the 1st set of subjective questions, a health care 
professional (or test administrator) will guide subject through the submission of the second 
breath sample of the paired breath sample using the predicate device. 
CO levels decrease gradually in the breath with time (half-life 4-5 hours), so it is not expected 
that the second sample in each paired sample will be substantially lower than the first sample.  
A third breath sample to measure hydrogen (Gastro+ Gastrolyzer ®, Bedfont Scientific Ltd.) will 
be completed 
.  
Upon completion of the paired breath sample and hydrogen breath sample, subjects complete the 
2nd set of subjective questions and then do the user documentation assessment. During this time, 
ease of use and clarity of content are assessed for the product labeling, with focus on the User 
Manual. This is followed by rating scales. 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
30
 Tasks: 
o Open the Carbon Monoxide Breath Sensor packaging and remove contents 
o Open the installed BSA on the smartphone 
o Pair Breath Sensor with the smartphone 
o Complete 
 tutorial 
o Complete 
 
 Prompt: 
How did that go for you? 
 
11.1.3 Use Scenario 3: Interpret your results – Part 1 
 Prompt: 
Please walk me through your test results. What does each result mean?  
 
 
 
 Tasks: 
o Access the CO Log  
o Understand results from the breath tests 
Note: This Use Scenario will be continued for interpreting results for Use Scenario 5. (See 
section 11.1.5 below.) 
Expected Response: 
For green value:  
 
 
 
 
Or, 
 
  
Or, 
 
 
 
For orange value:  
 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
33
o Submit a breath sample  
 Study observer confirms breath sample taken 3 to 5 minutes from last Guided Breath Test 
breath sample, and records the time of the sample submission. 
 Prompt: 
How did that go for you? 
(Next prompt is a Continuation from Use Scenario 3 above) 
 Prompt: 
Please walk me through your test result. What does this result mean?  
 
 
 
 Tasks: 
o Access the CO Log  
o Understand result from the breath test 
Expected response: See 11.1.3 response 
 
11.1.6 Subjective Feedback Part 1 
 Prompt: 
Were there any points during this portion of the study that you experienced difficulty or had 
concerns about the product? 
 
 Prompt: 
Did you experience any moments of hesitation while using the product? 
 
 Prompt: 
Do you have any safety concerns regarding the product or labeling? 
 
 
11.1.7 Comparative Breath Sample with the Predicate Device followed by Breath 
Sample to Measure Hydrogen  
Note this is NOT a Use Scenario but is included here to thoroughly represent the flow of this 
portion of the study. 
  
 Prompt: 
Now we are going to submit a sample using another Carbon Monoxide Sensor 
 
 Subject provides a breath sample using the predicate CO breath sensor (Bedfont), submitted 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
34
with guidance by a trained test administrator.  
 
 Study observer confirms breath sample taken 3 to 5 minutes after the previous breath sample 
(from section 11.1.5) and records the time of the sample submission. 
 The Bedfont CO breath sensor result will be documented. 
 Bedfont CO breath sensor result and COBSS result will be photographed side by side with 
participant number. 
 
 Prompt: 
 
 
 
 
 
  
 
 
 
 
 
 
 
11.1.8 Use Scenario 6: Charge the Carbon Monoxide Breath Sensor  
 Prompt: 
Please proceed to charge the Carbon Monoxide breath sensor. 
 Tasks: 
o Connect the micro-USB cable 
o Charge the device 
 
11.1.9  Use Scenario 7: Set reminders for future breath tests 
 Prompt: 
Please set reminders to perform breath tests every 2 hours from 8am to 10pm. 
 Tasks: 
o Access the BSA Settings Page 
o Set time limits and frequency for reminders 
 
11.1.10 Use Scenario 8: Access the Frequently Asked Questions Page in the Application 
 Prompt: 
Please access the Carbon Monoxide Breath Sensor Frequently Asked Questions page in the 
application.  

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
35
 Tasks: 
o Press Setting button 
o Press Frequently Asked Questions on Screen 
 
11.1.11 Subjective Feedback Part 2 
 Prompt:  
 
 
 
 
 
.   
 
Upon completion of the use scenarios and comparative performance paired breath samples, a 
user documentation assessment session and a subjective feedback session will be conducted to 
better inform the safety and efficacy of the COBSS.  
11.2 
USER DOCUMENTATION ASSESSMENT 
Participants will never be asked to refer to the User Manual, Quick Start Guide, or Reference 
Card during simulated use scenarios. Any instance of participants referring to the instructions in 
the User Manual, Quick Start Guide, or Reference Card during the simulated use scenarios will 
be recorded and assessed. 
After completion of the use scenarios and comparative performance paired breath samples, the 
user documentation (User Manual, Quick Start Guide, and Reference Card) will be tested for 
ease of use and clarity of content. Each user documentation assessment session will have test 
cases to assess the findability and the interpretation of knowledge task data found in the User 
Documentation. Participants will be asked to find and/or interpret 12 pieces of essential 
information. Participants will demonstrate understanding through verbal explanation. The 
questions and answers for the user documentation assessment are found in Table 3. 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
39
  
 
 
  
 
Prompt:  
Please read these pages (Provide User Manual with the Warnings, Precautions, Notes pages 
opened up). I am going to ask you some questions about these pages. 
 
KT 6 
 
 
 
  
 
 
 
 
  
 
  
 
 
 
 
  
  
 
 
 
 
 
 
 
 
 
KT 7 
 
 
 
 
 
 
). 
 
  
 
 
  
  
 
 
KT 8 
 
 
 
 
 
 
 
 
  
 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
41
expected 
answers) 
 
 
 
 
 
KT 12 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
11.3 
SUBJECTIVE FEEDBACK AND RATING SCALES 
In addition to the subjective feedback obtained periodically throughout the study session, 
subjective feedback may be collected during the use scenarios and user documentation 
assessment to inform root cause analysis. Following the user documentation assessment, the 
participants will be subjected to rating scale questions to rate the ease of use of the device, the 
clarity of the user documentation and their ability to understand and interpret the test results.  
11.4 
HUMAN FACTORS AND USABILITY ACCEPTANCE CRITERIA 
Each use scenario and user documentation assessment test case will be attributed one of the 
following assessments for task success: 
 Successful: User is able to complete the task safely and effectively. 
 Difficulty: User is able to complete the task safely and effectively, but has significant 
hesitation or challenges while completing the task. 
 Close Call: User is able to complete the task safely and effectively, but performs it in a 
way that presents the potential for user harm (i.e. near-miss). 
 Unsuccessful (Use Error): User is unable to independently complete the task or does not 
complete the task safely and effectively. Includes instances in which users are unable to 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
42
complete the task and as a result require study moderator assistance are assessed as 
unsuccessful. 
 Did Not Perform: User does not complete the task for safe and effective use of the 
device.  
Any instance of study moderator assistance will be noted and assessed depending upon the extent 
of the assistance provided.  
The COBSS Use FMEA (
 governs Acceptance Criteria for the COBSS Usability 
Validation Study. Each hazard-related use scenario observed during usability validation testing 
will be documented and assessed for risk acceptability as defined in the Use FMEA.  
In the case that a hazard-related use scenario is concluded to have an acceptable risk profile per 
the Use FMEA, its associated test case is considered to “pass.” In the case that a hazard-related 
use scenario is concluded to have an unacceptable risk profile per the Use FMEA, its associated 
test case is considered to “fail.” In the case of a failed test case, further risk mitigation and 
usability evaluation would be required. In the case that further risk mitigation is not possible, a 
risk-benefit analysis would be documented.  
11.5 
METHODS FOR CAPTURING USE ERRORS 
During each study session, observer(s), and moderator(s) will observe for difficulties, use errors 
and close calls that could inform modifications to the COBSS Use FMEA. The study moderator 
will also ask probing questions, at the end of the session, to reveal close calls that were not 
conveyed through observation. All use errors and difficulties will be followed with questions to 
identify how and why the participant believes they occurred. New, unforeseen use-related 
hazards will be assessed for risk acceptability and added to the COBSS Use FMEA.  
The study team will include subjective assessments by participants for any observed use-related 
issues that could potentially result in harm to the user.  
Users will perform tasks independently and in a natural manner, without guidance, coaching, 
praise, or critique from the test facilitator or moderator. Users will not be allowed a “second 
chance” to perform a task correctly after a failure. That is, if an irreversible safety or efficacy 
issue is observed in which a user would not normally have a “second chance” to resolve the 
issue, then any subsequent attempts to compete the task will serve as input only for continuous 
usability improvement purposes. 
The study team will ensure participant feedback is documented throughout the session, as well as 
any protocol deviations. Prior to the end of the test session, the study team will review all data 
sheets for completeness and pursue additional information or clarification as needed. The study 
team will then inform participants when the test session is complete. 
The study team will scan the data sheets and save them to a secure study file. All members of the 
study team will be familiar with the device operation to be qualified to administer the test and 
assess participant performance with the device. The study team will write the summary report. 
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
43
12.0 
STUDY RESULTS 
12.1 
DATA COLLECTION AND CONFIDENTIALITY 
Subjects will be assigned a unique subject ID which will be used for data collection. Data 
collection will take place on paper or electronic case report forms (CRF) completed by the study 
team and study subjects. CRFs will be reviewed by the study team prior to the end of the study 
session to ensure completeness and then stored in a secure filing cabinet at the end of the day 
with access limited to necessary study personnel. Study data will be kept in a secure database by 
the investigator. This database will be accessible only by necessary study personnel. The 
database will not contain personal identifying information; all subject data in the database will be 
associated with study identification numbers. The data will be kept for 5 years. 
12.2 
ANALYSIS 
The study will utilize paper-based data entry procedures, subject to audit per Carrot Sense 
standard operating procedures.  
12.2.1  Human Factors and Usability  
The data sheets will be printed and used to facilitate note taking and the capture of close calls 
and use errors that are observed during each session. Data collection and analysis will include 
both objective data and subjective data collected via test cases that are provided to the user.   
Objective data will include:  
 Participant answers to use scenarios and user documentation assessment test case 
questions assessing user comprehension. 
 Study team assessment by use scenario, based on pre-determined acceptance criteria (see 
sections 11.1 and 11.4) 
Subjective data will include: 
 Participant feedback on ease of use, understandability, and possible improvements. 
 Study team feedback on any issues with participant comprehension or ability to complete 
user tasks. 
In the case that a problem or a use error is observed, the study moderator will probe for root 
cause and record the participant’s assessment of safety and risk acceptability. This subjective 
input of risk acceptability will inform the final assessment and conclusion of risk acceptability 
made by the study personnel during data analysis. Both observational data and subjective input 
will facilitate identification and understanding of the root causes of use errors or difficulties that 
occur. 
As needed, photo and video recordings may be implemented during the study to facilitate post-
study analysis of user-feedback, use errors, and close calls. 
More than one study observer may record data using the data sheets. As a result, data may differ 
between note-takers. After the session is complete, the observer(s) and moderator(s) will meet to 
discuss any difficulties, use errors, or close calls. Data will be compiled in a spreadsheet, and 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
44
study team may re-assess ratings based on further analysis, video review, or discussion (e.g. a 
“Difficulty” may be re-assessed as a “Close Call” after the data sheets are complete). Hand-
written data will not require amendment based on the re-assessment. In any instances where 
ratings are modified, a comment will be made in the cell in the spreadsheet or a log will be 
otherwise recorded to indicate that each change was agreed upon by the study personnel. 
12.2.2 Performance 
See Sections 9.4 and 9.5 for statistical methodology. A paired breath sample is defined as subject 
submitting a breath sample unassisted using the study device, waiting 3-5 minutes, and 
submitting a breath sample assisted by a health care professional (or test administrator) using the 
predicate device. 
A final report will be created to capture the results and analyses of the findings.  
12.3 
REPORT 
A final report will be issued upon completion of testing and will include the following: 
 Report must include observations of task performance and occurrences of use errors, 
close calls, and use problems. 
 Report must include feedback from interviews with test participants regarding device use, 
use errors and problems (as applicable). 
 Report must include description and analysis of all use errors and difficulties that could 
cause harm, root causes of the problems, and implications for additional risk elimination 
or reduction. 
 Report must include testing details such as description of products tested and 
identification numbers. 
 Report must identify staff members who tested and evaluated/inspected samples. Test 
date(s) must be identified. 
 Report must identify any deviation from the protocol, and any anomalies in sample 
testing. Justification and approval of deviation by Carrot Sense must be documented. 
 Report must include test results section and must identify any changes to the testing 
samples. 
 Report must reference completed data sheets.  All data sheets will be signed and dated. 
 Report must be signed by the primary investigator or study team designee. It must also be 
reviewed and approved by the study team.  
12.4 
DEVIATIONS FROM PROTOCOL 
To be determined by the study team during usability study. Deviations will be assessed through a 
consensus review of the study team. All deviations will be identified, recorded and analyzed in 
the final report.  
 
 

Carrot Sense, Inc. 
 
Document No. 
 
 
CARBON MONOXIDE BREATH SENSOR SYSTEM (COBSS) 
PERFORMANCE, HUMAN FACTORS, AND USABILITY STUDY PROTOCOL 
 
CONFIDENTIAL 
 
 
 
 
 
 
 
 
45
13.0 
 
  
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
14.0 
RECORD RETENTION 
Record keeping is performed in accordance with the SOP Control of Quality Records, SOP-
1068. 
15.0 
LITERATURE REFERENCES