The Impact of Cigarette Nicotine Content, E-Cigarette Nicotine Content and E-Cigarette Flavoring on Smoking Behavior in an Electronic Tobacco Marketplace
Phase: NA
Conditions: Tobacco Use
Sponsor: Wake Forest University Health Sciences

 
 
Page 1 of 23 
 
PROJECT 2: EXPERIMENTAL MARKETPLACE, FLAVORS, AND NICOTINE CONTENT  
 
Principal Investigator: Eric Donny, PhD  
Co-Investigator: Rachel Denlinger-Apte, PhD MPH 
Study Coordinators: Darcy Lockhart (former name Amanda Laverdiere), Kayla Hicks, Cassidy 
White, Bailey Robinson 
 
Sponsor: NIH/NIDA 
 
BACKGROUND, RATIONALE, AND CONTEXT 
The Family Smoking Prevention and Tobacco Control Act (FSPTCA) passed in 2009 provides the 
FDA with the authority to regulate tobacco products.  One provision in this legislative act 
empowers the FDA to set limits on constituents in tobacco products, including nicotine.  Such a 
measure has the potential to reduce the chance of individuals experimenting with smoking from 
becoming dependent and enable current smokers to quit when they are motivated to do so.  
Although the proposal to reduce nicotine in cigarettes has been met with skepticism by some 
because of concerns over compensatory smoking behavior and the potential emergence of a 
black market1,2,3,4,5, this policy measure was considered to be technically feasible by the American 
Medical Association and the British Medical6,7, by tobacco control researchers, policymakers and 
governmental officials who were convened in a meeting on nicotine regulation8, and by the World 
Health Organization Study Group on Tobacco Product Regulation9. Most importantly, in July 
2017, FDA Commissioner Gottlieb announced he was “directing our Center for Tobacco Products 
to develop a comprehensive nicotine regulatory plan premised on the need to confront and alter 
cigarette addiction.” This announcement was followed on March 16, 2018 by the release of the 
Advance Notice of Proposed Rulemaking entitled “Tobacco Product Standard for Nicotine Level 
of Combusted Cigarettes” (ID: FDA-2017-N-6189-0001) “to obtain information for consideration 
in developing a tobacco product standard to set the maximum nicotine level for cigarettes.”  
 
Effects of Very Low Nicotine Content Cigarettes 
Studies of very low nicotine content (VLNC) cigarettes (e.g., <2.4 mg/g) suggest that, acutely, 
they produce many effects in smokers that are qualitatively similar to normal nicotine content 
(NNC; e.g., 15.8 mg/g) cigarettes, but with somewhat reduced efficacy.  VLNC cigarettes reinforce 
behavior10,11 and maintain similar rates of self-administration as NNC cigarettes in brief sessions 
even though participants prefer NNC cigarettes when given a choice11 Compared to not smoking, 
VLNC cigarettes increase ratings of satisfaction and liking12,13,14, although the magnitude of these 
effects is typically reduced compared to those produced by NNC cigarettes15,16,17. VLNC 
cigarettes also reduce withdrawal and craving18 although some symptoms (e.g., restlessness, 
impatience) may be more effectively alleviated by NNC cigarettes19.   
 
Studies in which VLNC cigarettes were utilized over a longer period suggest potential benefits in 
terms of reduced smoking. When only VLNC cigarettes were available in an inpatient setting, the 
number of cigarettes smoked and the motivation to smoke during periods of abstinence 
decreased over time12. Longer, outpatient studies have either found no change in smoking 
rates20,21,22,13, or a significant decline in smoking rates. 23,24  Some studies have also shown that 
VLNC cigarettes reduce nicotine dependence24 and increase quit attempts. 23 In one study 
conducted among individuals interested in quitting smoking, there was a significant increase in 
smoking cessation.24  
 
There is very little evidence of adverse outcomes associated with VLNC use, including 
compensatory smoking. As noted above, data available to date indicate that the number of 
cigarettes smoked per day tends to decrease, not increase, over time. 23,13,24  Furthermore, 

 
 
Page 2 of 23 
 
participants tend to reduce the volume of smoke inhaled and some studies have reported 
significant decreases in expired carbon monoxide (CO). 13,24  If compensatory smoking occurs, it 
appears to be short-lived25,26 and may be more likely in heavily dependent smokers. 27  Likewise, 
few other unintended consequences have been identified. In our recent clinical trial, there was 
also no evidence of an increase in depressive symptoms among those with higher depression 
scores at baseline28 and no evidence of an increase in alcohol or marijuana use. 29,30 However, 
individuals who were compliant with the VLNC cigarettes did gain weight over a six-week 
intervention period at a rate similar to what might be expected during smoking cessation. 31 
 
Most studies to date provide smokers with only two options: smoke study cigarettes (typically 
provided free to participants) or abstain from using tobacco, with clear instructions to not use non-
study products. In this context, most smokers not only continue to smoke, but, despite instructions 
to the contrary, continue to use non-study cigarettes. Based on self-report, more than 75% of 
participants assigned to VLNC cigarettes use at least one non-study (normal nicotine) cigarette 
during trials that last 6 weeks or longer, typically smoking 2-4 non-study cigarettes on at least a 
third of the days.23,32 Consequently, nicotine exposure (as measured by urine total nicotine 
equivalents) only decreases by about 60% compared to the 97% reduction of nicotine content in 
the product. 23 This pervasive noncompliance may attenuate the effects of VLNC cigarettes on 
abstinence33, but it also may foreshadow how patterns of product use would evolve if nicotine 
product standards were enacted for combusted tobacco. Some may seek out black market 
cigarettes or tamper with cigarettes to increase nicotine delivery, but many others may turn to 
easily accessible alternative products.  
 
Reducing the nicotine content of combusted products: the role of e-cigarettes 
 
E-cigarettes have rapidly emerged as a commonly used non-combusted product in the U.S.34, 35 
with a proposed rule by FDA that it they be deemed a "tobacco product” and regulated as such.36 
A study by Hatsukami and colleagues showed that approximately half of all smokers switched to 
VLNC cigarettes will try e-cigarettes when provided the opportunity and that there is a negative 
association between smoking VLNC cigarettes and use of alternative products.37 However, we 
know little about how the characteristics of e-cigarettes impact VLNC use and, conversely, how 
nicotine reduction in cigarettes impacts the use of e-cigarettes with different characteristics. Both 
nicotine and non-nicotine factors may be important. 
 
Nicotine. Nicotine delivered via e-cigarettes may serve a variety of functions that support its 
actions as a reinforcer and an appealing alternative to smoking VLNC cigarettes. Some e-
cigarettes can deliver as much or more (among experienced users) nicotine as a traditional 
cigarette with a relatively rapid rate of absorption (T max < 5 min).38 This pharmacokinetic profile 
suggests that e-cigarettes may have greater positive reinforcing effects than VLNC cigarettes.39 
E-cigarettes can also be used throughout the day to maintain nicotine levels that would otherwise 
fall by 95% or more once participants start using VLNC cigarettes.40 Indeed, e-cigarettes reduce 
withdrawal symptoms in cigarette smokers,41-45 and this suppression is dependent on the nicotine 
concentration of the e-liquid.45 Relatedly, greater nicotine delivery may also result in increased 
risk of dependence on e- cigarettes.46 Finally, nicotine also acts peripherally on receptors in the 
oropharyngeal cavity which may contribute to sensory effects (discussed below) that might 
generalize across products. Together, these data suggest that e-cigarettes that more effectively 
deliver nicotine may be more likely to function as both positive and negative reinforcers and more 
able to compete with the reduced reinforcing effects of VLNC cigarettes. 
 
Non-nicotine factors. E-cigarettes have a somewhat unique feature relative to other alternative 
products - they provide sensorimotor stimuli similar to combusted cigarettes, including a smoke-

 
 
Page 3 of 23 
 
like vapor and tobacco and other flavors. Decades of research highlight the importance of 
sensorimotor stimuli in maintaining behavior even when nicotine is absent.47,48 Stimuli that have 
been associated with nicotine can elicit craving,49 increase the probably of use,50 increase positive 
subjective experiences,51-54 reduce withdrawal,53,55,56 and support behavior even in the absence 
of the drug.57,58 These data suggest that the sensorimotor experience is an important determinant 
of whether a product will substitute for traditional cigarettes. E-cigarettes may serve this function 
better than other alternative products which might explain both why they were chosen most often 
in our pilot study (Overview D.2) and why they attenuate abstinence in cigarette users even if they 
do not deliver nicotine.43 
 
E-cigarettes are available in a variety of flavors that may impact the appeal and reinforcing effects 
of the product. The full impact of flavors on e-cigarette use is still unknown, but it seems likely that 
the availability of non-tobacco flavors influences the likelihood of use. Recent estimates are that 
53% of adults use e-cigarettes with flavors other than tobacco.59 Other studies also supported this 
assumption, demonstrating that liquid flavor choices influence appeal and choice of e-cigarette 
device.59, 60 If flavors impact the appeal of e-cigarettes, they may also impact the likelihood that 
smokers will switch from VLNC cigarettes to these products. 
 
Measures for assessing product use and the impact of product characteristics 
 
Behavioral Economics. Behavioral economics is an ideal framework for answering tobacco 
regulatory questions because it applies economic constructs such as product substitution and 
demand elasticity to individual product choice and decision-making. In behavioral economics, 
“price” is typically indicated in dollars but changes in price are also considered to be analogous 
to changes in effort required to obtain the product. The Cigarette Purchase Task (CPT) is an 
example of a behavioral economic assessment for smoking. In the task, participants are asked to 
indicate how many cigarettes they would purchase as the price of these products increases. From 
these data, a number of measures can be calculated including demand intensity (consumption at 
the lowest price) and demand elasticity (sensitivity to increases in price). A variation of the CPT 
is a cross-price task, which can be used to examine how changing the price of one commodity 
affects consumption of another commodity.  
 
The Experimental Tobacco Marketplace (ETM) task is an extension of the cross-price task and 
allows researchers to examine the effects of cigarette price increases on alternative product 
purchasing within a complex tobacco marketplace. In this paradigm, participants use account 
balances to buy tobacco products from an online store. Across multiple task iterations, the price 
of cigarettes increases while the prices of the other tobacco products remain constant. Assessing 
cigarette substitution across multiple tobacco products is more informative to tobacco regulatory 
science than simple cross-price tasks because it more closely simulates real world decision-
making from retailers selling multiple tobacco products. Recently, we conducted a pilot study 
using the ETM to examine the impact of a nicotine reduction policy on tobacco product 
purchasing. 
 
Discrete Choice. Interdisciplinary research describes choice as a function of the relative 
expected value of the options at hand.61, 62 Further, an option’s anticipated value is derived from 
the integrated values of its individual attributes.63, 64 Widely used in consumer research, discrete 
choice experiments (DCEs) reveal the extent to which individual product attributes determine 
choice.65 In DCEs, across several trials, participants chose their preferred option from a selection 
of products, each described as a set of attributes. The products/combinations of attributes 
presented in each trial are experimentally varied such that the relative importance of each attribute 
can be discerned using logistic regression. In tobacco regulatory science DCEs have been used 

 
 
Page 4 of 23 
 
frequently to evaluate general preferences for cigarette and vaping device characteristics.66-69 
Here, we will use the DCE method to investigate the contribution of nicotine content and flavor 
availability to choices between cigarettes and vaping devices. 
 
 
OBJECTIVES 
 
We are proposing a four condition ETM study to determine if banning NNC is sufficient for 
encouraging smokers to switch to potentially less harmful products or if the availability of higher 
nicotine and/or flavored alternative products (specifically e-cigarettes) are needed to achieve 
maximal reductions in smoking.   
 
 
METHODS AND MEASURES 
 
Overall Study Design 
 
We propose to conduct a within-subjects, multi-session virtual lab study and validation field 
assessment to determine how the availability of flavored e-liquids affects product purchasing 
among cigarette smokers. Adult daily cigarette smokers (N=64) will be recruited for this study. 
After initial eligibility is assessed, participants will complete six virtual sessions (1 baseline, 4 
experimental, and 1 follow-up) and three in person sessions with one field assessment to evaluate 
their tobacco use. 
 
Design and Setting 
 
Participants will be recruited from Winston-Salem, NC and the surrounding area using 
advertisements on public transit, community flyers, and social media.. In-person lab visits to drop 
off biosamples and pickup study products will be conducted at the Wake Forest Tobacco Control 
Center of Excellence located in Biotech Place in the Innovation Quarter.  
 
Subject Selection Criteria 
 
We anticipate recruiting up to 100 participants to achieve 64 completers.         
 
 
Inclusion criteria: 
1. 21+ years of age 
2. Self-report smoking at least 5 cigarettes per day for the past year 
3. Breathe carbon monoxide (CO) level > 8 ppm or positive urine NicCheck  
4. Willingness to use other tobacco products during the study 
5. Speak, comprehend, and read English sufficiently to complete study procedures 
6. Have home access to a computer, smart phone, or tablet with a web camera and internet 
access 
7. Tried a vaping device at least once in their lifetime 
 
Exclusion criteria:  
1. Currently seeking treatment to quit smoking 
2. Self-reported serious medical or psychiatric condition(s) including cardiovascular and 
chronic respiratory diseases 
3. Body temperature > 100.4 F 

 
 
Page 5 of 23 
 
4. Cold, flu or COVID-19 symptoms including fever, cough, and runny nose in the past 30 
days 
5. Currently pregnant, breastfeeding or intending to become pregnant for the duration of the 
study or unwilling to agree to use adequate protection to avoid pregnancy 
6. CO reading > 80 ppm 
 
Those with unstable medical or psychiatric conditions are excluded as these symptoms could 
affect a participant’s ability to complete the study and how they make decisions about what 
tobacco products to use. We will exclude those indicating immediate readiness to stop smoking, 
as participation in this study may not lead to reductions in smoking. We will exclude pregnant 
people, those trying to become pregnant and those currently breastfeeding due to the potential 
harmful effects of tobacco use on developing fetuses and infants. Because participants are 
required to complete portions of the protocol independently during the virtual lab sessions, they 
will need to be able to independently read and comprehend the study materials.  
 
Those who report past 30 day cold, flu or COVID-19 symptoms will be temporarily excluded as a 
safety precaution. They will be able to re-screen once they have been symptom-free for more 
than 30 days.   
 
Telephone Screening Survey 
 
Interested individuals will complete a telephone screening interview. The screener will assess: 
current cigarette smoking; cigarette flavor preference; past 30 day use of tobacco products; 
pregnancy/breastfeeding; current health status, and intention to quit smoking. The interviewer will 
record the potential participant’s responses in REDCap, web-based data collection platforms 
hosted by Wake Forest School of Medicine. The phone screening data will be used to establish 
preliminary eligibility. It will not be used for research purposes. Identifying information will be 
stored separately in a password-protected Excel spreadsheet on the Tobacco Control Center of 
Excellence server. 
 
Potential participants will be instructed to show a valid, state issued photo ID during the virtual 
screening visit.  Acceptable forms of identification include a Driver’s License, State Photo ID Card, 
State Voter ID Card, Passport, or Military ID. If the potential participant does not have a valid, 
state issued photo ID, the interviewer can provide them with information on obtaining one. 
Participants will be given the option of receiving study information or visit reminders by phone, 
email, and/or text message. 
 
A participant must complete their Virtual Visit 1 within 30 days of completing the Telephone 
Screening Survey. If the participant is not able to attend the virtual screening visit in that 
timeframe, they will need to complete the telephone recruitment questionnaire again but will 
maintain the same Subject ID number in the participant screening database. 
 
Visits  
 
Virtual Visit 1 
 
Research staff will schedule video calls (e.g., WebEx or Zoom) with the participants. At their 
scheduled call time, research staff and participants will complete the consenting procedures 
(more information available below) and electronically sign the consent form via DocuSign. 
Participants will complete one verbal assessment with research staff via REDCap to collect their 
identifying information. Participants will then be sent a link to complete the eligibility surveys. 

 
 
Page 6 of 23 
 
Research staff and participants will remain on video while the participant completes the REDCap 
survey. This will enable participants the ability to ask questions and share computer screens, if 
needed.  
 
The following forms will be administered verbally by research staff via REDCap: 
1. Identifying Information Form will include the participant’s Subject Identifier, name, address 
(including the county of residence), email address, phone number, age, and date of birth. 
2. Tobacco Use History and Exposure Questionnaire, which measures variables such as 
smoking amount, cigarette brand, age of initiation of smoking, number of quit attempts, 
duration of quit attempts and duration of smoking. 
3. Concomitant Medications Form will assess current medication usage.  
 
 The following screening surveys will be self-administered via REDCap: 
1. Brief Medical History Questionnaire to query current diagnoses, symptoms and past health 
problems. 
2. Demographic History Questionnaire, which will assess age, gender, ethnicity, race, 
education, income, marital status, living situation, and employment history. 
3. Contemplation Ladder34 to assess intention to quit smoking.  
 
Fagerstrom Test for Cigarette Dependence35 (FTCD) to assess nicotine dependence levels. In 
the event that the REDCap websites are not functioning, the virtual visits will be rescheduled to a 
later date and time.  
 
Eligibility Determination 
 
Research staff will determine eligibility after reviewing all criteria, sans the expired breath carbon 
monoxide reading and pregnancy test. Researchers will pay ineligible participants $20 for their 
time spent completing the screening surveys. Eligible participants will complete additional surveys 
and will receive an extra $10 for their time.  
 
The following baseline assessments for eligible participants will be administered as an interview 
and entered into REDCap by the interviewer: 
1. Timeline Follow Back (TLFB) Questionnaire,36 which will assess past 14-day tobacco and 
nicotine product use. If a participant’s reported cigarette use on the TLFB conflict with their 
reporting on the Tobacco Use History and Exposure Questionnaire, the RA will point out 
the discrepancy to the participant and ask whether they would like to modify their 
responses on the Tobacco Use History and Exposure Questionnaire or clarify why they 
are discrepant (e.g., participants had an upper respiratory infection in the past week that 
resulted in atypical smoking behavior). 
 
The following surveys will be self-administered using REDCap: 
1. Tobacco Policy Questionnaire (adapted from International Tobacco Control (ITC) – Four 
Country survey) which assess support for various tobacco control policies, including 
banning menthol flavoring in cigarettes and e-cigarettes. 
2. Perceived Health Risks Scale, a measure of the perceived addictive potential and other 
health risks associated with cigarettes, e-cigarettes and nicotine gum 
3. Tobacco Product Interest Scale, a visual analog scale assessing interest in use of various 
tobacco and nicotine products 
4. Environmental/Social Influence on Tobacco use to assess environmental and social 
impacts on tobacco use. 
 

 
 
Page 7 of 23 
 
In-Person Lab Visit 1 
 
After completing Virtual Visit 1, participants will complete the COVID screener within 24 hours of 
their Lab visit via email or phone. Research staff will wear personal protective equipment including 
face masks and gloves. The participants will be required to wear facemasks when interacting with 
the staff. If participants do not have a mask, one will be provided to them to use for the remainder 
of the study. Temperature checks will be conducted immediately upon arrival using a thermal heat 
scanner. 
 
Participants will provide carbon monoxide (CO) readings at least 6-feet away from other people, 
including the research staff, while they have their masks off. NicCheck Strips, which assess 
cotinine levels in urine, will also be used to confirm smoking status if participants have an expired 
breath CO reading less than or equal to 8 ppm. Participants with child-bearing potential will 
complete a urine pregnancy test.  
 
The following surveys will be self-administered using REDCap: 
1. Minnesota Nicotine Withdrawal Scale a measure of nicotine withdrawal symptoms 
2. Questionnaire of Smoking Urges-Brief – Usual Brand Cigarette, which measures the urge 
to smoke 
3. Cigarette Evaluation Scale, which measures responses to cigarettes (e.g., reward, 
satisfaction) 
 
Discrete choice task 1: The Sawtooth Lighthouse Studio platform will host a discrete choice task 
(See DCE Questionnaire). First, participants will rate the appeal of cigarettes and e-cigarettes 
characteristics, which helps familiarize them with the product features that will be manipulated in 
the choice task. Next, the choice task will begin, and they will make several choices between a 
cigarette option and an e-cigarette option responding to a prompt that asks which they would 
rather use exclusively for the next week. Each cigarette vs e-cigarette trial will present different 
combinations of cigarette nicotine content, e-liquid nicotine content, and e-liquid flavor. Finally, 
the participant will complete follow-up questions about which characteristics most informed their 
choices. The data from this task will reveal how participants prioritize and make tradeoffs 
between product type, nicotine content, and flavor when forming preferences.  
After completing the discrete choice procedure, participants will also be given the opportunity to 
sample the low and normal nicotine Spectrum investigational cigarettes available during the ETM 
(up to 4 puffs each). Other products will also be available on the ETM, but will not be available for 
sampling.  These include: fruit, dessert, menthol, and tobacco flavored e-liquids; and mint, fruit 
and cinnamon flavored nicotine gum.  
 
 
Virtual Visits 2-5  
Participants will complete Virtual Visit 2 between 1 and 21 days after completing In-Person Lab 
Visit 1. Virtual Visits 2-5 will ideally be conducted four days in a row to reduce online task fatigue. 
However, visits may be scheduled up to 7 days apart. 
 
The following assessment will be administered as an interview and entered into REDCap by the 
interviewer: 
1. Timeline Follow Back (TLFB) Questionnaire, which will assess tobacco and nicotine 
product use since their last virtual visit.  
 

 
 
Page 8 of 23 
 
Participants will complete the ETM task on the computer. If the participant needs assistance with 
navigating the online marketplace the research staff may screen share and allow the participant 
to verbally select the items that they would like to purchase. The products available on the ETM 
include:  
● Spectrum cigarettes  
● Fruit, dessert, mint and tobacco e-liquids (e.g., Vaporesso Xross Mini) 
● Mint, fruit, and cinnamon nicotine gum (e.g., Nicorette).  
 
Participants will undergo one of four marketplace conditions during Virtual Visits 2-5, with the 
order of administration randomized across participants.  
● Marketplace Condition 1 includes moderate nicotine content vape with all flavor e-liquids.   
● Marketplace Condition 2 includes moderate nicotine content vape with tobacco e-liquids.   
• 
Marketplace Condition 3 includes low nicotine content vape with all flavor e-liquids.  
• 
Marketplace Condition 4 includes low nicotine content vape with tobacco e-liquids.  
 
Staff will instruct participants to complete the ETM task as if they are purchasing the products 
from a retailer, and that one iteration of the task will be randomly selected and will determine 
which products they will use during the field assessment. In reality, we will randomly assign which 
condition is used (Condition 2 or 4) but will assign products based on when NNC were not 
available to simulate a nicotine content reduction policy.  
 
For each marketplace condition, participants will complete 7 iterations of the ETM task in 
ascending order. The prices for NNC cigarettes on the ETM will be set at: $0.12, $0.25, $0.50, 
$1.00, $2.00 per cigarette; in the 1st ETM condition, NNC cigarettes will be set at $0.25 with no 
VLNC available to purchase to simulate current market status; while in the 6th ETM condition, 
NNC will be eliminated from the marketplace to simulate a nicotine content reduction policy. The 
costs for the other products will reflect average prices in NC and will not change during the task. 
For each task iteration, participants will receive individually-tailored account balances 
approximately equal to the money they would spend on cigarettes for seven days. They can 
purchase as many or few products as their account balance allows. Participants are not required 
to buy products from the ETM or can buy nicotine gum if they prefer. If they do not buy products 
from the ETM and do not use tobacco products during the field assessment, then they will receive 
their ETM account balance at Virtual Visit 6. This payment is delayed to reduce the likelihood that 
they will use this money to buy their preferred brand cigarettes, thus decreasing the validity of the 
ETM task. 
 
Participants will be instructed to use only the products purchased from the ETM during the 
subsequent 7-day period after Virtual Visit 5 and told that we will assess adherence at Lab Visit 
3.  
 
In-Person Lab Visit 2 
 
In-Person Lab Visit 2 will be completed up to 48 hours after completing Virtual Visit 5. Participants 
will complete the COVID screener within 24 hours of their Lab visit via email or phone. Research 
staff will wear personal protective equipment during the visit including face masks and gloves. 
The participants will be required to wear facemasks when interacting with the staff. Temperature 
checks will be conducted immediately upon arrival using a thermal heat scanner.   
 
Participants will provide CO readings be at least 6-feet away from other people, including the 
research staff, while they have their masks off.  
 

 
 
Page 9 of 23 
 
Participants will be given the products they purchased during the ETM task and will be instructed 
to only use these products for the next seven days. They will also be given a urine collection cup 
and instructions for sample collection.  
 
The field assessment validates the ETM task by having participants use the products in the real 
world, simulating the different regulatory environments. Actual product use enables us to measure 
their subjective experiences such as withdrawal, craving, or product satisfaction as well as assess 
changes in carbon monoxide exposure, a biomarker of harm, when using products other than 
menthol cigarettes. We will conduct between-subjects analyses comparing these secondary 
outcomes by marketplace condition.  
 
 
Virtual Visit 6 
 
Participants will complete Virtual Session 6 seven days after completing In-Person Lab Visit 2.  
 
The following baseline assessments will be administered as an interview and entered into 
REDCap by the interviewer: 
1. Timeline Follow Back (TLFB) Questionnaire which will assess past tobacco and nicotine 
product use since Virtual Visit 5. 
 
The following surveys will be self-administered using REDCap: 
1. Tobacco Policy Questionnaire  
2. Minnesota Nicotine Withdrawal Scale 
3. Questionnaire of Smoking Urges-Brief – Usual Brand Cigarette 
4. Product Evaluation Scale – VLNC, E-Cigarette, Gum (whichever product(s) they select) 
5. Perceived Health Risks Scale 
6. Tobacco Product Interest Scale 
7. Health Changes Questionnaire  
 
Participants will complete the ETM task under the same marketplace condition from which they 
received their field assessment products. Exploratory pre-post field assessment analyses will 
examine how actual use of the products affects subsequent purchasing of those products.  
 
Research staff will conduct in-depth interviews with the participants to assess the following 
constructions: 1) decision-making during the ETM task; 2) use of alternative products during the 
field assessment; 3) product risk perceptions and; 4) opinions regarding possible tobacco control 
policies.  
 
In-Person Lab Visit 3 
 
In-Person Lab Visit 3 will be completed up to 48 hours after completing Virtual Visit 6. Participants 
will complete the COVID screener within 24 hours of their Lab visit via email or phone. Research 
staff will wear personal protective equipment during the visit including face masks and gloves. 
The participants will be required to wear facemasks when interacting with the staff. Temperature 
checks will be conducted immediately upon arrival using a thermal heat scanner. Participants will 
bring a first void urine sample, which will be banked for future biomarker analysis (e.g., total 
nicotine equivalents) and all of their used/unused products. 
 

 
 
Page 10 of 23 
 
Participants will provide CO readings at least 6-feet away from other people, including the 
research staff, while they have their masks off. Participants will complete a second iteration of 
discrete choice task 1. Data from the two choice tasks will be compared within participants to 
see whether product use affected preferences. 
All participants will be emailed a cessation manual and local smoking cessation resources. 
 
Product adherence 
 
To verify self-reported tobacco use, all used and unused products will be collected and counted 
at In-Person Lab Visit 3. Discrepancies between consumption stated on the TLFB versus 
consumption based on the products returned will be addressed. Non-adherence (i.e., using 
products not purchased from the ETM) will be assessed via the TLFB and qualitative interview 
with the importance of honest self-reporting emphasized. If participants report not using 
combusted tobacco, then CO readings < 6 ppm will verify their self-report.   
 
Compensation 
 
Participants will be paid $160 for completing the study. They will earn $30 for completing Virtual 
Visit 1, $15 per visit for completing Virtual Visits 2-5, $20 for Virtual Visit 6, plus a $50 bonus for 
completing the study. Participants who do not meet enrollment criteria at Virtual 1 will earn $20 
for attending the visit. Participants who start but do not complete the study will receive payments 
for the virtual visits attended. They will not receive the $50 completion bonus. Participants will 
also be reimbursed up to $20 per in-person lab visit for transportation costs.  
 
 
ANALYSIS PLAN 
Primary Outcome 
● Number of cigarettes purchased (analyzed as raw data and using derived behavioral 
economic measures). 
Secondary Outcomes 
● E-liquid purchased (analyzed as raw data and using derived behavioral economic 
measures). 
● Relative importance and utility data from discrete choice experiment 
● Perceived Health Risk Scale scores 
● Product Evaluation Scale scores 
● Minnesota Nicotine Withdrawal Scale scores 
● Questionnaire on Smoking Urges scores 
● Tobacco Product Interest scores 
● Tobacco Policy Questionnaire scores 
 
Exploratory Outcomes 
● Themes emerging from the qualitative interviews 
● Product use during Field Assessment 
● Biomarkers of exposure during Field Assessment 
 
Sample Size/Estimation/Analysis 
 
Statistical analysis plan  

 
 
Page 11 of 23 
 
This ETM study will use a 2x2 factorial design. The primary objective will be to evaluate the effect 
of the two factors – e-liquid nicotine content (moderate vs. low) and e-liquid flavors (all flavors 
available vs. only tobacco flavor) – on purchasing behavior during virtual visits 2 – 5. Product use 
behavior, biomarkers of exposure, and other measures of smoking behavior during the field 
assessment will be evaluated in exploratory analyses.  
Analysis of the Primary Outcome 
The primary outcome will be the number of cigarettes purchased during the virtual visits. These 
data will be analyzed both in their raw form (i.e. the raw number of cigarettes purchased) and 
using derived behavioral economic measures. This is a within-subject design, with each 
participant providing data under all four factor combinations, and, for the raw data, each 
participant will be provide data for all price combinations nested within a visit.  
Raw data will be analyzed using a linear mixed-effects model that includes main effects for e-
liquid nicotine content (low vs. moderate), e-liquid flavors (all vs. tobacco only), visit, price 
condition (analyzed as a categorical variable), and a carryover effect for the previous visits 
condition (treated as a four-level categorical variable, and random effects for participant and visit 
nested within participant. This will account for correlation from multiple observations from the 
same participant, both within and across visits. We will also evaluate whether there is an 
interaction between the e-liquid nicotine content and e-liquid flavors factors. If significant, the 
interaction will be included in the model; otherwise, it will be removed. The analysis of derived 
behavioral economic measures will follow an analogous approach, except that for each measure, 
there will only be a single outcome for visit. As a result, our linear mixed-effects model will not 
include a main effect for price condition, and there will only be a single random effect for each 
participant. Behavioral economic measures will be derived by generating demand curves using 
an exponential demand equation. Specific measures considered will include: Alpha - the rate of 
change in demand, and Q0 – peak consumption. For all analyses, assessments of key model 
assumptions will be completed and we will considering transformations or alternate analytic 
approaches if these assumptions are violated.   
Analysis of Secondary Outcomes  
The analysis of secondary endpoints will be analogous to the analysis described above for the 
primary endpoints. The amount of e-liquid purchased will be analyzed both as the raw data for 
each price condition and using derived behavioral economic measures. The analysis of these 
endpoints will be completed using linear mixed-effects models with the fixed and random effects 
terms described above. The analysis of the various scales and scores collected at each visit will 
follow the same approach as the analysis of derived behavioral economic measures described 
above. As before, assessments of key model assumptions will be completed and we will 
considering transformations or alternate analytic approaches if these assumptions are violated.   
Cross-price elasticity estimates will be calculated as the slope of the regression line (B1) fit to 
log-transformed purchasing of each alternative product when offered at fixed prices (C) versus 
log-transformed menthol cigarette price (P): logC = B0 + B1logP. Exploratory paired t-tests will 
compare product purchasing during the pre- and post-field assessment ETM task. 
Attribute-importance will be generated from discrete choice response data using logistic 
regression. A utility range (the difference between a characteristics highest and lowest estimated 
part-worth utility) will be calculated for device type, cigarette nicotine content, vaping device 
nicotine content and vaping device flavor availability. Part-worth utility refers to the relative 

 
 
Page 12 of 23 
 
contribution of each level of a product characteristic on choice (ie, as approximated by the 
estimated coefficients in the model). The relative importance of each characteristic on making a 
choice will be calculated as the utility range for each characteristic, divided by the sum of all the 
characteristics’ utility ranges. 
Analysis of Exploratory Outcomes 
Product use and biomarkers of exposure during the field assessment will be analyzed using linear 
regression or other appropriate regression models. The model for primary analysis of these data 
will include main effects for each of the two factors (for the ETM condition carried into the field 
assessment), and the corresponding baseline measure to improve precision. We will also test the 
two-way interaction between the two factors. If significant, the interaction will be included in the 
model; otherwise, it will be removed. We will also complete a secondary analysis that adjusted 
for age, race, sex, and other covariates that vary between treatment conditions. Biomarkers of 
exposure will be natural-log-transformed before analysis due to skewness. Key model 
assumptions will be evaluated for other endpoints and appropriate transformations will be 
considered if model assumptions are violated.  
For the qualitative data, we will use grounded theory to develop hypotheses related to the topics 
of interest. Interviews will be audio-recorded and transcribed verbatim. Audio transcriptions will 
be reviewed for accuracy and entered into Atlas.ti or another qualitative data management 
software program. Two research staff members will code the first five interviews to achieve greater 
than 90% interrater reliability and then every 10th interview to ensure integrity. The research team 
will analyze the interviews using iterative analyses including qualitative research techniques of 
open and axial coding; major thematic categories; analytical domains; triangulation of data; 
comparisons within and between groups; and documenting analytical pathways. Saturation will 
be achieved when no new themes emerge. 
Missing Data 
 
Study staff will make every effort to minimize missing data in the proposed study. Nevertheless, 
we plan to complete a sensitivity analysis to evaluate the impact of missing data on trial 
conclusions.  We will compare subjects with and without missing data in order to identify baseline 
covariates associated with missing data. Our primary approach to handling missing data will be 
multiple imputation where missing values are imputed using regression models developed from 
baseline covariates (Little and Rubin, 2002). In addition, we will conduct a sensitivity analysis of 
the primary and secondary endpoints using a complete-case analysis, baseline-carried-forward 
and last-observation carried forward. The results of these analyses will be compared to the 
primary analysis to evaluate the robustness of our conclusions. 
Power calculation 
Our objective is to enroll 64 completers, and we will test the main effects for e-liquid nicotine 
content, e-liquid flavors, and the interaction in a within-person design. In a within-person design, 
the power will depend heavily on the intra-participant correlation, which is unknown a priori. To 
evaluate the power, we simulate our trial under various assumptions regarding the effect size for 
the main effects and interactions, and the within-participant correlation (see Figure below). For 
the main effects of e-liquid nicotine content and e-liquid flavor, we have 80% power to detect an 
effect size of approximately 0.45 with a moderate within-participant correlation of 0.4, which 
ranges from approximately 0.49 with a within-participant correlation of 0.0, to 0.41 with a within-
participant correlation of 0.8. For the interaction, we have 80% power to detect an effect size of 

 
 
Page 13 of 23 
 
approximately 0.65 with a moderate within-participant correlation of 0.4, which ranges from 
approximately 0.75 with a within-participant correlation of 0.0, to 0.56 with a within-participant 
correlation of 0.8. 
 
 
 
 
HUMAN SUBJECTS PROTECTION 
 
Recruitment Methods 
 
Participants will be recruited by posting study information on flyers distributed in the community, 
Craigslist, social media channels, and Wake Forest Baptist’s “BeInvolved” site. Members of the 
PI’s research team will perform recruitment. Coordination of potential participants’ virtual visits will 
be done by phone or by email. Contact information will be stored on password-protected 
computers that are kept behind locked doors for which only the PI’s research team will have 
access keys and passwords. 
 
Recruitment will be continuous through the course of the study. If necessary, advertisements and 
recruitment strategies will be redesigned to target specific sub-populations if recruitment falls 
short of a representative demographic balance. 
 
Participants that have previously performed research with the PI and co-investigator research 
team and have agreed to be re-contacted for future studies may be contacted for recruitment 
purposes by email or phone according to their previously stated preference. 
 
Informed Consent 
 

 
 
Page 14 of 23 
 
Prior to Virtual Visit 1, participants will be briefly screened over the phone to determine initial 
eligibility. For this process alone, we will seek a waiver of signed informed consent. This part of 
the research procedure presents no more than minimal risk of harm to subjects and involves no 
procedures for which written consent is normally required outside of the research context. 
 
Before beginning the informed consent process during Virtual Visit 1, participants will need to 
show a valid, state issued photo identification. The interviewer will confirm the age and identity of 
the participant via video (e.g., WebEx, Zoom). The interviewer will then present a PowerPoint via 
screen share of the main takeaways of the informed consent form. Participants will be asked 
follow up questions to ensure comprehension of the informed consent form.  
 
In order to ensure adequate informed consent, participants will be asked to read the first several 
lines aloud to determine literacy. If the interviewer determines that the participant is not literate, 
they will be dismissed from the study but will receive $20 for attending the screening visit. The 
participant will be instructed to read several open-ended questions aloud and discuss the answers 
with the researcher. Only after the participant and the researcher are fully satisfied that the 
participant understands the purpose of the study, the confidentiality of the data, the procedures, 
the risks/benefits and his/her rights as a research participant will the consent form be signed 
electronically via DocuSign and the participant undergo screening procedures. The participant will 
be emailed a signed electronic copy of the consent document for their records.  
 
Confidentiality and Privacy 
 
Confidentiality will be protected by collecting only information needed to assess study outcomes, 
minimizing to the fullest extent possible the collection of any information that could directly identify 
subjects, and maintaining all study information in a secure manner. To help ensure subject privacy 
and confidentiality, only a unique study identifier will appear on the data collection form. Any 
collected patient identifying information corresponding to the unique study identifier will be 
maintained on a linkage file, stored separately from the data. The linkage file will be kept secure, 
with access limited to designated study personnel. Following data collection subject identifying 
information will be destroyed (three years after closure of the study; paper documents will be 
shredded, electronic documents will be securely deleted using software that will delete and 
overwrite sensitive disk space), consistent with data validation and study design, producing an 
anonymous analytical data set. Data access will be limited to study staff. Data and records will be 
kept locked and secured, with any computer data password protected. No reference to any 
individual participant will appear in reports, presentations, or publications that may arise from the 
study. 
 
Data Safety and Monitoring 
 
The principal investigator and co-investigators will be responsible for the overall monitoring of the 
data and safety of study participants. The principal investigator and co-investigators will be 
assisted by other members of the study staff. 
 
Potential risks of participation 
1. Survey Questionnaires: The interviews will include questions about medical history, 
tobacco use, and questionnaires about mood. Answering these personal questions could 
make the participant feel uncomfortable. 
2. Breach of Confidentiality:  The risk of the interview is loss of privacy if other people find 
out the results. 

 
 
Page 15 of 23 
 
3. Smoking Cigarettes: All cigarettes are detrimental to a person’s health and can lead to 
severe or fatal medical problems including: 
a. Cardiovascular diseases: Coronary heart disease, heart attack, stroke, peripheral 
vascular disease, reduced blood circulation, abdominal aortic aneurysm 
b. Respiratory diseases: Emphysema, bronchitis, tuberculosis and chronic airway 
obstruction 
c. Cancers: Lung, bladder, liver, colon, cervical, esophageal, kidney, larynx, mouth, 
pancreatic, throat, stomach cancers and acute myeloid leukemia 
d. Diabetes 
e. Abnormal immune function, rheumatoid arthritis 
f. Other health risks including but not limited to infertility, ectopic pregnancy, lower 
bone density in postmenopausal women, hip fracture in women, male sexual 
dysfunction, age-related macular degeneration, blindness and cataracts. 
4. Use of Vaping Devices: can exposure users to several chemicals including nicotine, 
carbonyl compounds, and volatile organic compounds, known to have adverse health 
effects. The health effects and potentially harmful doses of heated and aerosolized 
constituents of e-liquids, including solvents, flavorants, and toxicants, are not completely 
understood. Vaping device aerosol is not harmless “water vapor” although it generally 
contains fewer toxicants than combustible tobacco products. Specific potential known 
risks include 
a. Vaping devices contain nicotine which may contribute to some of the disease 
associated with smoking 
b. The most common side effects related to vaping devices are changes in taste, 
mucus in throat/sinus, dry mouth, dry cough, throat irritation, sore throat, mouth 
ulcers, dizziness, headache, and nausea. 
c. On rare occasions, batteries from vaping devices have exploded/ignited and 
injured users 
d. Vaping devices can overheat and present minor burn risks if the button is turned 
on repeatedly. Participants are told to be careful if storing the device in a place 
where the button might accidentally be pressed often. 
e. Ingestion of e-liquids containing nicotine can cause acute toxicity and possible 
death if the content of the cartridges containing nicotine are consumed. 
f. Participants are told to keep study vaping devices and all e-liquids away from 
children and pets. 
5. Use of Medicinal Nicotine: Most common adverse effects for medicinal nicotine include 
irregular heartbeat/palpitations, high blood pressure, mouth sores, mouth or throat 
irritation, heartburn, upset stomach, vomiting, diarrhea, dizzy or lightheadedness, and 
hiccups or belching. Additional adverse effects associated with nicotine gum include teeth 
or jaw problems. There may also be a risk of nicotine toxicity including symptoms such as 
nausea, dizziness, vomiting, diarrhea, and weakness. 
6. Dual use of tobacco or nicotine products: There is also the chance of use of more than 
one product and/or continued use of the products; however, cessation of all tobacco 
products will be strongly recommended to the participants at the end of the study. 
7. Smoking Withdrawal: Participants may experience smoking withdrawal symptoms during 
this study if they decide not to purchase tobacco products. The symptoms can be 
uncomfortable but are typically of minimal risk. Smoking withdrawal symptoms include: 
a. Anger, irritability, frustration 
b. Anxiousness, nervousness 
c. Depressed mood or sadness 
d. Desire or craving to smoke 
e. Difficulty concentrating 

 
 
Page 16 of 23 
 
f. Increased appetite, hunger or weight gain 
g. Insomnia, problems sleeping or waking at night 
h. Restlessness 
i. 
Impatience 
j. 
Constipation 
k. Dizziness 
l. 
Coughing 
m. Dreaming or nightmares 
n. Nausea 
o. Sore throat 
8. Returning to Regular Smoking: It is possible that if participants return to smoking their 
usual brand of cigarette at the end of the study they may experience mild and transient 
nausea, dizziness, and lightheadedness. 
9. Changes in Mood, Emotions and Psychiatric Symptoms: Smoking and nicotine can affect 
a person’s mood and emotions and are associated with psychiatric disorders including 
major depressive disorder, general anxiety disorder, bipolar disorder and eating disorders. 
Any changes in nicotine use or cigarette consumption could adversely affect mood, 
emotions and the symptoms related to psychiatric conditions in some individuals. 
10. Smoking and Oral Contraceptives in Women: Women who smoke and are over the age 
of 35 should not take oral contraceptives that contain estrogen without consulting their 
physician. Smoking while using oral contraceptives can increase the risk of having a 
cardiovascular event such as a heart attack or stroke. Additionally, there is a potential risk 
of thrombosis associated with hormonal therapy (including contraceptives) and smoking.   
 
Avoiding Risks during Pregnancy 
 
Smoking during pregnancy can lead to miscarriage, preterm delivery, stillbirth, birth defects, and 
other problems. To avoid risks to the participant and fetus, female participants will be tested for 
pregnancy at the Screening Visit. 
 
Expected benefits of participation 
 
There are no immediate benefits from participating in the study.  The information obtained from 
this study may ultimately help the Food and Drug Administration (FDA) decide how best to 
regulate tobacco products with the goal of improving public health.  
 
Reporting Unanticipated Problems, Adverse Events or Deviations 
 
Identifying Adverse Events 
 
While participating in the study, adverse events will be assessed at Virtual Visit 6 and carbon 
monoxide levels from In-Person Lab Visit 3.  Adverse events will typically be identified during the 
administration of the Health Changes Questionnaire. Other adverse events may be identified by 
spontaneous reports from the participant. Withdrawal symptoms are considered an adverse event 
if the symptom had a significant impact on the participant’s daily life, caused a major disruption of 
functioning, or took any medication for it.  
● Health Changes Questionnaire: If the participant answers “YES” to Questions 1, 2, or 3 
the interviewer should assess for adverse events. 
1. Have you had any negative changes in your physical or mental health since your 
last virtual visit? If yes, briefly describe. 

 
 
Page 17 of 23 
 
2. Since your last virtual visit, have you received any form of medical care?  If yes, 
briefly describe. 
3. Have you had any changes in medication since your last visit? 
 
Physiological data that will be reviewed: 
1. CO level: The ‘Adverse Event Form’ will be completed if the average of two (or three) 
consecutive measurements in the same visit is: 
o   CO is greater than 50 ppm if CO at Screening/Baseline Visit is < 20 ppm. 
o   CO is greater than 60 ppm if CO at Screening/Baseline Visit is 20 – 34 ppm. 
o   CO is greater than 70 ppm if CO at Screening/Baseline Visit is 35 – 49 ppm. 
 
For the participant’s protection, participants will be withdrawn immediately from the study if any 
of the following occur: 
1. Cardiovascular disease (CVD) event: Typically includes MI (heart attack), PTCA 
(angioplasty/stenting), bypass surgery, stroke, peripheral vascular disease (arterial 
blockages in arms or legs leading to procedure or surgery).  Less common CVD problems 
would be new cardiac arrhythmias (e.g., new atrial fibrillation) or new valvular disease 
(e.g., mitral or aortic regurgitation). 
2. DVT/PE (deep vein thrombosis/pulmonary embolism, i.e., blood clots in the venous 
system). 
3. Suicide Attempt:  A participant will be withdrawn if they attempt suicide at any time during 
participation in the study 
4. Psychiatric Hospitalization: A participant will be withdrawn if they are hospitalized for 
psychiatric reasons at any time during participation in the study. 
5. Pregnancy: If a participant indicates they are pregnant at Virtual Visit 6 or In-Person Lab 
Visit 3, they will be withdrawn from the study, and this event will remain open until delivery. 
At that time the license medical monitor will contact the participant to ask a few questions 
about the baby’s health and will update the open ‘Adverse Event Form’.  
 
The following will be monitored and can lead to the participant being withdrawn by the PI: 
1. Expired breath CO >80 ppm 
2. Any hospitalization or debilitation in which participation in the study could be detrimental 
to the recovery process. This will be self-reported by the participant and will be reviewed 
by the site PI and medical monitor to determine whether continued participation in the 
study is appropriate.  
3. If a participant is behaving in an inappropriate or threatening manner, admits to lying about 
eligibility criteria, including omitting previous medical diagnoses and medications, is 
participating in other smoking research studies that could affect the primary outcome 
measures, does not follow study instructions, etc., then the PI can withdraw them from the 
study at the PI’s discretion. 
4. If a participant fails to attend her/his Virtual Visit 1 within the 21-day allowable visit window, 
they will not be eligible to reschedule the visit or continue participation in the study. 
 
 
 
 
 
 
 
 
 

 
 
Page 18 of 23 
 
 
 
 
 
 
 
 
 
 
 

 
 
Page 19 of 23 
 
 
References  
 
1.  Jarvis MJ, Bates C. Eliminating nicotine in cigarettes. Tob Control 1999;8(1):106-7; author 
reply 
07-9 
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=10465828 
2. Kozlowski, L. T. (2015). Prospects for a nicotine-reduction strategy in the cigarette endgame: 
Alternative tobacco harm reduction scenarios. International Journal of Drug Policy, 26(6), 
543-547. doi:10.1016/j.drugpo.2015.02.001 
3. Kozlowski, L. T. (2016). Cigarette prohibition and the need for more prior testing of the WHO 
TobReg's 
global 
nicotine-reduction 
strategy. 
Tobacco 
Control. 
doi:10.1136/tobaccocontrol-2016-052995 
4. Kozlowski, L. T. (2017). Let actual markets help assess the worth of optional very-low-nicotine 
cigarettes before deciding on mandatory regulations. Addiction, 112(1), 3-5. 
doi:10.1111/add.13515 
5. Shatenstein S. Eliminating nicotine in cigarettes. Tob Control 1999;8(1):106; author reply 07-9 
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=10465827 
6. Henningfield J, Benowitz NL, Slade J, et al. Reducing the addictiveness of cigarettes. Council 
on Scientific Affairs, American Medical Association. Tob Control 1998;7(3):281-93 
http://tc.bmjjournals.com/cgi/content/full/7/3/281. 
7. Henningfield, J., Benowitz, N. L., Slade, J., Houston, T. P., Davis, R. M., & Deitchman, S. D. 
(1998). Reducing the addictiveness of cigarettes. Council on Scientific Affairs, American 
Medical Association. Tobacco Control, 7(3), 281-293. doi:10.1136/tc.7.3.281  
8. Hatsukami DK, Perkins KA, Lesage MG, et al. Nicotine reduction revisited: science and future 
directions. Tob Control 2010;19(5):e1-10 http://www.ncbi.nlm.nih.gov/pubmed/20876072. 
9. World Health Organization. (2015). WHO study group on tobacco product regulation: report on 
the scientific basis of tobacco product regulation: fifth report of a WHO study group. World 
Health Organization. https://apps.who.int/iris/handle/10665/161512 
10. Shahan, T. A., Bickel, W. K., Badger, G. J., & Giordano, L. A. (2001). Sensitivity of nicotine-
containing 
and 
de-nicotinized 
cigarette 
consumption 
to 
alternative 
non-drug 
reinforcement: A behavioral economic analysis. Behavioural Pharmacology, 12, 277-284.  
11. Shahan, T. A., Bickel, W. K., Madden, G. J., & Badger, G. J. (1999). Comparing the reinforcing 
efficacy of nicotine containing and de-nicotinized cigarettes: A behavioral economic 
analysis. Psychopharmacology, 147, 210-216. doi: 10.1007/s002130051162. 
12. Donny, E. C., Houtsmuller, E. J., & Stitzer, M. L. (2007). Smoking in the absence of nicotine: 
Behavioral, subjective and physiological effects over 11 days. Addiction, 102, 324-334. 
doi: 10.1111/j.1360-0443.2006.01670.x. 
13. Donny, E. C., & Jones, M. (2009). Prolonged exposure to denicotinized cigarettes with or 
without transdermal nicotine. Drug and Alcohol Dependence, 104(1-2), 23-33. 
doi:10.1016/j.drugalcdep.2009.01.021 
14. Rose, J. E., Behm, F. M., Westman, E. C., & Johnson, M. (2000). Dissociating nicotine and 
non-nicotine components of cigarette smoking. Pharmacology Biochemistry & Behavior, 
67, 71-81. doi: 10.1016/S0091-3057(00)00301-4. 

 
 
Page 20 of 23 
 
15. Butschky, M. F., Bailey, D., Henningfield, J. E., & Pickworth, W. B. (1995). Smoking without 
nicotine delivery decreases withdrawal in 12-hour abstinent smokers. Pharmacology 
Biochemistry & Behavior, 50, 91-96. doi: 10.1016/0091-3057(94)00269-O. 
16. Gross, J., Lee, J., & Stitzer, M. L. (1997). Nicotine-containing versus de-nicotinized cigarettes: 
Effects on craving and withdrawal. Pharmacology Biochemistry & Behavior, 57, 159-165. 
doi: 10.1016/S0091-3057(96)00309-7. 
17. Robinson, M. L., Houtsmuller, E. J., Moolchan, E. T. & Pickworth, W. B. (2000). Placebo 
cigarettes in smoking research. Experimental & Clinical Psychopharmacology, 8, 326-332. 
doi: 10.1037/1064-1297.8.3.326. 
18. Pickworth, W. B., Nelson, R. A., Rohrer, M. S., Fant, V. R., & Henningfield, J. E. (1999). 
Pharmacodynamic effects of new de-nicotinized cigarettes. Nicotine & Tobacco Research, 
1, 357-364. doi: 10.1080/14622299050011491. 
19. Buchhalter, A. R., Acosta, M. C., Evans, S. E., Breland, A. B., & Eissenberg, T. (2005). 
Tobacco abstinence symptom suppression: The role played by the smoking-related stimuli 
that are delivered by denicotinized cigarettes. Addiction, 100, 550-559. doi: 
10.1111/j.1360-0443.2005.01030.x. 
20. Benowitz, N. L., Dains, K. M., Hall, S. M., Stewart, S., Wilson, M., Dempsey, D. A., et al. 
(2009). Progressive commercial cigarette yield reduction: Biochemical exposure and 
behavioral assessment. Cancer Epidemiology, Biomarkers & Prevention, 18, 876-883. 
doi: 10.1158/1055-9965.EPI-08-0731. 
21. Benowitz, N. L., Dains, K. M., Hall, S. M., Stewart, S., Wilson, M., Dempsey, D. A & Jacob P. 
(2012). Smoking behavior and exposure to tobacco toxicants during 6 months of smoking 
progressively reduced nicotine content cigarettes. Cancer Epidemiology, Biomarkers & 
Prevention, 21(5), 761-769. Doi: 10.1158/1055-9965.EPI-11-0644. 
22. Benowitz, N. L., Hall, S. M., Stewart, S., Wilson, M., Dempsey, D. A., & Jacob, P. (2007). 
Nicotine and carcinogen exposure with smoking of progressively reduced nicotine content 
cigarettes. Cancer Epidemiology, Biomarkers & Prevention, 16, 2479-2485. doi: 
10.1158/1055-9965.EPI-07-0393. 
23. Donny, E. C., Denlinger, R. L., Tidey, J. W., Koopmeiners, J. S., Benowitz, N. L., Vandrey, R. 
G., . . . Hatsukami, D. K. (2015). Randomized Trial of Reduced-Nicotine Standards for 
Cigarettes. 
New 
England 
Journal 
of 
Medicine, 
373(14), 
1340-1349. 
doi:10.1056/NEJMsa1502403 
24. Hatsukami, D. K., Kotlyar, M., Hertsgaards, L. A., Zhang, Y., Carmella, S. G., Jensen, J. S., 
et al. (2010). Reduced nicotine content cigarettes: Effects on toxicant exposure, 
dependence 
and 
cessation. 
Addiction, 
105, 
343-355. 
doi: 
10.1111/j.1360-
0443.2009.02780.x. 
25. Macqueen, D. A., Heckman, B. W., Blank, M. D., Janse Van Rensburg, K., Evans, D. E., & 
Drobes, D. J. (2012). Transient compensatory smoking in response to placebo cigarettes. 
Psychopharmacology, 223(1), 47-54. doi:10.1007/s00213-012-2685-1 
26. Strasser, A. A., Lerman, C., Sanborn, P. M., Pickworth, W. B., & Feldman, E. A. (2007). New 
lower nicotine cigarettes can produce compensatory smoking and increased carbon 
monoxide 
exposure. 
Drug 
& 
Alcohol 
Dependence, 
86, 
294-300. 
doi: 
10.1016/j.drugalcdep.2006.06.017. 

 
 
Page 21 of 23 
 
27. Bandiera, F. C., Ross, K. C., Taghavi, S., Delucchi, K., Tyndale, R. F., & Benowitz, N. L. 
(2015). Nicotine Dependence, Nicotine Metabolism, and the Extent of Compensation in 
Response to Reduced Nicotine Content Cigarettes. Nicotine Tob Res, 17(9), 1167-1172. 
doi:10.1093/ntr/ntu337 
28. Tidey, J. W., Pacek, L. R., Koopmeiners, J. S., Vandrey, R., Nardone, N., Drobes, D. J., . . . 
Donny, E. C. (2017). Effects of 6-Week Use of Reduced-Nicotine Content Cigarettes in 
Smokers With and Without Elevated Depressive Symptoms. Nicotine Tob Res, 19(1), 59-
67. doi:10.1093/ntr/ntw199 
29. Dermody, S. S., Tidey, J. W., Denlinger, R. L., Pacek, L. R., al'Absi, M., Drobes, D. J., . . . 
Donny, E. C. (2016). The Impact of Smoking Very Low Nicotine Content Cigarettes on 
Alcohol Use. Alcoholism, Clinical and Experimental Research, 40(3), 606-615. 
doi:10.1111/acer.12980 
30. Pacek, L. R., Vandrey, R., Dermody, S. S., Denlinger-Apte, R. L., Lemieux, A., Tidey, J. W., . 
. . Donny, E. C. (2016). Evaluation of a reduced nicotine product standard: Moderating 
effects of and impact on cannabis use. Drug and Alcohol Dependence, 167, 228-232. 
doi:10.1016/j.drugalcdep.2016.08.620 
31. Rupprecht, L. E., Koopmeiners, J. S., Dermody, S. S., Oliver, J. A., al'Absi, M., Benowitz, N. 
L., . . . Donny, E. C. (2016). Reducing nicotine exposure results in weight gain in smokers 
randomised 
to 
very 
low 
nicotine 
content 
cigarettes. 
Tobacco 
Control. 
doi:10.1136/tobaccocontrol-2016-053301 
32. Benowitz, N. L., Nardone, N., Dains, K. M., Hall, S. M., Stewart, S., Dempsey, D., & Jacob, 
P., 3rd. (2015). Effect of reducing the nicotine content of cigarettes on cigarette smoking 
behavior and tobacco smoke toxicant exposure: 2-year follow up. Addiction, 110(10), 
1667-1675. doi:10.1111/add.12978 
33. Dermody, S. S., Donny, E. C., Hertsgaard, L. A., & Hatsukami, D. K. (2015). Greater 
reductions in nicotine exposure while smoking very low nicotine content cigarettes predict 
smoking cessation. Tobacco Control, 24(6), 536-539. doi:10.1136/tobaccocontrol-2014-
051797 
34. Arrazola RA, Singh T, Corey CG, Husten CG, Neff LJ, Apelberg BJ, et al. Tobacco use among 
middle and high school students - United States, 2011-2014. MMWR Morb Mortal Wkly 
Rep. 2015;64(14):381-5. PubMed PMID: 25879896. 
35. Agaku IT, King BA, Husten CG, Bunnell R, Ambrose BK, Hu SS, et al. Tobacco product use 
among adults- United States, 2012-2013. MMWR Morb Mortal Wkly Rep. 
2014;63(25):542-7. PubMed PMID: 24964880. 
36. Deeming tobacco products to be subject to the federal Food, Drug, and Cosmetic Act, as 
amended by the Family Smoking Prevention and Tobacco Control Act; Regulations on 
the sale and distribution of tobacco  
products and required warning statements for tobacco products; Proposed Rule. In: 
Administration DoHaHSFaD, editor. 2014. 
37. Polosa R. E-cigarettes: Public Health England's evidence based confusion? Lancet. 2015. 
doi: 
10.1016/S0140-6736(15)00133-6. PubMed PMID: 26346249. 
38. Spindle TR, Breland AB, Karaoghlanian NV, Shihadeh AL, Eissenberg T. Preliminary results 
of an examination of electronic cigarette user puff topography: the effect of a 
mouthpiece-based topography measurement device on plasma nicotine and subjective 

 
 
Page 22 of 23 
 
effects. Nicotine Tob Res. 2015;17(2):142-9. doi: 10.1093/ntr/ntu186. PubMed PMID: 
25239957. 
39. Ramoa CP, Hiler MM, Spindle TR, Lopez AA, Karaoghlanian N, Lipato T, et al. Electronic 
cigarette nicotine delivery can exceed that of combustible cigarettes: a preliminary 
report. Tob Control. 2015. doi: 10.1136/tobaccocontrol-2015-052447. PubMed PMID: 
26324250. 
40. Denlinger RL, Smith TT, Murphy SE, Koopmeiners JS, Benowitz NL, Hatsukami D, et al. 
Characterizing biomarkers of nicotine exposure when smoking very low nicotine content 
cigarettes in a controlled access environment. Cancer Epidemiol Biomarkers Prev. 
Under Review. 
41. Bullen C, McRobbie H, Thornley S, Glover M, Lin R, Laugesen M. Effect of an electronic 
nicotine delivery device (e cigarette) on desire to smoke and withdrawal, user 
preferences and nicotine delivery: randomised cross-over trial. Tob Control. 
2010;19(2):98-103. doi: 10.1136/tc.2009.031567. PubMed PMID: 20378585. 
42. Dawkins L, Turner J, Crowe E. Nicotine derived from the electronic cigarette improves time-
based prospective memory in abstinent smokers. Psychopharmacology (Berl). 
2013;227(3):377-84. doi: 10.1007/s00213-013-2983-2. PubMed PMID: 23344557. 
43. Vansickel AR, Cobb CO, Weaver MF, Eissenberg TE. A clinical laboratory model for 
evaluating the acute effects of electronic "cigarettes": nicotine delivery profile and 
cardiovascular and subjective effects. Cancer Epidemiol Biomarkers Prev. 
2010;19(8):1945-53. doi: 10.1158/1055-9965.EPI-10-0288. PubMed PMID: 20647410; 
PubMed Central PMCID: PMCPMC2919621. 
44. Vansickel AR, Weaver MF, Eissenberg T. Clinical laboratory assessment of the abuse 
liability of an electronic cigarette. Addiction. 2012; 107(8): 1493-500. doi: 10.1111/j.1360-
0443.2012.03791.x. PubMed PMID: 22229871; PubMed Central PMCID: 
PMCPMC3330136. 
45. Dawkins L, Turner J, Hasna S, Soar K. The electronic-cigarette: effects on desire to smoke, 
withdrawal symptoms and cognition. Addict Behav. 2012;37(8):970-3. doi: 
10.1016/j.addbeh.2012.03.004. PubMed PMID: 22503574. 
46. Etter JF, Eissenberg T. Dependence levels in users of electronic cigarettes, nicotine gums 
and tobacco cigarettes. Drug Alcohol Depend. 2015;147:68-75. doi: 
10.1016/j.drugalcdep.2014.12.007. PubMed PMID: 
25561385. 
47. Caggiula AR, Donny EC, White AR, Chaudhri N, Booth S, Gharib MA, et al. Cue 
dependency of nicotine selfadministration and smoking. Pharmacol Biochem Behav. 
2001;70(4):515-30. PubMed PMID: 11796151. 
48. Rose JE. Nicotine and nonnicotine factors in cigarette addiction. Psychopharmacology 
(Berl). 2006;184(34):274-85. doi: 10.1007/s00213-005-0250-x. PubMed PMID: 
16362402. 
49. Baker H. E-cigarettes and subsequent tobacco use in adolescence. The Lancet Oncology. 
2015. doi: 10.1016/S1470-2045(15)00272-7. PubMed PMID: 26321209. 
50. Thornley S, McRobbie H, Lin RB, Bullen C, Hajek P, Laugesen M, et al. A single-blind, 
randomized, crossover trial of the effects of a nicotine pouch on the relief of tobacco 
withdrawal symptoms and user satisfaction. Nicotine Tob Res. 2009;11(6):715-21. doi: 
10.1093/ntr/ntp054. PubMed PMID: 19454549. 
51. Furlow B. US Government warns against long-term dual use of conventional and e-
cigarettes. The Lancet Respiratory medicine. 2015;3(5):345. doi: 10.1016/S2213-
2600(15)00138-1. PubMed PMID: 25862134. 
52. McNeill A, Brose LS, Calder R, Hitchman SC, Hajek P, McRobbie H. E-cigarettes: the need 
for clear communication on relative risks. Lancet. 2015. doi: 10.1016/S0140-
6736(15)00079-3. PubMed PMID: 26338257. 

 
 
Page 23 of 23 
 
53. McRobbie H, Phillips A, Goniewicz ML, Smith KM, Knight-West O, Przulj D, et al. Effects of 
Switching to Electronic Cigarettes with and without Concurrent Smoking on Exposure to 
Nicotine, Carbon Monoxide, and Acrolein. Cancer prevention research. 2015;8(9):873-8. 
doi: 10.1158/1940-6207.CAPR-15-0058. PubMed PMID: 26333731. 
54. Glover M, McRobbie H. Electronic cigarettes appealing quit aids for young adult smokers. 
The New Zealand medical journal. 2015;128(1417):59-60. PubMed PMID: 26149908. 
55. Hajek P, Goniewicz ML, Phillips A, Myers Smith K, West O, McRobbie H. Nicotine intake 
from electronic cigarettes on initial use and after 4 weeks of regular use. Nicotine Tob 
Res. 2015;17(2):175-9. doi: 10.1093/ntr/ntu153. PubMed PMID: 25122503. 
56. Kalousova L. E-cigarettes: a harm-reduction strategy for socioeconomically disadvantaged 
smokers? The Lancet Respiratory medicine. 2015;3(8):598-600. doi: 10.1016/S2213-
2600(15)00239-8. PubMed PMID: 26116291. 
57.McRobbie H, Thornley S, Bullen C, Lin RB, Senior H, Laugesen M, et al. A randomized trial 
of the effects of two novel nicotine replacement therapies on tobacco withdrawal 
symptoms and user satisfaction. Addiction. 2010;105(7):1290-8. doi: 10.1111/j.1360-
0443.2010.02950.x. PubMed PMID: 20491724. 
58.McRobbie H, Bullen C, Hartmann-Boyce J, Hajek P. Electronic cigarettes for smoking 
cessation and reduction. Cochrane Database Syst Rev. 2014;12:CD010216. doi: 
10.1002/14651858.CD010216.pub2. PubMed PMID: 25515689. 
59. Yingst JM, Veldheer S, Hrabovsky S, Nichols TT, Wilson SJ, Foulds J. Factors Associated 
With Electronic Cigarette Users' Device Preferences and Transition From First 
Generation to Advanced Generation Devices. Nicotine Tob Res. 2015. doi: 
10.1093/ntr/ntv052. PubMed PMID: 25744966. 
60. Shiffman S, Sembower MA, Pillitteri JL, Gerlach KK, Gitchell JG. The Impact of Flavor 
Descriptors on Nonsmoking Teens' and Adult Smokers' Interest in Electronic Cigarettes. 
Nicotine Tob Res. 2015. doi: 10.1093/ntr/ntu333. PubMed PMID: 25566782. 
61. Plassmann H, Karmarkar UR. Consumer neuroscience: revealing meaningful relationships 
between brain and consumer behavior. In: Norton MI, Rucker DD, Lamberton C, eds. 
The Cambridge Handbook of Consumer Psychology. Cambridge University Press; 
2015:152-179. doi:10.1017/CBO9781107706552.006 
62. Herrnstein RJ. On the law of effect 1. Journal of the Experimental Analysis of Behavior. 
1970;13(2):243-266. doi:10.1901/jeab.1970.13-24 
63. Rangel A, Clithero JA. The computation of stimulus values in simple choice. In: 
Neuroeconomics. Elsevier; 2014:125-148. doi:10.1016/B978-0-12-416008-8.00008-5  
64. Muthukrishnan AV, Kardes FR. Persistent preferences for product attributes: the effects of 
the initial choice context and uninformative experience. J Consum Res. 2001;28(1):89-
104. doi:10.1086/321949 
65. Louviere JJ, Hensher DA, Swait J, Adamowicz WL. Stated Choice Methods: Analysis and 
Applications. 7. printing. Cambridge Univ. Press; 2010. 
66. Shang C, Weaver SR, White JS, et al. E-cigarette product preferences among adult 
smokers: a discrete choice experiment. tob regul sci. 2020;6(1):66-80. 
doi:10.18001/TRS.6.1.7 
67. Kistler CE, Ranney LM, Sutfin EL, et al. Product attributes important to US adult consumers’ 
use of electronic nicotine delivery systems: a discrete choice experiment. BMJ Open. 
2019;9(8):e027247. doi:10.1136/bmjopen-2018-027247 
68. Buckell J, Sindelar JL. The impact of flavors, health risks, secondhand smoke and prices on 
young adults’ cigarette and e‐cigarette choices: a discrete choice experiment. Addiction. 
2019;114(8):1427-1435. doi:10.1111/add.14610 
69. Czoli CD, Goniewicz M, Islam T, Kotnowski K, Hammond D. Consumer preferences for 
electronic cigarettes: results from a discrete choice experiment. Tob Control. 
2016;25(e1):e30-e36. doi:10.1136/tobaccocontrol-2015-052422