Helping People Adhere to Their Varenicline Treatment by Co-creating a Conversational Agent: A Feasibility Study
Phase: NA
Conditions: Medication Adherence, Smoking Cessation
Sponsor: Centre for Addiction and Mental Health

1 
 
Research protocol Title: The feasibility of an artificial intelligence conversational agent to 
improve varenicline adherence: A study protocol  
Date Approved by CAMH Ethics: 27 July 2023 
Abstract: 
Background: Interventions to increase medication adherence may help improve treatment and 
healthcare system efficiencies and cost-effectiveness. Digital health solutions are such 
intervention that can help improve medication adherence by providing reminders, answering 
questions and tracking medication intake. “ChatV” is an evidence-based, patient- and health care 
provider-informed healthbot to improve adherence to varenicline, a smoking cessation 
medication. 
Objective: To conduct a feasibility study to examine if the ChatV healthbot is used as intended, 
as well as to determine the appropriateness of proceeding with a randomized controlled trial.  
Methods: We will conduct a mixed-methods feasibility study where 40 participants will be 
prescribed varenicline and interact with the healthbot programmed to provide medication 
reminders, answering questions about varenicline and smoking cessation, and tracking 
medication intake and number of cigarettes. Follow-up survey data will be collected at 1, 4, 8, 
and 12 weeks and a semi-structured interview will be conducted at 12 weeks to understand 
participants’ experiences using the healthbot. A health equity lens will be adopted during 
participant recruitment and data analysis to understand and address the differences in uptake and 
utilization of this digital health solution among diverse socio-demographic groups. The 
Taxonomy of Implementation Outcomes will be used to assess the feasibility, i.e., acceptability, 
appropriateness, fidelity, adoption and usability. In addition, medication adherence and smoking 

2 
 
cessation will be measured to assess preliminary treatment effect. Interview data will be analysed 
using framework analysis method.   
Results: The study has been approved by the Centre for Addiction and Mental Health’s Research 
Ethics Board (#50/2022). Participant enrollment for the study began in May 2024.   
Conclusion: By employing predetermined progression criteria, the results of this preliminary 
study will inform the determination of whether to advance towards a larger randomised 
controlled trial to test the effectiveness of the healthbot. Additionally, this study will explore the 
acceptability, appropriateness, fidelity, adoption, and usability of the healthbot. These insights 
will be instrumental in refining the intervention/healthbot.   
Trial registration information:  
Registry name: Clinicaltrials.gov 
Registration URL: https://classic.clinicaltrials.gov/ct2/show/NCT05997901   
Registration number: NCT05997901    
Key words: Medication Adherence, Smoking Cessation, Varenicline, Healthbot, Evaluation 
 
 
 
 
 
 
 
Introduction 

3 
 
Medication adherence presents a significant and pressing challenge across global healthcare 
systems [1]. This problem spans across various medical conditions, from chronic illnesses like 
hypertension, diabetes, and cardiovascular disease, to acute infections [2]. Non-adherence to 
medication regimens can lead to a cascade of problems, including decreased treatment 
effectiveness, a higher risk of disease progression or complications, medication resistance, 
increased healthcare costs, and a reduced overall quality of life for patients [3]. It also places an 
additional burden on healthcare providers who must constantly navigate the complexities of 
patient adherence to optimize treatment outcomes. Addressing medication adherence is not only 
a matter of improving individual health, but also a vital step toward more efficient healthcare 
systems and improving population health. 
One notable area where medication adherence could have a significant effect on public health is 
with smoking cessation medications. Tobacco smoking is not only extremely addictive [4], it 
damages almost every organ in the body [5] and is a leading cause of health inequalities [6]. 
Despite decades of efforts to reduce the smoking prevalence in Canada [7], over four million 
(15%) Canadians smoked in 2020 [8]. Tobacco use remains the leading cause of preventable 
death in Canada [9]. and following smoking cessation, the risk for developing tobacco-related 
diseases gradually decreases over time [10].  
Varenicline, a partial agonist of the alpha4 beta2 nicotinic receptor, is one of the most effective 
smoking cessation medications associated with a doubling of long term quitting with 12 weeks of 
continuous treatment [11]; however, adherence is a major barrier to achieving and maintaining 
abstinence [12-14]. Among those who take varenicline for smoking cessation, over one third stop 
to taking the medication by the second week of treatment, and these patients are significantly less 
likely to quit smoking [15-18]. In addition, several studies that have examined the correlation 

4 
 
between socio-demographics and adherence to varenicline have found male sex [15, 19], older 
age [15, 19-21], white race [21, 22], and higher education to be associated with better adherence 
[21]. Conversely, one study found greater age was associated with poorer adherence [21]. 
One of the most effective strategies to improve adherence to medication is tailored behavioral 
interventions [23].  Research has also shown that the use of technological solutions is crucial to 
providing personalized, cost-effective, and evidence-based interventions [24]. One such digital 
health solution is web based programs integrating conversational agents or “healthbots” that can 
be helpful in improving adherence to medication [25]. Healthbots have greater functionality 
compared with applications providing simple text message reminders since they can track 
medication intake, provide information about side effects, and answer medication-related 
questions [26, 27]. 
The “ChatV” healthbot is an example of such a digital health solution that we developed to 
improve adherence to the smoking cessation medication varenicline [28]. We utilized a theory-
based patient-centred approach through the Discover Design Build and Test framework [29] to 
design the healthbot by conducting a rapid review to explore the barriers and facilitators to 
varenicline adherence, interviewing 20 patients and 19 healthcare providers to understand similar 
barriers and facilitators as well as features to be included in a healthbot that may improve 
varenicline adherence. The data from these sources was analyzed using the COM-B (Capability, 
Opportunity, Motivation- Behaviour Change) [30] model of behaviour change and its associated 
framework (Theoretical Domain Framework) [31] and taxonomy (behaviour change techniques 
taxonomy) [32] to identify the features to be included in the healthbot. We used the Wizard of 
Oz methodology [33] to build a library of responses that were programmed in the healthbot to 
answer users’ questions. We are currently beta testing the minimal viable product that has the 

5 
 
following features: 1) scheduling and receiving reminders for varenicline dosing; 2) setting a 
smoking quit date; 3) answering questions related to varenicline (e.g., guidance for missed doses) 
and smoking cessation (e.g., strategies to reduce cravings); 4) providing information and 
strategies to manage side effects [12, 15, 18]); 5) tracking medication use and smoking; and 6) 
encouraging participants to increase their motivation to maintain medication intake and continue 
their quit attempts. Throughout the development of the healthbot, a health equity lens was 
adopted to address any disparities in varenicline use and healthbot features between different 
demographic characteristics.  
In this protocol, we detail our plans to examine the uptake, usability, acceptability, and 
appropriateness of ChatV (i.e. feasibility) [34] and to determine whether progression to a fully 
powered randomized controlled trial (RCT) is warranted.  
Methods 
Design  
We will conduct a non-randomized single arm feasibility study. 
Participant recruitment. 
We will recruit 40 participants. In order to ensure diversity with respect to age, gender, race, and 
socio-economic status, we will purposefully recruit participants through a variety of methods, 
including: the Centre for Addiction Mental Health’s (CAMH) research boards where recruitment 
flyers are posted for patients to explore research opportunities; through healthcare providers at 
CAMH programs (e.g., Smoking Treatment for Ontario Patients (STOP) Program, Aboriginal 
services, Rainbow services) and partnering organizations (Rainbow Health Ontario); the STOP 
Program’s list of people who are seeking treatment for smoking and who have consented to be 
contacted for future research studies; and social media and community boards. Research staff 

6 
 
members will conduct a pre-screening for those people who are interested, and if eligible, 
participants will be enrolled in the study. 
Eligibility.  
Individuals seeking treatment for their tobacco use who smoke 10 or more cigarettes a day, are 
willing to set a quit date in the next 30 days, are willing to take varenicline for 12 weeks, are 
aged  18 years, speak/read English, have a smartphone with data plan, report being committed 
to answering questions during follow-up, and live in Ontario. 
Exclusion criteria.  
Have contraindications to varenicline use, are pregnant, planning to become pregnant, or 
breastfeeding, or participated in the co-design phase.  
Ethics.  
The CAMH Research Ethics Board (REB # 050/2022) has provided approval to conduct the 
study and all participants will provide written informed consent before participation. 
Procedures.  
The research team will inform potential participants about the study and evaluate if they meet the 
inclusion criteria. If they qualify, participants will receive a consent form and arrange a phone-
based discussion to obtain consent. After obtaining consent, participants  (n=40) will complete 
an online questionnaire, covering: socio-demographic characteristics, nicotine dependence [35], 
and an adapted scale to measure varenicline adherence self-efficacy [36-38]. After completing 
the questionnaire, participants will have a virtual or in-person visit with a healthcare provider for 
eligibility confirmation and to obtain a prescription for varenicline for the first four weeks (one 
tablet [0.5 mg] daily for the first three days, one tablet [0.5 mg] twice daily for the next four 
days, and one tablet [1 mg] twice daily for three weeks). In addition, research staff will show 

7 
 
participants how to use the healthbot on their phones. Medication will be provided in-person or, 
in the case of virtual contacts, mailed to the participants. A follow-up visit will be scheduled at 
two weeks where healthcare providers will assess the participant’s tolerance of varenicline and 
provide a prescription for the remaining eight weeks (one tablet (1 mg) twice daily or a different 
dose depending on the tolerance of each participant). Any unanticipated problems (i.e., adverse 
events that are unexpected in terms of severity, nature or frequency; are related, or possibly 
related to participation in the research; and that suggest that the research places other research 
participants at greater risk of harm) will be documented as applicable and reported to the 
Research Ethics Board, and adverse drug reactions will be reported to the Market Authorization 
Holder (APOTEX) as applicable.  
Participants will be required to complete surveys at 1, 4, 8, and 12 weeks. When participants join 
the study they will be able to opt to respond to these questions using either the healthbot or an 
online platform. Those who do not complete the online or healthbot follow-up within the 
designated timeframe will receive a phone call from our research staff. Additionally, after 
12weeks of joining the study, participants will be invited to participate in a 1-hour semi-
structured phone interview. Participants will receive honorarium in the form $30 gift card after 
completion of the 4 week follow up survey, and $40 gift card after completion of the 12 week 
interview. 
Measures.  
The Taxonomy of Implementation Outcomes of Proctor et al. [39] guides our selection of 
implementation outcomes (Table 1) as follows: 
Acceptability “is the extent to which an innovation is agreeable, palatable, or satisfactory 
to a stakeholder” [39]. In order to measure acceptability we will utilize the Acceptability of 

8 
 
Intervention Measure (AIM) in the follow up surveys, as well as in the semi-structured phone 
interviews. 
Appropriateness is the “perceived fit or compatibility of an innovation with a practice 
setting or context” [39]. In this study, appropriateness will be assessed on an individual basis, 
including factors such as alignment with users' attitudes, needs, and background, as measured in 
the 12-week survey., with the Intervention Appropriateness Measure (IAM), a validated 4-item 
intervention appropriateness scale [40], and explored in depth in the semi-structured interviews. 
Fidelity is “the extent to which an intervention is used as intended” [39].For this study, in 
the semi structured interviews we will investigate the functions of the healthbot that participants 
used, the purposes for which they employed it (such as answering questions or seeking support), 
the frequency of usage, and the quality, specifically whether the healthbot provided accurate 
information. 
Adoption is “the intention, decision, or initiation of use for an evidence-based practice, 
characterizing it at the level of the provider or organization” [39]. Since the concept of adoption 
is closely related to the actual usage of systems, researchers investigating behavioral intervention 
technologies, such as the healthbot, have extended this level of analysis to include consumer 
behavior. [41]. In this study, we will measure the adoption of the healthbot by analyzing 
analytics data throughout the 12 weeks that the participant is scheduled to take the varenicline. 
This data will allow us to evaluate the timing (day/time) of user interactions with the healthbot, 
the specific features utilized, and the duration of engagement. Thus, we will have objective data 
of actual adoption of the healthbot by each participant. Additionally, during the semi-structured 
interviews, we will probe for participants' subjective perceptions of their use of the healthbot  

9 
 
Usability will be assessed using the System Usability Scale (SUS) [42]. The SUS is a 10-
item measure assessing usability and user satisfaction with technology. 
The AIM, IAM and SUS all have items that are rated on a 5-point Likert scale 
(1=completely/strongly disagree to 5=completely/strongly agree). 
Medication Adherence and Smoking Status Participants will be asked about their smoking 
status [43] and adherence to varenicline, using timeline follow-back (TLFB) [44], in the surveys 
collected at 1, 4, 8, and 12 weeks. Participants will report the number of pills taken since the 
previous assessment. Participants who log their varenicline use in the healthbot will be able to 
refer to these data when filling out the TLFB. We chose the TLFB method for assessing 
adherence because the 12-week TLFB measure demonstrates a moderate correlation with saliva 
varenicline levels and is widely regarded as the most practical measure of varenicline adherence 
[16]. In order to assess the feasibility and accuracy of measuring adherence using TLFB, 
participants will also be asked to send pictures of their varenicline blister packs at 1, 4, 8 and 12 
weeks. This additional measure will also help explore which method to assess adherence should 
be used for any future research studies (e.g., RCT). Smoking abstinence will be defined by a 
negative response to the dichotomous 7-day point prevalence question, “Have you had a 
cigarette, even a puff, in the last seven days?”  
Table 1. Summary of measures  
Measures 
Survey timeline 
Interview
Analytics 
 
Baseline
Week 1 
Week 4 
Week 8 
Week 12 
Week 12  
Week 1-12 
Demographics  
X 
 
 
 
 
 
 
Adoption 
 
 
 
 
 
X 
X 

10 
 
Acceptability 
 
AIM 
AIM 
AIM 
 AIM 
X 
 
Appropriateness 
 
IAM 
IAM 
IAM 
 IAM 
X 
 
Fidelity 
 
 
 
 
 
X 
 
Usability 
 
SUS 
SUS 
SUS 
SUS 
X 
 
Medication 
Adherence 
 
TLFB 
TLFB 
TLFB 
TLFB 
X 
 
Smoking Status 
 
X 
X 
X 
X 
X 
 
Analysis of survey data.  
We will calculate descriptive statistics, including means, medians, ranges, and standard 
deviations, for the demographic data as well as for the AIM, IAM, and analytics data captured by 
the healthbot internal logs. All scores will be analyzed using the appropriate guidelines provided 
by Weiner[40] (acceptability and appropriateness) and the U.S. Department of Health and 
Human Services [45] (usability). Consistent with previous studies, adherence will be defined as 
taking  80% of prescribed varenicline [46-49]. We will not treat adherence as a continuous 
variable because it is likely to be strongly bimodal, and is not necessarily linearly associated with 
the cessation outcome. We will calculate the percentage of participants that were adherent at 
week 12 and who reported not having smoked in the last seven days. These results will serve as 
progression criteria to determine whether a full scale RCT is warranted. Although the sample 
size will be insufficient for meaningful regression analysis, especially given potentially non-
linear effects of age, we will also test for differences across age and gender groups using t-tests 
and chi-squared tests. Power for these comparisons will be low; therefore, this analysis will be an 
exploratory evaluation of large adherence or outcome differences that may be important to future 
study design, and interpretation will focus on observed differences and confidence intervals. 

11 
 
Analysis of semi-structured interviews.  
We will use framework analysis [50] to analyze the data. Specifically, we will use the Proctor et 
al.’s implementation outcomes [39], with an intersectional lens, as the a priori framework. 
Framework analysis is method used to draw descriptive and explanatory conclusions from 
qualitative data and includes six steps: 1) verbatim transcription of interviews; 2) familiarization 
of interviews; 3) coding/labelling the data; 4) developing a framework by grouping and 
categorizing codes; 5) indexing transcripts using the developed framework; and 6) reducing and 
charting data into a matrix. Interview findings will help us contextualize the findings from the 
surveys and information collected from the use of the healthbot. For example, we will understand 
why the healthbot was or was not used (adoption): if it was used as intended (fidelity), what 
about the healthbot participants’ found agreeable and what they did not (acceptability); what 
features, if any, aligned with participants’ attitudes, needs, and background (appropriate); and 
how other features would need to change in order to align with their needs. In addition, we will 
be able to understand how social identity (e.g., age, gender) and power structures (e.g., ageism, 
sexism) played a role in these results. This will provide us with important information to improve 
the healthbot through an equity lens. 
Triangulation of findings.  
Once the survey data and interviews are analyzed, we will triangulate the results by listing all the  
findings on the same page to identify instances where findings from each method align 
(convergence), provide additional insights on the same topic (complementarity), or seem to 
conflict with one another (discrepancy or dissonance) [51, 52]. Triangulating the data will allow 
us to contextualize the quantitative findings and explore “inter-method discrepancy” that might 
lead to a better understanding of how the healthbot works and the modifications needed. 

12 
 
Exploring the feasibility of the approaches on recruitment, retention and data collection  
This study, using the  ‘Stop, Amend, Go’ progression criteria for pilot studies [52], will help 
determine if proceeding with a RCT is a logical next step and whether additional adjustments are 
necessary. We will only conduct a full scale RCT if none of the criteria outlined in Table 2 meet 
the ‘Stop’ criterion (unless we see that there are contextual issues that can be modified to 
overcome the identified problems). If one or more of the concepts in Table 2 meets the ‘Amend’ 
criteria, the trial will proceed to an RCT only if the issue(s) can be addressed. If all concepts in 
Table 2 satisfy the 'Go' criterion, we will proceed with planning an RCT. 
Table 2. Criteria to decide if RCT is warranted 
Criteria 
Stop 
Amend 
Go 
Patient recruitment 
If <30 patients are 
recruited within 12 
months 
If 30-40 patients are 
recruited within 12 
months 
If 40 patients are 
recruited within 
12 months 
Patient Retention  
If <25% patients 
answer the 12 week 
survey 
If 25.1-79.9% 
patients answer the 
12 week survey 
If ≥ 80% patients 
answer the 12 week 
survey 
Missing Data – 
Medication 
Adherence or 
Smoking Status 
If > 60% of 
adherence data and/or 
smoking status are 
missing 
If 21–59% of 
adherence and/or 
smoking status data 
are missing 
If ≤ 20% of 
adherence and/or 
smoking status data 
are missing 

13 
 
Acceptability, 
Appropriateness 
If  mean score for the 
AIM and/or IAM ≤ 8  
If  mean score for the 
AIM and/or IAM 9-
12 
If  mean score for the 
AIM and/or IAM 
≥ 13 
Usability  
If mean SUS score 
≤ 40 
If mean SUS score 
41-67 
If mean SUS score 
≥ 68 
Adoption 
If mean use ≤  20 
times  
If mean use 21-79 
times  
If mean use  ≥ 80 
times 
Results 
Participant enrollment for the study began in May 2024. 
Discussion 
Despite the proven effectiveness of varenicline in aiding smoking cessation [53], a significant 
number of individuals attempting to quit continue to smoke because they struggle with adhering 
to their varenicline regimen [12-14]. While there are proven strategies to help people adhere to 
their medications, they have been hard to implement. Digital technology for medication 
adherence has started to emerge [54, 55]; whether these tools can be feasibly implemented and 
adopted, especially within diverse and disadvantaged populations, remains largely unknown.  
In this protocol, we outline the methodology to assess the uptake, usability, acceptability, and 
appropriateness of ChatV and assist in determining whether it is appropriate to advance to an 
RCT to gauge the effectiveness of the intervention. This study will address many limitations of 
previous mHealth trials [56, 57] including using an intervention that clearly outlines the 
behaviour-change theory used, and intervention content. In addition, this study provides well 
outlined a priori criteria for progressing to an RCT. This will enhance its replicability. 

14 
 
Furthermore, this study will provide insights into potential mechanisms underlying the 
intervention's impact, shedding light on the effectiveness, or ineffectiveness, of its various 
components and offering possible explanations. 
In summary, the results of this feasibility study will provide much needed data and insight into 
the potential implementation of ChatV to augment adherence with smoking cessation 
medications. In addition, this project has the potential to increase our understanding of how 
digital tools can improve adherence to other medications in a cost-effective manner. 

15 
 
Acknowledgements 
This research is funded by a Proof of Concept Intervention Grant in Primary Prevention of 
Cancer (Action Grant) of the Canadian Cancer Society and the Canadian Institutes of Health 
Research-Institute for Cancer Research (grant #707218) and a Canadian Institutes of Health 
Research Project Grant: Funding Reference Number: PJT 180405 (Multimedia Appendix 1). 
Thank you to MEMOTEXT for their contribution to the development of the ChatV healthbot.  
Data Availability 
The data sets generated during this study will be available from the corresponding author upon 
reasonable request. 
Authors’ contribution  
NM conceived the study with the support of JR, PS, SV, LZ, MR. 
NM (lead), KM, ME, SH, RT, PS, have shaped the intervention (healthbot). 
NM (lead), KM, ME and were involved in gaining ethical approval.  
NM and KM wrote the first draft of the manuscript.  
All authors reviewed and edited the manuscript and approved the final version of the manuscript.   
Conflict of interests 
The authors declare that they have no conflicting interests with respect to this manuscript; 
however some authors have general disclosures to report. NM reports receipt of funding from 
Canadian Institutes of Health Research, Canadian Cancer Society, and from the Discovery Fund 
of CAMH. PS reports receipt of funding from Canadian Institutes of Health Research, Canadian 
Cancer Society, Pfizer Inc., Pfizer Canada, and Ontario Lung Association. PS also reports that 
through an open-tender process, Johnson & Johnson, Novartis, and Pfizer Inc. are vendors of 

16 
 
record for having provided smoking cessation pharmacotherapy for research studies at free or 
discounted rates. PS holds the Vice-Chair, Research and Giblon Professor in Family Medicine 
Research, a University Named Professorship at the University of Toronto. OM reports receiving 
grant and salary support from the Centre for Addiction and Mental Health and the Department of 
Family and Community Medicine at the University of Toronto through the New Investigator 
Award. 
Abbreviations  
RCT: Randomized Controlled Trial 
CAMH: Centre for Addiction Mental Health 
STOP: Smoking Treatment for Ontario Patients 
REB: Research Ethics Board  
AIM: Acceptability of Intervention Measure 
IAM: Intervention Appropriateness Measure 
SUS: System Usability Scale 
TLFB: Timeline Follow-Back 
 
 
 

17 
 
References 
1. 
Sabaté E. Adherence to long-term therapies: evidence for action: World Health 
Organization; 2003. ISBN: 9241545992. 
2. 
Mongkhon P, Ashcroft DM, Scholfield CN, Kongkaew C. Hospital admissions associated 
with medication non-adherence: a systematic review of prospective observational studies. BMJ 
quality & safety. 2018;27(11):902-14. 
3. 
Osterberg L, Blaschke T. Adherence to medication. New England journal of medicine. 
2005;353(5):487-97. 
4. 
National Center for Chronic Disease Prevention and Health Promotion (US) Office on 
Smoking and Health. The health consequences of smoking—50 years of progress: a report of the 
Surgeon General.: Atlanta, GA: US Department of Health and Human Services, Centers for 
Disease Control and Prevention (US). 2014. 
5. 
Office of the Surgeon General OoS, Health. Reports of the Surgeon General,. The health 
consequences of smoking: a report of the Surgeon General. Atlanta (GA): Centers for Disease 
Control and Prevention (US). 2004. 
6. 
Actions on Smoking and Health (ASH). Briefing: Health Inequalities and Smoking 2019. 
2022; Available from: https://ash.org.uk/wp-content/uploads/2019/09/ASH-Briefing_Health-
Inequalities.pdf. 
7. 
Reid JL HD TU, Burkhalter R, Rynard VL, Douglas O. Tobacco Use in Canada: Patterns 
and Trends. Propel Centre for Population Health Impact, University of Waterloo. 2019. 
8. 
Statistics Canada. Table 13-10-0096-10  Smokers, by age group. 2022; Available from: 
https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310009610. 
9. 
Janz T. Current smoking trends: Statistics Canada; 2012. 

18 
 
10. 
U. S. Public Health Service. The Health Benefits of Smoking Cessation: A Report of the 
Surgeon General. U.S. Public Health Service, Office of the Surgeon General & Office on 
Smoking and Health. 1990. 
11. 
Cahill K, Stevens S, Perera R, Lancaster T. Pharmacological interventions for smoking 
cessation: an overview and network meta‐analysis. Cochrane database of systematic reviews. 
2013 (5). 
12. 
Kaur K, Kaushal S, Chopra SC. Varenicline for smoking cessation: a review of the 
literature. Current therapeutic research. 2009;70(1):35-54. 
13. 
Pacek LR, McClernon FJ, Bosworth HB. Adherence to pharmacological smoking 
cessation interventions: a literature review and synthesis of correlates and barriers. Nicotine and 
Tobacco Research. 2018;20(10):1163-72. 
14. 
Liberman JN, Lichtenfeld MJ, Galaznik A, Mastey V, Harnett J, Zou KH, et al. 
Adherence to varenicline and associated smoking cessation in a community-based patient setting. 
Journal of Managed Care Pharmacy. 2013;19(2):125-31. 
15. 
Catz SL, Jack LM, McClure JB, Javitz HS, Deprey M, Zbikowski SM, et al. Adherence 
to varenicline in the COMPASS smoking cessation intervention trial. Nicotine & Tobacco 
Research. 2011;13(5):361-8. 
16. 
Peng AR, Morales M, Wileyto EP, Hawk Jr LW, Cinciripini P, George TP, et al. 
Measures and predictors of varenicline adherence in the treatment of nicotine dependence. 
Addictive behaviors. 2017;75:122-9. 
17. 
Peng AR, Schnoll R, Hawk Jr LW, Cinciripini P, George TP, Lerman C, et al. Predicting 
smoking abstinence with biological and self-report measures of adherence to varenicline: Impact 
on pharmacogenetic trial outcomes. Drug and alcohol dependence. 2018;190:72-81. 

19 
 
18. 
Peng AR, Swardfager W, Benowitz NL, Ahluwalia JS, Lerman C, Nollen NL, et al. 
Impact of early nausea on varenicline adherence and smoking cessation. Addiction. 
2020;115(1):134-44. 
19. 
Zeng F, Chen C-I, Mastey V, Zou KH, Harnett J, Patel BV. Effects of copayment on 
initiation of smoking cessation pharmacotherapy: an analysis of varenicline reversed claims. 
Clinical therapeutics. 2011;33(2):225-34. 
20. 
Crawford G, Weisbrot J, Bastian J, Flitter A, Jao NC, Carroll A, et al. Predictors of 
varenicline adherence among cancer patients treated for tobacco dependence and its association 
with smoking cessation. Nicotine and Tobacco Research. 2019;21(8):1135-9. 
21. 
Suehs BT, Davis C, Galaznik A, Joshi AV, Zou KH, Patel NC. Association of out-of-
pocket pharmacy costs with adherence to varenicline. Journal of Managed Care Pharmacy. 
2014;20(6):592-600. 
22. 
Shelley D, Tseng T-Y, Gonzalez M, Krebs P, Wong S, Furberg R, et al. Correlates of 
adherence to varenicline among HIV+ smokers. Nicotine & Tobacco Research. 2015;17(8):968-
74. 
23. 
Hollands GJ, Naughton F, Farley A, Lindson N, Aveyard P. Interventions to increase 
adherence to medications for tobacco dependence. Cochrane Database of Systematic Reviews. 
2019 (8). 
24. 
Pavel M, Jimison HB, Wactlar HD, Hayes TL, Barkis W, Skapik J, et al. The role of 
technology and engineering models in transforming healthcare. IEEE reviews in biomedical 
engineering. 2013;6:156-77. 

20 
 
25. 
Aggarwal A, Tam CC, Wu D, Li X, Qiao S. Artificial Intelligence–Based Chatbots for 
Promoting Health Behavioral Changes: Systematic Review. Journal of Medical Internet 
Research. 2023;25:e40789. 
26. 
Bates M. Health care chatbots are here to help. IEEE pulse. 2019;10(3):12-4. 
27. 
Olano-Espinosa E, Avila-Tomas JF, Minue-Lorenzo C, Matilla-Pardo B, Serrano MES, 
Martinez-Suberviola FJ, et al. Effectiveness of a Conversational Chatbot (Dejal@ bot) for the 
Adult Population to Quit Smoking: Pragmatic, Multicenter, Controlled, Randomized Clinical 
Trial in Primary Care. JMIR mHealth and uHealth. 2022;10(6):e34273. 
28. 
Minian N, Mehra K, Rose J, Veldhuizen S, Zawertailo L, Ratto M, et al. Cocreation of a 
conversational agent to help patients adhere to their varenicline treatment: A study protocol. 
DIGITAL HEALTH. 2023;9:20552076231182807. 
29. 
Lyon AR, Munson SA, Renn BN, Atkins DC, Pullmann MD, Friedman E, et al. Use of 
human-centered design to improve implementation of evidence-based psychotherapies in low-
resource communities: protocol for studies applying a framework to assess usability. JMIR 
research protocols. 2019;8(10):e14990. 
30. 
Michie S, Van Stralen MM, West R. The behaviour change wheel: a new method for 
characterising and designing behaviour change interventions. Implementation science. 
2011;6(1):1-12. 
31. 
Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use 
in behaviour change and implementation research. Implementation science. 2012;7:1-17. 
32. 
Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The 
behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an 

21 
 
international consensus for the reporting of behavior change interventions. Annals of behavioral 
medicine. 2013;46(1):81-95. 
33. 
Kelley JF. An iterative design methodology for user-friendly natural language office 
information applications. ACM Transactions on Information Systems (TOIS). 1984;2(1):26-41. 
34. 
Whitehead AL, Sully BG, Campbell MJ. Pilot and feasibility studies: is there a difference 
from each other and from a randomised controlled trial? Contemporary clinical trials. 
2014;38(1):130-3. 
35. 
Fagerström K. Determinants of tobacco use and renaming the FTND to the Fagerström 
Test for Cigarette Dependence. Nicotine & tobacco research. 2011;14(1):75-8. 
36. 
Fucito LM, Toll BA, Salovey P, O'Malley SS. Beliefs and attitudes about bupropion: 
implications for medication adherence and smoking cessation treatment. Psychology of 
Addictive Behaviors. 2009;23(2):373. 
37. 
Fernandez S, Chaplin W, Schoenthaler AM, Ogedegbe G. Revision and validation of the 
medication adherence self-efficacy scale (MASES) in hypertensive African Americans. Journal 
of behavioral medicine. 2008;31:453-62. 
38. 
Tseng T-Y, Krebs P, Schoenthaler A, Wong S, Sherman S, Gonzalez M, et al. Combining 
text messaging and telephone counseling to increase varenicline adherence and smoking 
abstinence among cigarette smokers living with HIV: a randomized controlled study. AIDS and 
Behavior. 2017;21:1964-74. 
39. 
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for 
implementation research: conceptual distinctions, measurement challenges, and research agenda. 
Administration and policy in mental health and mental health services research. 2011;38:65-76. 

22 
 
40. 
Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric 
assessment of three newly developed implementation outcome measures. Implementation 
Science. 2017;12:1-12. 
41. 
Hermes ED, Lyon AR, Schueller SM, Glass JE. Measuring the implementation of 
behavioral intervention technologies: recharacterization of established outcomes. Journal of 
medical Internet research. 2019;21(1):e11752. 
42. 
Brooke J, Jordan PW, Thomas B, Weerdmeester BA, McClelland I. Usability evaluation 
in industry. 1996. 
43. 
Heatherton TF, Kozlowski LT, Frecker RC, Rickert W, Robinson J. Measuring the 
heaviness of smoking: using self‐reported time to the first cigarette of the day and number of 
cigarettes smoked per day. British journal of addiction. 1989;84(7):791-800. 
44. 
Brown RA, Burgess ES, Sales SD, Whiteley JA, Evans DM, Miller IW. Reliability and 
validity of a smoking timeline follow-back interview. Psychology of addictive behaviors. 
1998;12(2):101. 
45. 
U.S. Department of Health and Human Services System Usability Scale (SUS). 2021. 
46. 
Hays JT, Leischow SJ, Lawrence D, Lee TC. Adherence to treatment for tobacco 
dependence: association with smoking abstinence and predictors of adherence. Nicotine & 
Tobacco Research. 2010;12(6):574-81. 
47. 
Nollen NL, Mayo MS, Ahluwalia JS, Tyndale RF, Benowitz NL, Faseru B, et al. Factors 
associated with discontinuation of bupropion and counseling among African American light 
smokers in a randomized clinical trial. Annals of Behavioral Medicine. 2013;46(3):336-48. 

23 
 
48. 
Buchanan TS, Berg CJ, Cox LS, Nazir N, Benowitz NL, Yu L, et al. Adherence to 
varenicline among African American smokers: an exploratory analysis comparing plasma 
concentration, pill count, and self-report. Nicotine & Tobacco Research. 2012;14(9):1083-91. 
49. 
Nollen NL, Cox LS, Nazir N, Ellerbeck EF, Owen A, Pankey S, et al. A pilot clinical trial 
of varenicline for smoking cessation in black smokers. Nicotine & Tobacco Research. 
2011;13(9):868-73. 
50. 
Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for 
the analysis of qualitative data in multi-disciplinary health research. BMC medical research 
methodology. 2013;13(1):1-8. 
51. 
Farmer T, Robinson K, Elliott SJ, Eyles J. Developing and implementing a triangulation 
protocol for qualitative health research. Qualitative health research. 2006;16(3):377-94. 
52. 
O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed 
methods studies. Bmj. 2010;341. 
53. 
Garrison GD, Dugan SE. Varenicline: a first-line treatment option for smoking cessation. 
Clinical therapeutics. 2009;31(3):463-91. 
54. 
Thomson K, Brouwers C, Damman OC, de Bruijne MC, Timmermans DR, Melles M. 
How health care professionals evaluate a digital intervention to improve medication adherence: 
qualitative exploratory study. JMIR Human Factors. 2018;5(1):e8948. 
55. 
Musiimenta A, Tumuhimbise W, Mugaba AT, Muzoora C, Armstrong-Hough M, 
Bangsberg D, et al. Digital monitoring technologies could enhance tuberculosis medication 
adherence in Uganda: Mixed methods study. Journal of clinical tuberculosis and other 
mycobacterial diseases. 2019;17:100119. 

24 
 
56. 
McClure JB, Anderson ML, Bradley K, An LC, Catz SL. Evaluating an adaptive and 
interactive mHealth smoking cessation and medication adherence program: a randomized pilot 
feasibility study. JMIR mHealth and uHealth. 2016;4(3):e6002. 
57. 
Gordon JS, Armin JS, Cunningham JK, Muramoto ML, Christiansen SM, Jacobs TA. 
Lessons learned in the development and evaluation of RxCoach™, an mHealth app to increase 
tobacco cessation medication adherence. Patient education and counseling. 2017;100(4):720-7.