Integrating Patient-Reported Outcomes Into Routine Primary Care: Monitoring Asthma Between Visits
Phase: NA
Conditions: Asthma
Sponsor: RAND

Supplement 1: Trial protocol and statistical analysis plan 
1 
 
2 
PROTOCOL TITLE 
3 
Integrating Patient-Reported Outcomes into Routine Primary Care: Monitoring Asthma Between Visits 
4 
 
5 
FUNDING 
6 
AHRQ (1R18HS026432-01) 
7 
 
8 
VERSION DATE 
9 
03/03/23  
10 
 
11 
I. BACKGROUND AND SIGNIFICANCE  
12 
 
13 
Asthma affects more than 25 million individuals in the United States, and its incidence is 
14 
increasing.1 Uncontrolled asthma causes substantial suffering (disproportionately among 
15 
minorities and those of lower socioeconomic status),2 and often results in the use of emergency 
16 
medical services and/or hospitalization.3 Poor asthma control may significantly degrade quality 
17 
of daily life, impairing daily physical activities, sleep, and ability to work. Yet poor health 
18 
outcomes from inadequate asthma control are preventable, especially with the provision of more 
19 
timely treatment and adherence to clinical practice guidelines that call for clinicians to adjust 
20 
treatment based on serial monitoring of patients’ reported symptoms.1 4-8 In fact, adhering to 
21 
clinical guidelines has been shown to improve health outcomes that are important to patients.7, 9 
22 
In contrast to relying on clinical measures such as lung function (which do not directly reflect 
23 
patients’ lived experience) routine collection of asthma symptoms in the form of patient-reported 
24 
outcomes (PROs) could focus treatment on what matters most to patients–their symptoms and 
25 
quality of life. Unfortunately, intensive symptom monitoring and serial measurement is largely 
26 
not happening.10, 11 Though clinicians are increasingly collecting asthma-related PROs 
27 
immediately prior to a patient visit, there has been little progress in collecting PROs during the 
28 
interval between visits. Interventions that facilitate monitoring of asthma symptoms during this 
29 
interval offer a promising strategy to improve symptom control and quality of life, as well as 
30 
reduce healthcare utilization for patients with asthma. 
31 
 
32 
In our previous AHRQ-funded pilot, we demonstrated the success of a simple health IT-enabled 
33 
practice model developed to facilitate asthma symptom monitoring via a clinically integrated 
34 
mHealth app installed on patients’ smartphones during the interval between subspecialty clinic 
35 
visits in the ambulatory setting. We used an iterative, user-centered design process to ensure our 
36 
health IT tools maximally engage patients between visits, and to identify optimal clinic practice 
37 
workflow to minimize burden on clinicians.12 Preliminary data from our 6-month pilot suggest 
38 
strong engagement: of 26 patients of pulmonologists, 24 (92%) continued to use the app after 4-6 
39 
months, and 18 (69%) had 100% adherence to completion of the weekly questionnaires. Exit 
40 

interviews show positive support, and patients feel that the intervention facilitates symptom 
41 
tracking, self-awareness of symptoms, and more timely access to a clinician when symptoms 
42 
worsen. As smartphone adoption—now exceeding 81% of the U.S. population—continues to 
43 
increase, an mHealth strategy for collecting PROs as we describe is ready for scale and spread. 
44 
Specifically, an intervention comprised of our health IT-enabled practice model is well-suited to 
45 
be adapted to the primary care setting to facilitate PRO collection between visits for a larger 
46 
population of patients with the goal of improving guideline-concordant monitoring of asthma 
47 
symptoms. 
48 
 
49 
We are adapting our novel practice model to the primary care setting within an accountable care 
50 
organization that includes a large and diverse population of asthma patients, including Spanish-
51 
speaking and low health literacy patients. Specifically, we will work with 4 primary care clinics 
52 
affiliated with Brigham Health’s Practice-Based Research Network. To adapt our practice model, 
53 
we will use a novel framework designed specifically to scale and spread healthcare technology 
54 
innovations—Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS). This 
55 
framework emphasizes the importance of simplicity in designing technology and practice models 
56 
to scalability. We will work closely with the primary care clinics to understand their 
57 
requirements, adapt our technology, and simplify workflows as per each dimension of the 
58 
NASSS framework. We will then implement our adapted practice model at participating clinics 
59 
and conduct a rigorous evaluation to understand the impact on key asthma-related health 
60 
outcomes.  
61 
 
62 
If successful, this project will provide new knowledge about how to scale and spread a health IT-enabled 
63 
practice model that incorporates the collection asthma-related PROs into routine primary care, the 
64 
setting in which most asthma patients are treated. This work is innovative because it adapts an existing 
65 
and proven health IT-enabled practice model that leverages widespread adoption of smartphones 
66 
among patients and is guided by a novel framework specifically designed to scale and spread health 
67 
technology innovations. Furthermore, our findings will produce rigorous evidence regarding impact, 
68 
thereby providing a foundation for developing a set of best practices to scale and spread this or similar 
69 
health IT-enabled practice models for between-visit collection of PROs for patients with asthma as well 
70 
as other conditions. 
71 
 
72 
II. SPECIFIC AIMS  
73 
 
74 
1. Adapt our existing health IT-enabled practice model for asthma symptom monitoring using 
75 
patient-reported outcome (PROs) to a primary care population. 
76 
 
77 
2. Implement the adapted health IT-enabled practice model in 7 primary care community 
78 
clinics, identify a cohort of eligible asthma patients to participate, and train clinicians and 
79 
clinical staff. 
80 
 
81 
3. Rigorously evaluate the impact of this new health IT-enabled practice model using a 
82 
randomized controlled trial study in which we enroll up to 750 asthma patients (375 
83 
intervention, 375 usual care) by primary care clinician.  
84 
 
85 

 
86 
III. SUBJECT SELECTION  
87 
 
88 
Phase I – Design Sessions 
89 
During Phase I, any male or female, English-speaking patient (>18 years of age) able to provide 
90 
verbal consent will be eligible. Any male or female healthcare provider (physicians and nurses) 
91 
who is at least 18 years of age is eligible to participate. Additionally, we will recruit a cohort of 
92 
people during Phase I to participate in further validation of the Asthma Control Measure (ACM).  
93 
 
94 
We will use four different strategies to recruit patients for design sessions during Phase I: 
95 
1. PCPs Primary care physicians will give their approval to contact patients on their 
96 
respective panels; 
97 
2. Medical directors will give their approval to contact patients in their respective clinics;  
98 
3. Care team members will give approval to recruit patients from Medicine floors who are 
99 
nearing discharge to home; and 
100 
4. IRB-approved flyer or Rally advertisement. 
101 
 
102 
We will work with clinic leads to identify physicians. Research assistants, trained in the 
103 
protection of human subjects, will identify eligible patients by looking through participating 
104 
physicians’ panels for patients who have asthma and using a purposeful sample. The study team 
105 
will then ask for permission from the physician or the clinic medical director before contacting 
106 
the patient. Participating BWH/BWFH physicians may also recommend patients from their 
107 
respective panels who may be eligible. The PI (Dr. Dalal) will then mail the patients two letters 
108 
(one signed by the patient’s primary care physician or clinic medical director and one signed by 
109 
the PI) inviting them to join the study.  
110 
 
111 
We will also approach hospitalized patients admitted with asthma or asthma-comparable illness 
112 
(e.g., COPD) from Medicine floors who are nearing discharge to home. Patients will be 
113 
identified from administrative databases and recruited in-person after getting approval from the 
114 
patient’s nurse or member of the care team  
115 
 
116 
ACM validation survey participants will be recruited through the Rally website. 
117 
 
118 
Phase II - RCT 
119 
During Phase II, we will recruit participants according to the inclusion and exclusion criteria specified 
120 
below. 
121 
 
122 
Inclusion Criteria 
123 
 
English- or Spanish-speaking  
124 
 
>18-years of age 
125 
 
Regularly uses (most days) a compatible smartphone (iOS or Android) 
126 
 
Able to provide consent 
127 
 
Seen for primary care within the last 12 months at: 
128 
o REDACTED 
129 
 
At least one visit coded for asthma within 12 months prior to screening 
130 

 
131 
Exclusion Criteria 
132 
 
Deemed inappropriate for study, per judgment of BWH/BWFH primary care provider 
133 
 
Unable to provide consent 
134 
 
135 
We will use any of the following strategies to recruit patients during Phase II: 
136 
1. We will directly contact eligible patients who have opted-in via the Partners RODY 
137 
(Research Opportunities Direct to You) program 
138 
2. We will obtain approval from Brigham Health’s Primary Care Practice-Based Research 
139 
Network (PBRN) Director to contact patients affiliated with each participating primary 
140 
care clinic. Medical directors of each clinic have agreed to this process and primary care 
141 
physicians will be provided a 7-day period to opt-out of having their patients contacted. 
142 
3. Care team members will give approval to recruit patients from Medicine floors who are 
143 
nearing discharge to home. 
144 
4. IRB-approved flyer or Rally advertisement. 
145 
 
146 
The research team will advertise the study using approved flyers and web advertisements on the 
147 
Rally website to invite interested patients to contact us. We will also identify eligible asthma 
148 
patients of participating clinics through review of administrative databases. PCPs may also elect 
149 
to identify patients from their respective panels who have a diagnosis of asthma and meet our 
150 
inclusion criteria. The PI (Dr. Dalal) will send the MyChart Recruitment Message to eligible 
151 
patients through Patient Gateway, a PeC approved recruitment process. Additionally, eligible 
152 
patients may be sent recruitment letters (from the PI and the PBRN Director) via mail inviting 
153 
them to participate. We may also approach patients in-person in outpatient settings, after getting 
154 
approval from the patient’s PCP or the Clinic Medical Director or PBRN Director. Finally, we 
155 
may approach hospitalized patients with a diagnosis of asthma who are nearing discharge to 
156 
home. Patients will be identified from the EHR and/or administrative databases and recruited in-
157 
person after getting approval from the patient’s nurse or member of the care team.  
158 
 
159 
IV. SUBJECT ENROLLMENT  
160 
 
161 
Phase I – Design Sessions 
162 
A research assistant will follow up with a phone call to patients. If the patient agrees to 
163 
participate, the research assistant will schedule the interview. For hospitalized patients admitted 
164 
with asthma or asthma-comparable illness (e.g., COPD) from Medicine floors who are nearing 
165 
discharge to home, the nurse or care team member will introduce the study staff to the patient 
166 
and gain permission of the patient to be approached by study staff. If the patient agrees to 
167 
participate, we will verbally consent them for participation.  
168 
 
169 
Prior to each interview, all physicians, nurses, and patient subjects will be read consent language 
170 
and asked to provide verbal consent as well as permission to have the interview audio recorded. 
171 
Thus, for phase I patient subjects will be recruited if their physicians or clinic medical 
172 
directors provide permission, are informed, and agree to participate in this study.  
173 
 
174 
Patient demographic data will be collected including age, gender, and race/ethnicity. Patients 
175 
will be informed that they may decline or opt-out from participating at any time (see below). 
176 

Clinician demographic data will be collected including age, gender, race/ethnicity, years of 
177 
experience, types of position in health care, years working in facility, education, etc. Clinicians 
178 
will be informed that they may decline or opt-out from participating at any time.\ 
179 
 
180 
ACM validation survey participants will be recruited through Rally to participate in a two short 
181 
surveys by email (via REDCap). The surveys contain a total of 10 questions and should take less 
182 
than five minutes to complete. ACM validation survey participants will be provided with a 
183 
consent statement to read at the start of the REDCap survey. 
184 
 
185 
Phase II - RCT 
186 
A research assistant will follow-up with eligible patients by phone if necessary. In some cases, 
187 
we may approach hospitalized patients with a diagnosis of asthma who are nearing discharge to 
188 
home. Patients will be identified from the EHR and/or administrative databases and recruited in-
189 
person after getting approval from the patient’s nurse or member of the care team. The nurse or 
190 
care team member will introduce the study staff to the patient and gain permission of the patient 
191 
to be approached by study staff.  
192 
 
193 
Prior to randomization and enrollment in the study, patients will read the consent form in 
194 
REDcap and provide oral consent via phone to participate.  
195 
 
196 
All potentially eligible participants will complete a pre-screening questionnaire over the phone, 
197 
in-person, or through REDCap. Potential participants that pass the screening questions will be 
198 
asked to answer additional questions through REDcap, over the phone, or in-person about their 
199 
asthma (e.g., asthma-related quality of life, patient activation, and health literacy) as well as 
200 
demographics and healthcare utilization to inform eligibility for the study. Eligible patients who 
201 
meet the study criteria will then contacted and consented by phone or in-person, and then 
202 
randomized to the intervention or control group.   
203 
 
204 
For patients randomized to the intervention group, a member of the study team will provide 
205 
assistance with installing and using the mHealth app as required. Intervention patients will be 
206 
prompted via the app on a weekly basis to complete a 5-item questionnaire about their asthma 
207 
symptoms (the Asthma Control Measure). If the patients’ symptom score meets specific criteria 
208 
for severity, the app will allow the patient to request a call from a nurse in the clinic. The app 
209 
will also allow the patient to enter notes, peak flows, and to review previous symptoms. The app 
210 
will also provide other relevant health information such as educational videos. The patient-
211 
reported data will not be part of the legal electronic health record (EHR) but will be accessible 
212 
from the EHR. The app will be approved by RISO.  Patients randomized to usual care will not 
213 
have access to the mHealth app. 
214 
 
215 
V. STUDY PROCEDURES  
216 
 
 
217 
 
Phase I – Design Sessions 
218 
 
In-person interviews or phone will be conducted with participants up to two times with an 
219 
approved member of the research team. Participants will be asked about their experience with 
220 
asthma, how a smartphone app might be helpful to them, and about our materials and workflows 
221 
for the RCT. Some patients may be asked to complete the ACM, the Mini Asthma Quality of 
222 

Life Questionnaire (Mini AQLQ), Short Literacy Survey, Patient Activation Measure, RCT 
223 
screening, and other surveys by email (via REDCap). Each meeting will last approximately one 
224 
hour. 
225 
 
226 
Additionally, we will administer the ACM and Asthma Control Questionnaire by email (via 
227 
REDCap) with up to 400 people. 
228 
 
 
229 
 
Phase II - RCT 
230 
 
Training.  At the beginning of the intervention, patients will be trained to use the app via 
231 
a brief in-app video that explains the key features. We will train the nurses in charge of 
232 
responding to the notifications to use the aggregate dashboard to view their panel of enrolled 
233 
patients (Appendix A). They will follow a triage script developed by our clinical team for our 
234 
previous pilot. Finally, we will train the participating physicians to access the PRO data in the 
235 
EHR from the patient’s chart (Figure 2). We expect this training to be minimal.  
236 
 
 
237 
 
Usual Care. Patients randomized to usual care will continue to receive care as prescribed 
238 
by their PCP; they will not have access to the mHealth app.  
239 
 
240 
 
Randomization.  Eligible patients will be randomly allocated to the intervention or usual 
241 
care. The random assignment will be made by computer-generated randomization to each 
242 
treatment group by permuted blocks of a size of 2 within each clinician, ensuring that for all 
243 
consecutive blocks of a size of 2 within each clinician, 1 subject will be randomized to the 
244 
intervention and 1 to usual care.  
245 
 
246 
Participants assigned to the control arm will be mailed asthma educational materials currently used at 
247 
BWH. Participants assigned to the intervention arm will download the ASTHMA app to their 
248 
smartphone. This app will send them notifications every week to prompt them to answer 5 standardized 
249 
questions about their asthma symptoms. If their responses indicate severe or worsening symptoms, 
250 
patients will be offered the option to request a call from a nurse within their clinic. The app will also 
251 
contain a graph of participants’ previous questionnaire responses, links to asthma educational materials, 
252 
and it will allow patients to record notes and asthma triggers. The data will be available to the 
253 
participants’ providers from within their chart in Epic.  
254 
 
255 
Data Collection. We will collect data on asthma-related healthcare utilization during the prior 12 months 
256 
from our EHR and administrative sources. The asthma quality of life questionnaire will be administered 
257 
through REDCap at baseline, 6-months, and 12-months post-randomization. Patient demographics, the 
258 
patient activation measure (PAM-13), and the short literacy survey (SLS) will be administered at baseline 
259 
only. We will also collect data on asthma-related healthcare utilization during the 12-month study 
260 
period from the EHR and administrative sources. 
261 
 
262 
 
263 
Figure 1. Study Schedule 
264 
 
Screen 
Baseline 
Midpoint 
Endpoint 

 
Month 
0 
Month 0 
Month 6 
Month 12 
Administered with Patient 
Inclusion/Exclusion Criteria 
X 
 
 
 
Informed Consent 
X 
 
 
 
Demographics 
X 
 
 
 
Randomization 
 
X 
 
 
Training on ASTHMA App 
 
  X* 
 
 
Mini AQLQ 
 
X 
X 
X 
PAM-13 
 
X 
 
 
SLS 
 
X 
 
 
Collected from EHR and administrative sources 
Asthma-related healthcare utilization 
X 
 
 
X 
*For participants randomized to intervention group only  
265 
 
266 
We will also conduct phone interviews a sample of patients and clinicians about their experience 
267 
with the intervention using the same procedures as we used for design sessions in Phase I. 
268 
 
269 
VI. BIOSTATISTICAL ANALYSIS  
270 
 
271 
 
Phase I – Design Sessions 
272 
Interviews with clinicians and patients will be used to inform the design of the smartphone app 
273 
and clinician dashboard. We will use NASSS, a novel framework specifically designed for 
274 
developing and evaluating the scale and spread of technology-supported healthcare interventions. 
275 
Consistent with our experience, NASSS emphasizes the importance of simplicity to scale and 
276 
spread (e.g., interoperable technology, minimal training required, minimal changes in staff roles 
277 
or workflows). We will therefore use this framework to systematically simplify as many 
278 
dimensions as possible as we adapt and scale-up the practice model.   
279 
 
280 
 
Phase II - RCT 
281 
 
RCT Statistical analysis plan  
282 
 
 
283 
 
Overview: 
284 
 
For our primary analysis, we will test the hypothesis that asthma patients who use a 
285 
clinically integrated asthma symptom monitoring smartphone application (intervention arm) will 
286 
have greater improvement in asthma-related patient-reported quality of life over the course of the 
287 
~12-month study compared with patients who do not use the smartphone application (control 
288 
arm). We will conduct secondary analyses of healthcare utilization, and exploratory analyses (see 
289 

below). We will use a generalized estimating equations (GEE) z-test (used for all types of data), 
290 
which accounts for clustering by primary care provider (PCP).  
291 
 
 
292 
 
Measures: 
293 
Primary outcome: The primary outcome will be change in the 15-item MiniAQLQ between baseline 
294 
measure and final measure. We define each subject’s baseline date as the date they completed the 
295 
baseline MiniAQLQ. Subjects who were randomized but did not complete the baseline MiniAQLQ will be 
296 
excluded from the primary / secondary analyses.  Additionally, subjects who completed the baseline but 
297 
not the final outcome measure will be excluded from the primary analysis. Thus, our primary analysis 
298 
will only include participants who have a completed both the baseline and final outcome measure. The 
299 
values of the MiniAQLQ range from 1 (worst quality of life) to 7 (highest quality of life). We will analyze 
300 
MiniAQLQ as a continuous variable. We will administer this measure to all study participants (English 
301 
and Spanish versions) at the beginning of the study, at 6 months, and at 12 months. This measure is 
302 
validated; takes about 4 minutes to administer, and offers a comprehensive assessment of quality of life 
303 
across four sub-domains: symptoms (5 items), activity limitations (4 items), emotional function (3 
304 
items), and exposure to environmental stimuli (3 items).1 2 The MiniAQLQ score is calculated as an 
305 
average across these domains, and the minimum clinically important difference is 0.5.3 In a prior 
306 
mHealth study, mean MiniAQLQ scores increased from 4.3 to 4.8 post-intervention.4  
307 
 
308 
Secondary outcome. The secondary outcome is non-routine asthma-related utilization, measured as the 
309 
number of asthma-related emergency department (ED) visits, urgent care visits, and hospitalizations 
310 
identified in the electronic health record (EHR) during the 12-month study period. From the EHR, we will 
311 
include:  
312 
 
Hospitalization: Primary or secondary encounter codes includes asthma 
313 
 
ED: Primary or secondary encounter codes includes asthma 
314 
 
UC: Primary or secondary encounter codes includes asthma 
315 
 
 
316 
Exploratory outcomes: See exploratory analyses below. 
317 
 
318 
Baseline demographics and characteristics of patient participants: From EHR: age, sex assigned at birth, 
319 
race/ethnicity, marital status, primary language, education, socioeconomic status (median income by zip 
320 
code), insurance status, clinic, PCP type, patient portal status, smoking status, visits to subspecialist 12 
321 
M before baseline, number of asthma medications at time of enrollment, presence or environmental or 
322 
seasonal allergy or allergic rhinitis comorbidity based on problem list at time of baseline. From REDcap: 
323 
phone type (e.g., iPhone, Android), baseline ACM score, baseline PAM score, baseline Health Literacy 
324 
Score (SLS), study start date, time between completion of baseline and final surveys. We will also count 
325 
                                                            
1 Wilson SR, Rand CS, Cabana MD, Foggs MB, Halterman JS, Olson L, Vollmer WM, Wright RJ, Taggart V. 
Asthma outcomes: quality of life. Journal of Allergy and Clinical Immunology. 2012;129(3):S88-S123. 
2 Juniper E, Guyatt G, Cox F, Ferrie P, King D. Development and validation of the mini asthma quality of life 
questionnaire. European Respiratory Journal. 1999;14(1):32-8. 
3 Ibid. Juniper et al. (1999) 
4Licskai CJ, Sands TW, Ferrone M. Development and pilot testing of a mobile health solution for asthma self-
management: asthma action plan smartphone application pilot study. Canadian respiratory journal. 2013;20(4):301-
6. 

the number of unique PCP’s with at least one patient in each arm and average number of patients per 
326 
PCP in each arm.  
327 
 
 
328 
 
Primary and secondary outcome analysis:  
329 
 
For our primary analysis, to estimate differences between intervention and control groups 
330 
in our primary outcome (change in MiniAQLQ score) we will use robust linear regression 
331 
models (generalized estimating equations [GEE]) with treatment arm as the only covariate, and 
332 
clustering by PCP using GEE. Linear regression using GEE to compare mean scores between 
333 
treatment and control groups is robust to non-normality of the outcomes. While we anticipate 
334 
that baseline covariates (patient characteristics) will balance out across the two arms due to 
335 
randomization, we may adjust for baseline characteristics that differ across arms for our primary 
336 
and secondary analyses.     
337 
 
 
338 
 
Baseline Data: 
339 
 
We will summarize all data using simple descriptive statistics (means with standard 
340 
deviations for continuous variables, frequencies with percentages for categorical variables) 
341 
overall and in each arm. We will then summarize baseline characteristics among 1) all enrolled 
342 
patients in each group, 2) enrolled patients in each group who completed 12M surveys (for 
343 
primary outcome only), and 3) enrolled patients in each group with missing 12M surveys (for 
344 
primary outcome only). 
345 
 
 
346 
 
Missing data: 
347 
As per Jakobsen et al.,5 if there are less than 5% missing data, analysis of observed data (dropping 
348 
subjects with missing data, also called a complete case analysis) will not lead to any bias. Also if there is 
349 
> 5% missing data but are no significant differences across arms for distribution of follow up times (i.e., 
350 
between baseline and completion of final primary outcome (MiniAQLQ)), the complete case analysis (in 
351 
which we adjust for baseline covariates that differ b/w arms) is most appropriate along the guidelines 
352 
given in Jakobsen et al. (2017). Furthermore, we will conduct a comparative analysis of the distribution 
353 
of follow-up times (i.e., between baseline and completion of final primary outcome (MiniAQLQ) in the 
354 
intervention and control arms. If we do not observe a significant difference across arms, then the 
355 
complete case analysis is the most appropriate (with respect to power and bias). If there are significant 
356 
differences (p<0.05) across arms for the distribution of follow up times, we will add follow-up time as a 
357 
covariate the primary outcome. In essence, the treatment effect in the model for the primary outcome 
358 
is adjusted for differences in follow-up times across groups. Furthermore, in that case, we will also 
359 
calculate adjusted MiniAQLQ (primary outcome) means in the two groups, which will be adjusted to the 
360 
average follow-up time across the two groups combined. 
361 
 
 
362 
 
Secondary outcomes will be derived from the EHR – we will have all within-system EHR 
363 
data for all patients in the study and therefore no missing data; secondary outcome data from 
364 
outside health systems will not be included for the secondary outcomes analyses (see exploratory 
365 
analyses), but we have no reason to believe such data will be different between intervention and 
366 
                                                            
5 Jakobsen, J.C., Gluud, C., Wetterslev, J. et al. When and how should multiple imputation be used for handling 
missing data in randomised clinical trials – a practical guide with flowcharts. BMC Med Res Methodol 17, 162 
(2017). https://doi.org/10.1186/s12874-017-0442-1 

control arms. 
367 
 
 
368 
 
Power and sample size:  
369 
 
We computed our power calculation based on the original proposed number of clinics 
370 
(originally proposed: 4 clinics; final number included in study: 7 clinics), the number of patients 
371 
recruited (originally proposed: 500 patients; final number recruited and randomized: 413 
372 
patients), and the number of PCPs with at least one patient in the trial (originally proposed: 75; 
373 
final number included in study: 104). We estimated a cluster size of 10 asthma patients per PCP.  
374 
 
For our primary outcome, asthma-related quality of life, we estimate mean MiniAQLQ 
375 
scores of 4.3 at baseline. We conservatively estimate the intra-cluster (clinic) correlation 
376 
coefficient (ICC) between MiniAQLQ scores from two patients with the same clinic to be 
377 
approximately 0.01, commonly used for clustering of patients by PCP in this type of 
378 
randomization study. We also assume that the SD for a patient’s MiniAQLQ score is 1.0. Using 
379 
a robust GEE z-test for ordinal categorical data (GEE does not assume normality of the 
380 
outcome), and accounting for clustering of patients by PCP, we will have >90% power (alpha = 
381 
0.05) to detect a 0.5 point increase in the mean MiniAQLQ score (i.e., from 4.3 to 4.8 favoring 
382 
the intervention). (Note: even though we expect the mean MiniAQLQ score to be approximately 
383 
4.3 in the control group, we will have >90% power to detect a 0.5 point decrease in mean 
384 
MiniAQLQ score regardless of the mean MiniAQLQ score in the control group (i.e., from 5.1 to 
385 
5.6 favoring the intervention)).  
386 
 
For our secondary outcome, asthma-related healthcare utilization, we estimate a 
387 
minimum of 10 ED visits (excluding hospitalizations and urgent care visits) per 100 asthma 
388 
patients per year at baseline from national data sources.(1) Using a GEE Poisson z-test for rates 
389 
(GEE is still unbiased even if the data are not Poisson) and accounting for clustering of patients 
390 
by PCP (ICC = 0.01), we will have 80% power (alpha = 0.05) to detect a 50% decrease in the 
391 
rate of ED visits per year (i.e., from 10 to 5 per 100 asthma patients per year). 
392 
 
 
393 
 
Sensitivity analyses:  
394 
 Redo the primary analysis using imputation of missing 12M MiniAQLQ. Our imputation 
395 
method will be to use the latest MiniAQLQ score completed, which might be the 12M, 
396 
6M or baseline.  
397 
 Redo the secondary outcome analysis with the subset of patients who completed the 12M 
398 
MiniAQLQ.  
399 
 Redo the secondary outcome analysis including any utilization from external to the health 
400 
system identified in the 6- and 12-month REDcap surveys. 
401 
 
 
402 
 
Exploratory analyses: 
403 
 Redo the primary analysis for each of the 4 sub-domains of the MiniAQLQ as outcome 
404 
measures: symptoms (5 items), activity limitations (4 items), emotional function (3 
405 
items), and exposure to environmental stimuli (3 items). 
406 
 Impact of intervention on outpatient utilization (visits and portal messages): As 
407 
exploratory outcomes, we will compute the number of asthma-related outpatient visits 
408 
within the MGB system as identified in the EHR during the study period. We will count 
409 
any visit with an asthma-related billing code. We will compare results between the 
410 
intervention and control. Separately, we will count the number of patient-generated portal 
411 
messages sent by patients during the study period and compare results between the 
412 

intervention and control. 
413 
 Impact of intervention on prednisone prescriptions. We will test the hypothesis that the 
414 
intervention group receives the same level or more prednisone prescriptions compared 
415 
with the control group. We will calculate the 95% confidence interval for the difference. 
416 
We will build GEE linear models with adjustment as described above for our primary 
417 
analysis. 
418 
 Subgroup analyses: To explore how the effect of the intervention on the primary outcome 
419 
varies within relevant population strata, we use regression models to assess the 
420 
intervention’s impact on the following variables:  
421 
o Age at baseline: 18-44 vs 45-64 vs 65+ years  
422 
o Sex: Male vs Female vs. Other  
423 
o Race/ethnicity: Hispanic or Black vs. All other 
424 
o Smoking status: Current vs. All other 
425 
o Patient Activation Measure (PAM): High vs Low 
426 
o Health Literacy (HL): High vs Low 
427 
o Questionnaire completion percentage: High (>50%) vs Low  
428 
o Retention: High (continued using app for 12 M) vs Low 
429 
 The subgroup analyses will be performed separately for each variable, and will fit models 
430 
with interactions between treatment group and the subgroup to explore if the treatment 
431 
effect is different across subgroups (a non-significant interaction implies that the 
432 
treatment effect is similar across subgroups). 
433 
 Repeat measures analysis. For subjects with completed MiniAQLQ data at baseline, 6M, 
434 
and 12M, we will compare change in MiniAQLQ baseline-6M, and 6M-12M for 
435 
intervention and control arms using a differences-in-differences analysis. This will allow 
436 
us to assess the degree to which any impact of the intervention occurred in the first 6M or 
437 
the second 6M of the study period.  
438 
 
 
439 
 
Definition of significance:  
440 
 
We will consider 2-sided P values to be significant at <0.05 for primary and secondary 
441 
analyses.   
442 
 
 
443 
 
Multiple hypothesis testing:  
444 
 
We will consider the intervention successful if the intervention arm shows a statistically 
445 
significant improvement compared to the control arm on the primary outcome (improvement in 
446 
MiniAQLQ over the course of the study period). Therefore, multiple testing is not needed. We 
447 
do not plan to conduct multiple hypothesis testing for the subgroup analyses as their purpose is 
448 
exploratory and for generating hypotheses. 
449 
 
450 
VII. RISKS AND DISCOMFORTS   
451 
 
452 
There are no known risks to participating in this study. Patient and clinician subjects may voluntarily 
453 
participate and withdraw at any time. There is a theoretical risk of breach of data confidentiality and 
454 
that PHI could become known to unauthorized persons, but we will take all steps necessary to protect 
455 
PHI (see below). To mitigate this risk, we will follow all compliance and data confidentiality procedures 
456 
for research at Partners. Specifically, we will use procedures consistent with sound research design and 
457 

which do not expose subjects to unnecessary risk. Data monitoring will be conducted regularly (see 
458 
below). All data analyzed will be de-identified and reported in aggregate. 
459 
 
460 
This is a minimal risk, quality improvement research study. There is no known physical or medical 
461 
safety threat to patient or clinician subjects who participate this quality improvement research study. 
462 
Furthermore, participation in this study is voluntary and all subjects will be informed that they may opt-
463 
out any time (see below). Given the minimal risk to subjects, our data safety and monitoring procedures, 
464 
and the relatively small sample size, we do not plan on having drop criteria. 
465 
 
466 
For patient and clinician subjects, there are no known risks to participating in our data collection 
467 
activities. The main risks are breach of confidentiality and privacy of information shared. Every effort 
468 
will be made to maintain confidentiality and privacy of information shared and collected and 
469 
subsequent data analysis (see below), and efforts will be made to minimize the duration of data 
470 
collection activities.  
471 
 
472 
VIII. POTENTIAL BENEFITS  
473 
 
474 
Patients may benefit by having the opportunity to discuss their experiences with asthma and 
475 
contribution to research.  Based on our previous pilot, ultimately, if successful, patients receiving 
476 
the intervention may receive faster access to care, be more aware of their asthma symptoms, feel 
477 
more connected to their clinician, and have better asthma control, possibly resulting in fewer 
478 
hospitalizations and emergency room visits. 
479 
 
480 
IX. MONITORING AND QUALITY ASSURANCE  
481 
 
482 
The principal investigator and research assistant/project coordinator will monitor data collection 
483 
activities, maintain data integrity and quality control, protect the rights, safety, and welfare of study 
484 
subjects, and adhere to standards set by the Partners IRB. All data will be stripped of PHI and electronic 
485 
data will be kept on a secured shared file area (SFA) behind the Partners firewall with anti-virus software 
486 
or Partners Research Computing approved cloud storage services. Data about consented patients will 
487 
also be stored behind the RAND Corporation firewall in accordance with the terms of the consent. Any 
488 
paper forms and/or audio recordings will be destroyed after being transcribed into electronic format. 
489 
Monitoring will be performed on a monthly basis during the study period and will include a review of 
490 
enrollment, data analysis, and breaches of confidentiality, and any adverse events that may occur. Any 
491 
adverse events will be graded as to their attribution to the intervention and reported according to IRB 
492 
guidelines. The principal investigator will review any complaints reported from patient and clinician 
493 
subjects. These will be reported to the Partners IRB annually or immediately if the complaint is serious. 
494 
Given the minimal risk of the study, we are not planning to use automatic stopping rules or a DSMB. We 
495 
emphasize that as a quality improvement intervention, this study does not involve any invasive 
496 
procedures. 
497 
 
498 

Voluntary participation in data collection activities are not expected to cause adverse events to patient 
499 
or clinician subjects. However, if an adverse event occurs, the principal investigator will follow PHS HRC 
500 
guidelines for adverse event reporting.  
501 
 
502 
The principal investigator and research assistant/project coordinator will monitor and assure the 
503 
validity and integrity of the data collection and adherence to the procedures outlined in this 
504 
protocol. They will review the study database on a monthly basis, ensuring data integrity, 
505 
accuracy, and completeness. The principal investigator will supervise research assistants/project 
506 
coordinators in all data collection activities over the course of the study.  
507 
 
508 
 
 
509 

X. REFERENCES 
510 
1. 
Braido F. Failure in asthma control: reasons and consequences. Scientifica. 2013;2013. 
511 
2. 
Wang T, Srebotnjak T, Brownell J, Hsia RY. Emergency department charges for asthma-
512 
related outpatient visits by insurance status. Journal of Health Care for the poor and 
513 
underserved. 2014;25(1):396. 
514 
3. 
Centers for Disease Control and Prevention. CDC Vital Signs: Asthma in the US. 
515 
http://www.cdc.gov/vitalsigns/asthma/. 
516 
4. 
Gibson P, Powell H. Written action plans for asthma: an evidence-based review of the 
517 
key components. Thorax. 2004;59(2):94-99. 
518 
5. 
Kessler KR. Relationship Between the Use of Asthma Action Plans and Asthma 
519 
Exacerbations in Children With Asthma A Systematic Review. Journal of Asthma & 
520 
Allergy Educators. 2011;2(1):11-21. 
521 
6. 
Lang DM. New asthma guidelines emphasize control, regular monitoring. Cleveland 
522 
Clinic journal of medicine. 2008;75(9):641-653. 
523 
7. 
van der Meer V, van Stel HF, Bakker MJ, et al. Weekly self-monitoring and treatment 
524 
adjustment benefit patients with partly controlled and uncontrolled asthma: an analysis of 
525 
the SMASHING study. Respiratory research. 2010;11(1):74. 
526 
8. 
Wechsler ME. Getting Control of Uncontrolled Asthma. The American journal of 
527 
medicine. 2014. 
528 
9. 
Van Gaalen JL, Beerthuizen T, van der Meer V, et al. Long-Term Outcomes of Internet-
529 
Based Self-Management Support in Adults With Asthma: Randomized Controlled Trial. 
530 
Journal of medical Internet research. 2013;15(9). 
531 
10. 
Okelo SO, Butz AM, Sharma R, et al. Interventions to modify health care provider 
532 
adherence to asthma guidelines: a systematic review. Pediatrics. 2013:peds. 2013-0779. 
533 
11. 
Wisnivesky JP, Lorenzo J, Lyn-Cook R, et al. Barriers to adherence to asthma 
534 
management guidelines among inner-city primary care providers. Annals of Allergy, 
535 
Asthma & Immunology. 2008;101(3):264-270. 
536 
12. 
Rudin RS, Fanta CH, Predmore Z, et al. Core Components for a Clinically Integrated 
537 
mHealth App for Asthma Symptom Monitoring. Appl Clin Inform. Oct 2017;8(4):1031-
538 
1043. 
539 
13. 
Wilson SR, Rand CS, Cabana MD, et al. Asthma outcomes: quality of life. Journal of 
540 
Allergy and Clinical Immunology. 2012;129(3):S88-S123. 
541 
14. 
Juniper E, Guyatt G, Cox F, Ferrie P, King D. Development and validation of the mini 
542 
asthma quality of life questionnaire. European Respiratory Journal. 1999;14(1):32-38. 
543 
15. 
Licskai CJ, Sands TW, Ferrone M. Development and pilot testing of a mobile health 
544 
solution for asthma self-management: asthma action plan smartphone application pilot 
545 
study. Canadian respiratory journal. 2013;20(4):301-306. 
546 
16. 
Anselm S, Corbin J. Basics of qualitative research: Techniques and procedures for 
547 
developing grounded theory. Thousand Oaks, CA: Sage Publications; 1998. 
548 
17. 
Silverman D. Doing qualitative research: A practical handbook: SAGE Publications 
549 
Limited; 2013. 
550 
18. 
Cresswell J. A Concise Introduction to Mixed Methods Research. Los Angeles, CA: Sage 
551 
Publications; 2015. 
552 
 
553