Evaluation of a Mobile Application to Enhance Medication Management Following Hospital Discharge
Phase: NA
Conditions: Medication Adherence, Mobile Application
Sponsor: McGill University

Research Protocol – Mobile App for Medication Management 
1 
 
 
 
 
 
 
 
 
Evaluation of a Mobile Application to Enhance Medication Management Following Hospital Discharge 
Research Protocol 
April 26, 2019 
NCT04676165 
 
 

Research Protocol – Mobile App for Medication Management 
2 
 
Background 
Adverse drug events (ADEs) are among the leading causes of death in North America1,2. Over 2 million 
ADEs occur each year, resulting in 100,000 deaths and $36 million in economic costs3. In recently 
hospitalized patients, 70% of adverse events that occur within 30 days of discharge are medication 
related4,5. Of these, at least 58% are preventable and tend to be the result of prescribing or dispensing 
errors, incomplete drug information, and underuse or overuse of medications6,7. 
A number of studies have demonstrated that patients are usually discharged from hospital on 
substantially different medication regimens than those prior to admission. However, discharge 
prescriptions are typically not well followed in the community post discharge. We have further explored 
this phenomenon through secondary analysis of data from a randomized controlled trial (RCT) recently 
conducted by our study team at the McGill University Health Centre (MUHC), which evaluated the impact 
of electronic medication reconciliation on ADEs post hospital discharge. Our analyses suggested that 54% 
of study patients were non-adherent to at least 1 in-hospital medication change, and that patients who 
were non-adherent to 4 or more in-hospital medication changes experienced a 64% increased risk of 
emergency department visits and hospital re-admissions compared to those who were adherent to all 
changes (Figure 1). Furthermore, analyses of patient interview data suggested that unclear 
communication with patients about medication changes and the reasons for these changes may be driving 
non-adherence, as might difficulties in managing complex drug regimen information and dosing schedules 
(data from in-progress manuscript). There is therefore a clear need to implement and evaluate patient 
support mechanisms that reduce non-adherence to essential changes in therapy following hospitalization. 
Figure 1. Impact of non-adherence to in-hospital medication changes on ED visits and hospital re-
admission 30 days post discharge. 
. 
 
 
 
 
 
 
0
5
10
15
20
25
30
35
40
45
1
2
3
4
% increased risk of ED visit
or readmission
In-hospital medication changes not adhered to per patient

Research Protocol – Mobile App for Medication Management 
3 
 
Objective 
The objective of this project is to evaluate the usability and efficacy of a mobile application aimed at 
improving medication management and subsequent adherence to in-hospital medication changes among 
high risk patients.  
Methodology 
We will conduct an RCT that will assess the usability of a medication management mobile application and 
its efficacy in decreasing non-adherence to in-hospital medication changes among patients in the internal 
medicine ward of the Royal Victoria and Montreal General sites of the MUHC. One hundred patients will 
be randomized 1:1 to either the intervention arm, in which they will receive access to the mobile app, or 
the control arm, in which they will receive usual care (i.e., no medication management support). We will 
conduct a usability assessment at hospital discharge (baseline) and at 1 week post-discharge to obtain 
patient feedback on the app (via a technology acceptance questionnaire) and to document usability using 
the “think aloud” protocol8,9. We will also compare, between the 2 arms, the rates of non-adherence to 
treatment changes in the 30-days post-discharge.  
Study population 
The study population will comprise 100 patients who are discharged from the internal medicine unit of 
the Royal Victoria and Montreal General sites of the MUHC. To be eligible for participation in the trial, 
patients must also be covered by the Quebec public insurance drug plan (i.e. they are 65 years or older, 
are recipients of social assistance, or do not have private drug insurance), must be 18 years or older at the 
time of admission, must own a tablet, smartphone, desktop PC, or portable computer, must have internet 
connection on their tablet/smartphone/laptop/PC, must speak and read either English or French, must 
not be discharged to a rehabilitation or long-term care center, and must have a prognosis of more than 3 
months.  
Patients who are cognitively impaired or otherwise unable to provide consent will be eligible for inclusion 
in the study if surrogate consent is granted by a legally authorized representative and if a caregiver has 
agreed to use the app on the patient’s behalf. Similarly, patients who are completely dependent on a 
caregiver but capable of providing consent will be enrolled only if the caregiver has agreed to use the app.  
Participant recruitment, consent, and enrollment 
Hospital pharmacists working on the internal medicine ward of the Glen site of the MUHC will briefly 
introduce patients to the study. During routine collection of community medication information from 
patients, hospital pharmacists will hand out a flyer that briefly describes the study and will ask patients if 
they would be interesting in learning more about it. If patients express interest, a study field coordinator 
will then approach them to assess eligibility and describe the study in more detail to patients. If patients 
are both eligible and willing to participate, they will be provided with a consent form, which the field 
coordinator will go over in detail, explaining various aspects such as study objectives and procedures, 
participant responsibilities, potential benefits and risks of participating, confidentiality of data collected, 
etc. Patients will be provided an opportunity to ask any questions they might have. If willing to participate, 
they will then be asked to sign the consent form and will be given a copy of the signed form. 
During the enrollment process, patients will also be asked if they wish to authorize a chosen caregiver to 
access the app. If patients do wish to grant access to a caregiver, information on the caregiver will be 
collected (name, gender, date of birth, telephone or mobile number, email address, preferred language) 
and study field coordinators will demonstrate the app and its features to both the patient and caregiver. 

Research Protocol – Mobile App for Medication Management 
4 
 
If a chosen caregiver is not present at the time of discharge, they will be sent an email that describes the 
study and the app, as well as provides a link to access the app.  
If the patient is cognitively impaired or otherwise unable to provide consent, consent will be obtained 
from the patient’s legally authorized representative. In such cases and in cases in which the patient is 
capable of providing consent but completely dependent on a caregiver, consent will also be obtained from 
the caregiver. The caregiver must agree to use the app on the patient’s behalf and to undergo the usability 
assessment at discharge and at 1 week post-discharge. 
Randomization and blinding 
Permuted block randomization with varying block sizes of 2 and 4 will be used to randomize 100 patients 
1:1 to either the intervention or the control group. Random assignment will be conducted using the 
Statistical Analysis Software (SAS). Given the nature of the intervention, study participants and personnel 
will not be blinded to group allocation, but data analysts will be. 
Intervention 
Usual care  
At discharge, patients will be provided with a written discharge prescription to be filled at their community 
pharmacy, and may or may not receive written or verbal instructions about changes made to therapy. 
Intervention arm 
At discharge, patients (and/or caregivers) will receive access to a medication management application 
that has been developed by the McGill Clinical and Health Informatics (MCHI) Research Group. The 
application will retrieve all medications prescribed to patients at discharge from an electronic form with 
structured data fields that has been completed by a study field coordinator using the patient’s discharge 
prescription. The app will then generate a patient-friendly medication list and will offer a number of 
important features aimed at addressing barriers to adherence, including: 
• Pill identification: This feature displays an image of a pill next to each medication in the list, with the 
intention of ensuring patients take the correct medications at the appropriate dosage and at the right 
times. 
• Drug information: Patient-friendly monographs are provided to help patients and caregivers 
understand the indications for treatment, as well as the harms and benefits of their medications. 
Improved health literacy with respect to treatment has the potential to increase appropriate 
utilization of medications by patients. 
• Drug alerts: Using prescription claims data of the 30 days post-discharge, which will be obtained from 
the RAMQ, this feature alerts patients to their non-adherence, such as in cases in which they have 
filled prescriptions that were stopped at hospital discharge, have yet to fill prescriptions for 
medications newly prescribed at discharge, or have been dispensed medications at the incorrect dose. 
• Daily pill reminders: This feature reminds patients to take their medication on the day/time they are 
supposed to. Patients can choose whether or not they want this feature activated. 
• Weekly dosing schedule: This feature presents patients with an easy-to-interpret schedule of which 
medications need to be taken and when (with special instructions, e.g. needs to be taken with food). 
Medication names are displayed with images of the corresponding pill to prevent mix-ups. 
• Symptom checker: This feature allows patients to enter their symptoms into the application and 

Research Protocol – Mobile App for Medication Management 
5 
 
determine which of the medications they are taking has side-effects similar to those they are 
experiencing. 
• Rate my med: This feature allows patients to rate the medications they are taking (1-5 stars) and 
comment on how effective they have been or if they have experienced any negative side-effects. 
Patients can also view ratings given by other patients using the application for medications they have 
been prescribed. 
• Caregiver connect: This feature provides the patient’s caregiver(s) with access to their medication 
lists, as well as notifies them of nonadherence to treatment changes, with the aim of reducing its 
occurrence. Caregivers will have access to all the same information that patients have access to 
through the app. 
• Pharmacy connect: This feature allows patients to connect with their hospital pharmacist and ask any 
questions they might have regarding their discharge prescription, therapy changes, and drug alerts. 
Based on their communication with the patient, hospital pharmacists then have the option to (1) 
recommend a course of action to the patient regarding their medications, (2) consult with the 
attending physician who was on duty at the time of the patient’s discharge regarding a potential 
change in the patient’s prescription, (3) contact the patient’s community pharmacist to inform them 
of any errors in dispensation or any changes made to the patient’s prescription, and (4) contact the 
patient’s primary care physician, if and as needed. 
 
Pilot-testing the app 
Prior to enrolling patients in the study, we will pilot test the app using data from the RightRx study, 
which was also conducted in the internal medicine units of the MUHC (study #10-180 GEN). 
Anonymized data of RightRx patients will be fed into the app to test whether the app appropriately 
lists prescribed medications and generates the appropriate alerts for nonadherence. This process will 
help us identify any bugs in the app so that we can fix them prior to the app’s use in the pilot trial. 
Once the 30-day follow-up period is complete, patients will retain access to the application for 1 year. The 
application will still display medications dispensed to patients (obtained through prescription claims data 
from the RAMQ) to help them keep track of their medications, but adherence alerts will no longer be 
generated and the pharmacy connect feature will be modified so that patients’ messages are faxed to 
their community pharmacist, who they will be advised to contact regarding any questions they have. 
Patients/caregivers will also be able to update their medication list manually. 
Outcome measures 
Usability assessment 
Two measures will be used to conduct usability assessment at baseline (i.e. hospital discharge) and during 
a follow-up visit at 1 week post-discharge. The first, a technology acceptance questionnaire, is a self-
administered questionnaire that patients in the intervention group (or their caregivers) will complete and 
that will assess their satisfaction with the application and collect their feedback on its usability. The second 
will consist of observing patients (or their caregivers) as they use the app, following the “think aloud 
protocol”.8,9 This protocol is widely used to collect data in usability testing10-13 and involves users 
verbalizing their thoughts as they perform a standardized set of tasks, while being audio-recorded. In this 
project, a study field coordinator who is familiar with the mobile application will observe and record 
intervention patients (or their caregivers) as they access the app and complete the following standardized 
tasks: view their medications lists and dosing schedule, peruse drug information, and examine any alerts 
they might have been issued. Each session will then be transcribed and transcriptions will be qualitatively 

Research Protocol – Mobile App for Medication Management 
6 
 
analyzed to assess the following usability concepts: the technology’s (1) ease of use, (2) user-friendliness, 
(3) efficiency, and (4) features that may cause confusion, frustration, or user errors.  
Patients who complete the home visit at 1 week post-discharge will be provided with a gift card in the 
amount of $20 as compensation for their time. 
Non-adherence to treatment changes 
Pharmacy claims data from the RAMQ will be used to identify community medications dispensed to 
participating patients in the 30 days following hospital discharge. These data will be compared with the 
patient’s hospital discharge prescription to identify non-adherence to in-hospital medication changes. 
Non-adherence to in-hospital treatment changes in the 30-day post-discharge period will be defined as 
(1) failure to fill newly prescribed medications, (2) re-filling a previous prescription that was stopped at 
discharge, or (3) filling a previous prescription whose dosage was modified at discharge, but was 
dispensed post-discharge at the incorrect daily dose. 
Data sources 
Prescription claims data from the Régie de l’assurance maladie du Québec (RAMQ) will be used to identify 
prescriptions filled in the 3 months prior to admission and the 1 year following discharge.  
Medications prescribed at discharge and in-hospital medication changes will be determined using 
patients’ discharge prescriptions, which will be obtained from patients’ hospital charts. 
Data Analysis  
Descriptive statistics will be used to summarize demographic and clinical characteristics of the study 
population, results of the technology assessment questionnaire, and the frequency of nonadherence 
events (as defined in the previous section) in the 30 days post-discharge, overall and in each study group. 
We will then compare, between intervention and control groups, the average number of in-hospital 
medication changes not adhered to in the 30-day follow-up period using the Student’s t-test. All 
quantitative statistical analyses will be conducted using SAS version 9.4. Data from the “think aloud” 
protocol will be analyzed qualitatively.  
Project timeline 
The RCT will be conducted at the internal medicine unit of the Royal Victoria and Montreal General sites 
of the MUHC over a period of 1.5 years (Figure 2). Following approval of the study by the MUHC’s 
institutional review board, study coordinators will begin recruiting, consenting, enrolling, and randomizing 
eligible patients who are admitted to the internal medicine unit of the MUHC Glen site. Based on 
recruitment rates from our recently completed trial which was also conducted at the MUHC, we expect 
this process to take approximately 5-6 months. At discharge, the study coordinator will provide enrolled 
patients in the intervention arm with access to the mobile application, enter their discharge prescription 
into the app, acquaint them with and train them in the various features of the app, and conduct a baseline 
usability assessment.  
One week following discharge, study coordinators will conduct usability assessments at patients’ homes. 
Assessment of non-adherence will be conducted by a data analyst at the MCHI Research Group using data 
from the RAMQ feed and patients’ discharge prescriptions. Subsequent data analyses will be conducted 
over a period of 4 months, beginning 30 days following the enrollment of the last study patient. All data 
will be stored on a secure server at the MCHI centre, which can only be accessed using employee 

Research Protocol – Mobile App for Medication Management 
7 
 
identification cards. Following data analysis, an evaluation report will be disseminated to relevant 
stakeholders at the MUHC (e.g. Committee of Quality and Risk), and a manuscript will be prepared for 
potential publication in a peer-reviewed journal. 
Figure 2. Gantt Chart. 
 
Data storage, privacy, and confidentiality 
An account will be created for each newly consented and enrolled patient using their first and last names. 
Data collected via the mobile application and acquired from the RAMQ and patients’ hospital records will 
be stored on secure servers at the McGill Clinical and Health Informatics Research Group with daily 
backup. Information collected on caregivers (name, gender, date of birth, telephone or mobile number, 
email address, preferred language) will also be stored on these secure servers. Patient identifying 
information will be replaced with unique encrypted numbers and the file linking these unique numbers to 
patients’ names will be accessible only by the Principal Investigator (Robyn Tamblyn) and the system 
administrator. All research staff involved in this project and who have access to study data will sign 
confidentiality agreements. 
No data or information from the app will be accessible by third parties; only the patient, their chosen 
caregiver, the hospital pharmacist, Dr. Tamblyn, and Dr. Tamblyn’s research team will have access to 
data in the app. Furthermore, the app will not be able to access any other apps on the patient’s (or 
caregiver’s) device, nor will it access any personal information besides that which has been entered into 
the app or has been obtained from the RAMQ or the patient’s discharge prescription. Once the 30-day 
follow-up period of the study has ended, the patient/caregiver will retain access to the app and the app 
will continue to be linked to the RAMQ to display dispensed medications; however, the app will no 
longer display nonadherence alerts and will not connect patients to hospital pharmacists (messages 
patients enter into the feature will instead be faxed to their community pharmacists, who they will be 
advised to contact). Patients/caregivers will also be able to update their medication list manually. 
The backend/server of the app will be hosted on the McGill network. The network is a highly restricted 
environment with no access to the internet. The user accesses the app via a remote proxy server which 
acts as a mediator from the internet to the server. A robust firewall system restricts access to and from 
the server to the internet. There is also a campus-wide antivirus system that monitors network traffic and 
keeps our systems safe. User access directly to the server is highly restricted to the support staff and is 
logged. 
Any research publications related to this study will present data in aggregate form only. Study data will 
be used for the exclusive purposes described in this protocol and will be retained for a period of 7 years 
following study completion, after which they will be destroyed. 
Project monitoring mechanisms 

Research Protocol – Mobile App for Medication Management 
8 
 
We will monitor consent and enrollment rates weekly to ensure we are meeting expected targets and 
will modify procedures accordingly if needed. We will also continually monitor use of the mobile 
application throughout the 30-day follow-up period to ensure patients are utilizing the app consistently 
for its intended purposes. 
 
 

Research Protocol – Mobile App for Medication Management 
9 
 
References 
 
1. 
Rosenbloom D, Wynne C. Dectecting adverse drug reactions. Canadian Medical Association 
Journal. 1999;161(3):247. 
2. 
Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: 
a meta-analysis of prospective studies. Jama. 1998;279(15):1200-1205. 
3. 
Wu C, Bell CM, Wodchis WP. Incidence and economic burden of adverse drug reactions among 
elderly patients in Ontario emergency departments: a retrospective study. Drug safety. 
2012;35(9):769-781. 
4. 
Forster AJ, Clark HD, Menard A, et al. Adverse events among medical patients after discharge 
from hospital. CMAJ : Canadian Medical Association journal = journal de l'Association medicale 
canadienne. 2004;170(3):345-349. 
5. 
Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse 
events affecting patients after discharge from the hospital. Annals of internal medicine. 
2003;138(3):161-167. 
6. 
Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention 
Study Group. Jama. 1995;274(1):35-43. 
7. 
Bell CM, Bajcar J, Bierman AS, Li P, Mamdani MM, Urbach DR. Potentially unintended 
discontinuation of long-term medication use after elective surgical procedures. Archives of 
Internal Medicine. 2006;166(22):2525-2531. 
8. 
Charters E. The Use of Think-aloud Methods in Qualitative Research: An Introduction to Think-
aloud Methods. Brock Education Journal. 2003;12(2). 
9. 
Cooke L. Assessing Concurrent Think-Aloud Protocol as a Usability Test Method: A Technical 
Communication Approach. IEEE Transactions on Professional Communication. 2010;53(3):202-
215. 
10. 
Bond RR, Finlay DD, Nugent CD, Moore G, Guldenring D. A usability evaluation of medical 
software at an expert conference setting. Computer Methods and Programs in Biomedicine. 
2014;113(1):383-395. 
11. 
Siek KA, Khan DU, Ross SE, Haverhals LM, Meyers J, Cali SR. Designing a Personal Health 
Application for Older Adults to Manage Medications: A Comprehensive Case Study. Journal of 
Medical Systems. 2011;35(5):1099-1121. 
12. 
Simon AC, Holleman F, Gude WT, et al. Safety and usability evaluation of a web-based insulin 
self-titration system for patients with type 2 diabetes mellitus. Artificial intelligence in medicine. 
2013;59(1):23-31. 
13. 
Harris LT, Tufano J, Le T, et al. Designing mobile support for glycemic control in patients with 
diabetes. Journal of Biomedical Informatics. 2010;43(5, Supplement):S37-S40.