The Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial
Phase: NA
Conditions: Drug-Related Side Effects and Adverse Reactions, Adverse Drug Events
Sponsor: VA Office of Research and Development

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 1 of 23 
 
The Secure Messaging for Medication Reconciliation Tool 
(SMMRT) Trial 
NCT02482025 
 
August 31, 2018 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 2 of 23 
 
Research Protocol Narrative 
Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial 
IRB #3156 
Protocol Version 2.0, August 31, 2018 
 
 
 (1)  RATIONALE 
 
(a) Statement of the Problem.  Briefly state the problem to be investigated. 
   Medication discrepancies, which occur in all healthcare systems including VA, lead to adverse drug 
events (ADEs). ADEs contribute to readmissions and emergency department visits. 
   Medication reconciliation, especially with pharmacist involvement, can reduce medication 
discrepancies and prevent readmissions, but little is known about interventions occurring post-discharge. 
   Evaluation of the effectiveness of pharmacist-mediated medication reconciliation via Secure 
Messaging (SM) represents an opportunity to reduce Veterans’ hospital utilization after discharge and 
may reveal potential future use of this technology to engage Veterans in their care.  
 
 
(b)  Hypotheses or Key Question.  We hypothesize that a Secure Messaging for Medication Reconciliation 
Tool (SMMRT) will reduce medication discrepancies among Veterans discharged from the hospital and skilled 
nursing facility. 
 
 
(c)  Specific Objectives.  
Aim 1. To conduct a RCT of usual care vs. usual care plus SMMRT to reduce medication discrepancies; 
Aim 2. To evaluate how Veterans and staff perceived the impact of SMMRT on routine clinical practices and, 
specifically, on Veterans’ interactions with their primary care providers. 
 
 
(2)BACKGROUND AND SIGNIFICANCE 
Summary Points 
➢   Medication discrepancies, which occur in all healthcare systems including VA, lead to adverse drug events 
(ADEs). ADEs contribute to readmissions and emergency department visits. 
➢   Medication reconciliation, especially with pharmacist involvement, can reduce medication discrepancies 
and prevent readmissions, but little is known about interventions occurring post-discharge. 
➢   Evaluation of the effectiveness of pharmacist-mediated medication reconciliation via Secure Messaging 
(SM) represents an opportunity to reduce Veterans’ hospital utilization after discharge and may reveal 
potential future use of this technology to engage Veterans in their care. 
➢   Wagner’s Chronic Care Model, emphasizing system transformation and prevention of complications, is a 
conceptual framework for the Secure Messaging for Medication Reconciliation Tool (SMMRT) Trial. 
1.  a. Medication Discrepancies, Adverse Drug Events and Hospital Utilization. 
Medication discrepancies are defined as unintentional differences found in the patient's medical record 
compared with the patient's medication information.7 Discrepancies may be commissions, omissions, 
duplications or alterations in dose or frequency. Medication discrepancies are associated with adverse drug 
events (ADEs), which are broadly defined as “injury resulting from the use of a drug.”8   In the US, ADEs result in 
7,000 deaths annually and cost the health system $4.2 billion. One in four ambulatory patients in experience 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 3 of 23 
 
ADEs,9,10 while as many as 60% of patient records contain medication discrepancies.11-13 Patients experiencing 
transitions in care, such as hospital discharge, are particularly vulnerable to medication discrepancies and 
ADEs, the latter occurring in as many as one in five patients within 30 days of discharge.14-16 Medication 
discrepancies and ADEs are a major contributor to hospital utilization among patients recently discharged from 
the hospital,17 a period frequently marked by multiple medication changes, alterations in health status, and 
extended period of time before return to primary care. Among patients discharged from the hospital, 14-20% will 
be readmitted within 30 days,18,19 and more than 30% will seek emergency care during the same period.20 
While numerous tools and approaches have been developed to improve care transitions,21 medication safety 
after discharge remains a concern.22 
2. b. Medication Reconciliation: Policies and Approaches. 
The Joint Commission introduced medication reconciliation as a National Patient Safety Goal in 2005 and 
continues to emphasize its importance in 2014,23 including the imperative to “Make sure the patient knows 
which medicines to take when they are at home.” VA has similarly mandated that medication reconciliation 
occur “at every episode or transition in level of care.”7   Considerable research has demonstrated benefits of 
medication reconciliation at care transitions. Two recent systematic reviews identified a relatively small number 
of rigorous studies of medication reconciliation at the time of hospital discharge.24,25 Kwan et al included 3 
RCTs and estimated that medication reconciliation at discharge reduces readmissions and emergency 
department visits by 23%; the authors noted that the effect may be larger when medication reconciliation is 
coupled with additional post-discharge follow-up, as it was in Project RED (Re-Engineered Discharge),20 and 
as we propose in The SMMRT Trial. Additionally, Mueller et al noted that available evidence, albeit scarce, 
supports medication reconciliation interventions that rely on pharmacy staff,25 as we propose herein. 
Within VA, Boockvar et al found that a Computerized Patient Record System (CPRS)-based medication 
reconciliation tool used on hospital admission reduced both medication discrepancies and ADEs.26 They are 
currently conducting a trial (IIR 10-146) of medication reconciliation on hospital admission with CPRS 
information enhanced by regional health information. A QUERI-funded pilot (RRP 11-242) identified 
medication reconciliation as a key element of successful interventions to improve hospital-to-home transition. 
Lesselroth et al have implemented an outpatient medication reconciliation: the Automated Patient History 
Intake Device 3,27 and are developing an interface for medication reconciliation on hospital admission.28 
2. c. My HealtheVet (MHV) and Secure Messaging (SM). 
MHV, VA’s online patient portal, enables Veterans to access their health information and VA health care team 
on the Internet. MHV has over 2.5 million registered users, including 1.4 million Veterans who have completed 
in-person authentication (IPA). With IPA status, Veterans can use SM, akin to E-mail but behind the VA firewall 
and limited to Veterans, their designees, and VA providers; more than 789,000 Veterans have used SM.29 
A growing body of literature indicates that Veterans are using the Internet in general and MHV in particular 
to access health information.30,31  Early studies suggest the potential to employ MHV and SM for outreach to 
Veterans.32 Aside from our pilot (described below, 4.a.1), no studies have examined the role of pharmacist- 
 
 
(CONTINUED ON NEXT PAGE)

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 4 of 23 
 
mediated medication reconciliation by SM. A recent study identified a pathway for SM to reduce hospital 
utilization.33 In a recent HSR&D supported systematic review, Goldzweig et al found insufficient evidence 
linking patient portals to improved health outcomes,34 but concluded that interventions using patient portals in 
conjunction with case management were most effective. Analogously, our study features pharmacists to review 
and reconcile medications. Moreover, Goldzweig et al called not only for rigorous RCTs in this area but also for 
studies that examine “organizational and provider context and implementation processes,” as we propose herein 
(Aim 3). The SMMRT Trial will yield valuable information on the effects of MHV and SM-mediated medication 
reconciliation on health outcomes, guiding future efforts to employ these technologies. 
2. d. Conceptual Framework. 
Wagner’s  Chronic Care Model (CCM) provides the study’s conceptual framework.35-37 The CCM recognizes 
FIGURE 1. Chronic Care Model Elements and Key Innovative Features of The SMMRT Trial 
the importance of 
Elements of 
the Chronic 
Care Model 
 
 
 
 
Innovative 
Elements in 
the SMMRT 
Trial 
Patient self- 
management 
support 
 
▪MHV,SMMRT 
support self- 
management 
▪Usability 
testing with 
Veterans to 
optimize 
SMMRT 
Delivery 
system 
redesign 
 
▪Pharmacists 
play key role 
in SMMRT 
▪Usability 
testing leads 
to redesign 
▪Aim 3 
heralds 
implement’n 
Use of clinical 
information 
systems 
 
▪SMMRT sent 
via MHV 
Secure 
Message 
▪Includes 
med images 
from central 
data base 
Provider 
decision 
support 
 
▪Pharmacists 
included in 
usability 
testing 
▪Pharmacists 
update CPRS 
and notify 
PCP/PACT 
Linkage to 
community 
resources 
 
▪Partnerships 
with all VISN- 
1 VAMCs 
▪Pharmacists 
communicate 
with non- 
Boston PCPs 
via local CPRS 
Health care 
organization 
support 
 
Operational 
Partnerships: 
▪Layden, Co-I, 
MedRec,PBM 
▪Nazi, Co-I, 
MHV, OIA 
▪Calls + face- 
to-face mtgs 
transforming 
systems to focus 
on preventing 
disease and its 
complications. 
Figure 1 shows 
how key innovative 
features of The 
SMMRT Trial are 
linked to elements 
of CCM 
interventions. 
 
Summary Points 
➢   This project is highly significant and concordant with HSR&D Research Priorities, VA’s Transformational 
Initiatives and operational partners’ objectives. 
➢   The study design will distinguish between the effects of MHV and the effects of the Secure Messaging 
Medication Reconciliation Tool (SMMRT) on health outcomes and will include formative evaluation. 
➢   Operational partners responsible for MHV, medication use and patient safety have invested in this research 
with active participation, which will ensure that study results will be useful in future operational planning. 
3.  a. HSR&D Priority C: Healthcare Informatics to Improve Veteran Care. 
This priority integrates "biomedical knowledge systems with technology to improve decision-support systems, 
evidence-based practices, collaboration and continuity of care among providers, and Veteran and provider 
education." This project capitalizes on a widely used existing technology, SM within MHV, to facilitate Veteran- 
centered medication reconciliation. Aim 1 employs usability testing to optimize the tools. The trial design (Aim 
2) will allow us to distinguish the independent effects of MHV engagement and pharmacist-mediated 
medication reconciliation by SM. Aim 3 features a formative evaluation for future implementation. 
3. b. Transformational Initiative (T21): Employ state-of-the-art IT in Veterans’ health care. 
The SMMRT Trial directly addresses VA’s vision of leveraging cutting-edge technology for improving Veterans’ 
access to health care. MHV and, specifically, SM, enable Veterans to communicate directly with their health 
care providers from their own homes, at their own pace (i.e., asynchronous communication). 
 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 5 of 23 
 
(3) WORK ACCOMPLISHED 
 
The study team has experience in an array of scientific fields relevant to The SMMRT Trial. This section 
highlights studies laying the foundation for the proposed research. 
#The SMMRT Pilot Study. 
The study, published in JAMIA,38 field tested the methods of the current proposal. We initially developed 
SMMRT as an interactive PDF file (Figure 2) for pharmacist and Veteran to interact asynchronously via SM to 
review and reconcile medications following 
FIGURE 2. PDF File Version of SMMRT 
discharge. Because the attachment 
feature of SM was not available at 
the time of the pilot, we instead used 
a text-based format, embedded in 
the body of the SM (figure 1 in 
Appendix 2), for the pharmacist and 
Veteran to reconcile medications. 
As we will do in the proposed 
SMMRT Trial, we recruited 
hospitalized Veterans, registered 
them for MHV, and trained them to 
use MHV and SMMRT. After 
discharge, our pharmacist reviewed 
the CPRS records of 51 eligible 
Veterans and identified 108 clinically 
important medication discrepancies 
(median 2 per Veteran), mostly 
medications that the Veteran was 
taking but were omitted from the 
discharge summary or discharge medication list. After correcting these discrepancies, the pharmacist sent 
SMMRT via SM to the Veteran. A total of 34 Veterans (67%) returned SMMRT. Of these 34 Veterans, 17 
(50%) had additional discrepancies, most commonly duplicative medications prescribed for the Veteran at VA 
Boston and another VA facility.  Nine of 10 Veterans completing post-intervention in-depth interviews (similar 
to those proposed in Aim 3) said they would use SMMRT again. Appendix 5 presents direct quotes about 
SMMRT from these Veteran participants. This study demonstrated the feasibility of recruiting hospitalized 
Veterans and training them in MHV and SM. Results highlighted the high prevalence of medication 
discrepancies immediately after discharge and the ability to detect and correct them by SMMRT. 
#Aligning Medication Reconciliation and SM: A Qualitative Study of Providers’ Perspectives. 
In this study, published in JMIR,39 we conducted in-depth interviews with 15 primary care providers to 
characterize ambulatory medication reconciliation, the use of SM in primary care, and perceptions of a SMMRT-
like medication reconciliation system. Providers recognized the value of medication reconciliation, especially 
after hospital discharge, and suggested numerous challenges to medication reconciliation. Providers 
emphasized the importance of collaborating with pharmacists in reviewing Veterans’ medication regimens. 
#Qualitative Study of Pharmacists’ Role in Medication Review and Reconciliation. 
Dr. Linsky recently completed a series of interviews with clinical pharmacists regarding their role in medication 
management. Preliminary analyses revealed considerable enthusiasm among pharmacists for developing 
relationships and monitoring patients to improve medication safety. 
#Medication Discrepancies. 
We evaluated the accuracy of VA’s computer-generated medication listing to determine prevalence of 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 6 of 23 
 
medication discrepancies; 60% of ambulatory Veterans had at least one medication discrepancy.13 
#Usability Studies. 
Colleagues Drs. Alissa Russ and Alan Zillich have conducted research at the VA HSR&D Human-Computer 
Interaction Laboratory in Indianapolis, IN that directly informs this investigation. These studies have included 
usability testing with VA patients or providers. For example, in a study published in JAMIA, Dr. Russ and 
colleagues evaluated usability of basic MHV features.41  In this study, Dr. Russ designed scenarios to test the 
pharmacy refill function and improve its usability for Veterans; she will develop analogous scenarios for SMMRT 
in the proposed study. In the precursor project to the work proposed herein, Dr. Russ led usability studies to 
refine and enhance the SMMRT, leading to two publications recently accepted for publication: 
• Russ AL, Jahn MA, Patel H, Simon SR. Usability Evaluation of a Medication Reconciliation Tool as a 
Precursor to a Clinical Trial: Blending Factual Scenarios with Artificial Safety Probes. J Biomed Inform. 
2018 (In Press). 
• Jahn MA, Porter BW, Patel H, Simon SR, Russ AL.  Usability Assessment of Secure Messaging for 
Clinical Document Sharing between Healthcare Providers. Appl Clin Inform. 2018 (In Press). 
 
(4) WORK PROPOSED 
Timeline 
Month 
Research Activity 
0 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
IRB Approval 
  
x 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
AIM 1 (SMMRT RCT) 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Training of Research Assistant 
  
x 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Re-introduction of study to clinical 
stakeholders 
  
x 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Testing data flow processes (med 
list into SMMRT tool) 
  
x 
x 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Recruitment and enrollment (West 
Roxbury and Brockton) 
  
  
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
  
  
  
  
  
  
SMMRT Intervention Activities 
  
  
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
  
  
  
  
  
Data Collection (30-day follow-up 
interviews) 
  
  
  
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
  
  
  
  
Data Analyses 
  
  
  
  
  
  
  
  
  
  
  
  
  
x 
x 
x 
x 
  
  
Manuscript: Main outcomes of the 
SMMRT RCT 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
x 
x 
x 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
AIM 2 (Implementation-oriented 
formative evaluation) 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Recruitment and enrollment 
  
  
  
  
  
  
  
  
x 
x 
x 
x 
  
  
  
  
  
  
  
Telephone interviews 
  
  
  
  
  
  
  
  
  
x 
x 
x 
  
  
  
  
  
  
  
Transcription and data analysis 
  
  
  
  
  
  
  
  
  
  
x 
x 
x 
x 
  
  
  
  
  
Manuscript: Veterans' and Staff 
experiences with SMMRT 
  
  
  
  
  
  
  
  
  
  
  
  
  
x 
x 
x 
x 
x 
x 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 7 of 23 
 
(4) WORK PROPOSED (continued) 
 
Methods for Aim 1 (SMMRT RCT): 
1. Study setting and population 
The setting of Aim 1 will include not only the acute inpatient hospital (located on the West Roxbury 
Division Campus) but also the CLC, located on the Brockton Division Campus. 
2. Recruitment and enrollment 
Approximately ten (10) patients per week are discharged to home from the CLC each week. Because of 
the strong relationships developed between the clinical teams and the Veterans and their families 
during the long (compared with acute inpatient) hospitalizations, typically 3-4 weeks, Veterans are likely 
to be amenable to enrolling in a clinical research trial that they see their clinical team members 
advocating. Conservatively, we estimate that at least 2 patients each week will enroll in SMMRT, such 
that over the course of 12 months (50 weeks), a minimum of 100 patients would be enrolled. In 
addition, we have ensured that resources are available to enroll 140 patients at the West Roxbury 
setting, resulting in a total sample size of 240 subjects. Recruitment and enrollment processes will 
follow these steps: 
1. Research staff will communicate regularly with clinical staff (nurses and clinicians) on the inpatient service at VA 
Boston’s  West Roxbury and Brockton Divisions 
2. Research staff will identify any Veterans hospitalized on the Inpatient Service meeting the following criteria by  
reviewing CPRS, patient lists, and consulting with clinical staff: 
a. age 18 years or older 
b. having a VA primary care provider (PCP) at any VA facility in VISN-1 
c. planned discharge home (as opposed to another facility) 
d. anticipated to be discharged with at least 5 medications 
3. Research staff will confirm with clinical staff that the Veteran/patient may be approached and informed about the 
study. 
4. Research staff will approach the hospitalized Veteran to describe the study and seek informed consent for 
participation. 
5. The research staff will briefly describe the study and then administer the Callahan cognition screener (see separate 
document) and the eligibility screening questions below. 
6. If the Veteran passes the Callahan cognition screener and eligibility screening questions, the research staff 
member will seek to obtain informed consent and HIPAA Authorization. 
 
Script for Aim 2 Eligibility Screening Questions (after Callahan Screening and before Informed Consent): 
 
• 
Thank you for allowing me the chance to tell you about our study. I would like to ask you a few questions first to 
see if you meet the requirements for eligibility. Would that be okay? 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 8 of 23 
 
1. Do you have access to a computer? Y/N (If yes, continue. If no skip to question 4) 
2. Do you use a computer to connect to the internet, say to search websites or for email? 
3. If yes, Do you use secure messaging? If no, would you be willing to use it if required for this study? 
4. Does anyone help you manage your medicines? Y/ N (If yes, continue. If no, not eligible, go to Verbal consent) 
5. Does that person use a computer/ internet/ email? Y/N (If yes, continue. If no, not eligible, Would you be willing 
to have your caregiver use secure messaging on your behalf  if required for this study?  
 
If no to either 2 or 3: Unfortunately, you do not meet the criteria to participate in this study. Thank you for your time 
and interest. 
→If Yes to 3 and 4, administer the Callahan screener. If patient passes the screener, then the patient is eligible. Continue 
on to ICF and HIPAA forms. 
 
3. Randomization and intervention 
Veterans successfully recruited and consented to participate will be randomly allocated to either UC or 
UC+SMMRT. Veterans randomized to UC+SMMRT will receive introduction to My HealtheVet and 
SMMRT. Within 3 business days of discharge, the SMMRT Research Pharmacist will engage with 
Veterans and/or their designated family members to conduct medication review and reconciliation via 
Secure Messaging. See randomization scheme below. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 9 of 23 
 
 
After baseline assessment is complete, participants will be randomly assigned via computer program to receive 
1) usual care  or 2) UC plus pharmacist-mediated medication reconciliation (SMMRT). After opening, the 
research assistant will inform the Veteran of the “treatment assignment.” 
 
Overview of the Intervention 
The intervention principally entails the creation of a personalized medication list for each 
Veteran/subject, incorporating most current data from Vista (the medical record) into a user-friendly 
PDF-file interface, i.e. the SMMRT.  The research pharmacist will send the SMMRT via attachment to 
Secure Message to the Veteran and will then interact with the Veteran to ensure that the medical 
record accurately reflects what medications the Veteran is taking.  
What Happens in the Intervention? 
The SMMRT Intervention. Figure 4 shows the elements of the SMMRT intervention. In the figure, diamonds 
show Veteran activity; ovals show pharmacist activity; and dotted boxes show information exchanges. 
Not shown in the figure, Veterans 
will be trained to use MHV while still 
FIGURE 4. Flow Diagram of the SMMRT Intervention 
 
Pharmacist confers with 
hospitalized. Starting 
at top left of the figure, within 3 
business days after the Veteran is 
discharged 
Veteran 
discharge
d home 
 
Pharmacist reviews 
SMMRT, updates and 
reconciles 
PCP for clinically urgent 
medication issues 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 10 of 23 
 
home, the pharmacist reviews 
CPRS, reconciles medications, and 
in CPRS 
 
Pharmacist 
documents medication 
prepares the SMMRT, including 
photographic images (JPEG files) of 
each medication from the 
Medication Image Library (MIL). The 
MIL is a database developed and 
maintained by VA Consolidated Mail 
Order Pharmacy with pharmacist- 
validated images for use in 
 
Pharmacist 
reviews CPRS, 
reconciles medications, 
prepares SMMRT, 
including pill images 
from Medication Image 
Library (MIL) database 
 
Veteran 
uploads 
corrected 
SMMRT and 
sends via 
Secure 
Message to 
Pharmacist 
Pharmacist 
clarifies 
discrepancies 
via Secure 
Message or 
telephone 
with Veteran 
reconciliation in CPRS, 
adds PCP co-signer 
 
 
 
Pharmacist sends final 
revised SMMRT to 
Veteran via Secure 
Message. Loop closed. 
medication identification. The MIL 
has accurate image matches to 
99.9% of VA-dispensed medications 
based on national drug index (NDI) 
numbers (personal communication, 
Richard Pham, CDW). Following 
Pharmacist sends SMMRT to 
Veteran via Secure Message 
Veteran receives, 
downloads and 
reviews SMMRT for 
accuracy 
Abbreviations: 
CPRS=Computerized Patient 
Record System; SMMRT=Secure 
Messaging for Medication 
Reconciliation Tool; PCP=Primary 
Care Provider 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 11 of 23 
 
the approach developed by Lesselroth et al,3 we will use automated processes to populate the SMMRT with 
the medication list and MIL images. After reviewing the SMMRT, the Pharmacist sends it via SM to the 
Veteran, who receives the SM, downloads the SMMRT, and reviews it for accuracy (in comparison with home 
medications). We learned from our pilot study that only about half of Veterans had their medications present 
when completing the SMMRT; our training for the proposed trial will include encouragement to have 
medications present while completing the SMMRT. Following review and correction, the Veteran returns the 
SMMRT to the Pharmacist within 3 business days, initiating a possible back-and-forth exchange of messages 
(or telephone calls) to confirm medication accuracy; this exchange is represented in the middle of the figure with 
bidirectional arrows. The pharmacist confers with PCP for clinically urgent or uncertain issues (occurred in only 
two cases during the pilot study) and documents medication reconciliation in CPRS. Finally, the pharmacist 
sends the final revised SMMRT to the Veteran. The research associate monitors the initiation of the exchange 
and will prompt either the pharmacist or the veteran to respond within 3 business days, via phone. The veteran 
will be informed at the beginning of the study and at the end if appropriate that access to the pharmacist ends 
in 30 days. 
 
 
4. Ascertainment of Outcome 
Outcomes will be ascertained via 30-day telephone interview (see Telephone Script for 30-Day Follow-up Call) 
and chart review. The research associate will send out a written reminder a few days before the 30 day follow-
up call, and make a reminder call the day before the interview, to remind the veteran to have their medications 
ready. The research associate will ascertain the medications that the subject is currently taking, and the 
research associate will then compare that list with the medication list in the medical record to identify 
discrepancies. We will distinguish medication discrepancies as being of a) high significance, b) moderate 
significance, or c) low significance, based on how likely they would be to result in patient harm. 
In the event the veteran was not reached for the interview, the research associate will send out a letter and call 
the veteran to reschedule the interview.  
5. Sample size and power 
We propose a two-arm RCT, with 120 participants in each arm. The main outcome measure will be combined 
number of medium- and high-significance medication discrepancies present at the 30-day follow-up. 
Conservative estimates suggest that we would expect a rate of 3.0 medium- or high-significance medication 
discrepancies per patient in the Usual Care group. Using a custom simulation, with two-tailed alpha error set at 
5%, we calculated that this sample size (120 subjects per arm) would have 99.9% power to detect a lowering of 
the rate from 3.0 to 2.0 and 82% power to detect a lowering of the rate from 3.0 to 2.4.  
6. Measures. 
Study measures will be derived from baseline and 1-month follow-up interviews, as well as from CPRS. 
Predictor Variable. The predictor variable will be the experimental condition (i.e., Usual Care 
 SMMRT), assigned by random allocation (see above). 
Main Outcome Variable. The primary outcome measure will be medication discrepancies detected 30 days after 
discharge. Secondary outcome measures will be 30-day hospital utilization (combined readmissions plus emergency 
department use) and Veterans’ self-efficacy in medication use. 
Secondary Outcome Variables. Secondary endpoints, ascertained through a combination of CPRS review 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 12 of 23 
 
plus follow-up interview with participants, include community tenure (days spent outside of the hospital), 
medication discrepancies (overall and by subtype, i.e., omissions, commissions, duplications, and alterations 
in dose or frequency); health status (SF-12 health status scale; 12 items, 2-3 minutes), medication use self-
efficacy (score of 0 to 8, based on validated 8-item yes/no MUSE scale;62 3 minutes); and the Care Transition 
Measure (score of 0 to 100, based on validated 15-item, 4-point [strongly agree, agree, disagree, strongly 
disagree] scale, linear transformed to 0 to 100 scale,63 5 minutes). 
Other Variables. Prior to randomization, we will stratify participants on the basis of My HealtheVet in-
person authentication status (yes/no) and VA Boston primary care (yes/no); see above Figure 2 and Section 
4.e.5. Baseline data will include age, educational level, race/ethnicity. At baseline, we will administer the 7-
item REALM-SF health literacy scale,64 the SF-12, and the medication use self-efficacy scale (MUSE). 
During the 30-day follow-up we will re-administer the SF-12, the CARE Transition measure and the 
MUSE.  
 
7. Statistical Analysis. 
Preliminary Analyses. In preliminary analyses, descriptive statistics and bivariate associations will be 
computed. Then we will assess whether there are differences between the study arms for socio- demographic 
variables. We will assess imbalance formally using Analysis of Variance for continuous variables and Chi 
Square tests for dichotomous variables. In order to confound any observed effect of the treatment, a 
covariate must be imbalanced between the arms and also associated with the outcome. Thus, we will judge 
how plausible confounding is based on the formal tests, the degree of imbalance, and the plausibility of 
association with the outcome. Primary analysis will not include any potential confounders, but sensitivity 
analyses will adjust for plausible confounding variables with large imbalance between the arms. 
Statistical Analyses. The primary hypothesis is that SMMRT will result in a reduction in 30-day 
medication discrepancies as compared with UC. The main outcome will be tested by a two-step analysis. 
First, we will conduct a logistic regression to compare SMMRT vs. UC. We will take an intent-to-treat 
approach to the analysis; Veterans will be included in our primary analyses regardless of their level of 
adherence with their assigned intervention. For participants who do not complete the telephone 
assessment, we will rely on CPRS/CDW data for outcome assessment. 
Checks and treatment for “non-Response” Bias. The potential for differential drop-out rates among the 
three groups is itself an interesting and important empirical question, with clinical implications. Drop outs will 
be defined as active drop outs (Veterans who request discontinuation of participation) and passive drop outs 
(Veterans who cannot be reached for follow-up within 60 days of discharge). Factors that may contribute to 
drop-out rate will be tested by logistic regression. This logistic regression will be used to generate completion 
probabilities, which we will use as Inverse Probability Weights (IPWs). We will then use the IPWs in a 
weighted logistic regression for the outcome as predicted by study arm. Heuristically, Veterans with low 
probabilities of response who do respond are upweighted to represent Veterans who did not respond. 
Veterans who were quite likely to respond will not be upweighted as much. The result is a weighted sample 
that resembles the original population. In the weighted analysis, we will use the robust or “sandwich” 
variance estimator to account for the variability added by estimating the IPWs. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 13 of 23 
 
Methods for Aim 2: In-Depth Interviews 
 
Aim 2 . To evaluate MHV training and SMMRT interventions for potential future implementation. For 
this formative evaluation, we will use qualitative research methods to examine in-depth how Veterans 
perceived their interactions with the MHV Training and SMMRT intervention components. We will also 
interview PACT RNs and Pharmacists to solicit their perspective on the integration of SMMRT into the post-
discharge PACT workflow and will analyze related information collected from chart abstraction. 
 
Rationale for formative evaluation. 
Formative evaluation will inform future implementation efforts, specifically in the area of medication 
reconciliation and more generically for interventions using health IT to engage Veterans in their care. 
Formative evaluation is valuable in the context of an intervention that results in quantitative improvement of 
outcomes because it can yield information regarding characteristics of the intervention that seemed most 
useful and effective from the perspective of the study participants. Perhaps more importantly, formative 
evaluation is crucial in the context of an intervention that does not result in quantitative improvement in the 
target outcome (i.e., in the setting of a “negative study”), because it can identify the factors that may have 
prevented an otherwise-potent intervention from achieving the intended outcome. 
 
Procedure for recruitment and consent of participants in Aim 2 
 
Recruitment of Veterans 
All Veterans recruited for Aim 1 and randomized to the SMMRT study arm will be eligible for Aim 2. At the 
time of initial recruitment, Veterans will be told that they may be selected to participate in this in-depth 
interview after completion of the 30-day outcome assessment interview for Aim 1. Because Veterans have 
already provided informed consent and signed the HIPAA authorization for participation in Aim 1, and 
because only the information obtained by virtue of that authorization will be used to determine eligibility for 
participation in Aim 2, no waiver of HIPAA authorization is needed for this Aim. 
 
Recruitment of Staff (Nurses and Pharmacists) 
We will send an email to PACT nurses and pharmacists who have had at least one Veteran participate in the 
SMMRT trial in the SMMRT study arm. Please see separate Staff Recruitment Email document. We will follow 
up the e-mail with a telephone call to recruit the staff member and to review elements of informed consent. 
Please see separate Telephone Script.  Those staff members willing to participate will receive the informed 
consent form and will return it by whichever route/method they prefer (e.g., in-person, fax, e-mail). 
 
Informed Consent of Veterans 
A single Veteran informed consent form will be used for both Aim 1 and Aim 2. The ICF indicates that a 
subset of participants in Aim 1 (the RCT) will be invited to participate in Aim 2 (in-depth interviews). The ICF 
distinguishes the two study components and the compensation offered for participation in each study 
component. This informed consent form will be completed at the time of initial enrollment in the study, i.e., 
while the Veteran is hospitalized and prior to randomization for Aim 1. 
 
Informed Consent of Nurses and Pharmacists 
The Staff version of the informed consent form will be completed prior to the conduct of the in-depth interview. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 14 of 23 
 
In-depth Interviews. 
 
Veteran Interviews 
Note: The interviews with Veterans will be conducted by telephone or, if the Veteran requests, in person at 
VA Boston. We will carry out in-depth interviews with Veterans within 2 weeks after completion of the study to 
minimize perturbation of the experimental setting. In-depth interviews are characterized by extensive probing 
and the use of open-ended questions to elicit participants’ thoughts in their own words. As we have done in 
prior studies,66 we have prepared a semi-structured interview guide that includes a core list of questions to be 
supplemented by prepared and spontaneous follow-up questions and probes to seek clarification, expansion 
and examples, and to follow new relevant lines of inquiry. Interviews will focus on identifying and 
characterizing factors related to successful or unsuccessful implementation of the MHV Training and SMMRT 
interventions, beginning with study recruitment and enrollment, including MHV registration, training and 
encouragement, and continuing through actual engagement with SMMRT. To assess the fidelity of the 
intervention, the interview will encourage Veterans to provide feedback on specific details of the intervention 
that they found helpful and useful, as well as those features that were of little value or even counter-productive. 
Broad topic areas will include the ease or difficulty associated MHV and SM in general, and SMMRT in 
particular; experiences with the medication reconciliation process at home; attitudes toward and reactions to 
SMMRT; and perceptions of how MHV and SMMRT influenced their health and health care. (See attached 
Interview Guide). 
We will use purposive criterion-based extreme-case sampling67,68 to identify potential Veteran participants 
(N=20) allocated to the SMMRT Arm. We will recruit approximately equal numbers of Veterans who were at 
the higher and lower extremes of number of medication discrepancies detected at the 1-month follow-up and 
will include both Veterans who experienced post-discharge hospital utilization as well and those who did not. 
As we have done previously, interviews will be conducted by telephone and, with the permission of the 
participants, will be audio-recorded for transcription and subsequent analysis. (See Human Subjects for 
details on protection of human subjects.) 
 
Staff Interviews 
Note: Interviews with staff members will be conducted by telephone or, if the staff member requests, in person 
at VA Boston. We will recruit 10-15 PACT nurses and 5-10 PACT pharmacists from multiple VISN-1 facilities for 
brief (~20-minute) focused interviews to explore how SMMRT influenced post-discharge calls and more 
generally their workflow and communication with Veterans. We will specifically inquire about potential 
duplication of effort and whether the SMMRT led to any unanticipated consequences. With permission, we will 
audio-record the interviews for transcription. 
 
Analysis. 
We will conduct content analysis of the transcribed in-depth interviews, incorporating the principles of the 
immersion-crystallization method.69 This qualitative approach consists of repeated cycles of immersion into the 
collected data with subsequent emergence, after reflection, of an intuitive crystallization of the dominant 
themes.69 Dr. Simon and research staff will independently listen to selected interview tapes, read all transcripts 
and write analytic notes for each interview. We will meet regularly to discuss each transcript and will compile 
detailed notes to document emerging themes and maintain a permanent record of the analysis. 
We will then compare the data from the transcript under discussion with the data from other analyzed 
transcripts. Through this process, we will identify salient themes that crystallize from the interviews, and code 
categories for managing further interpretation of the data. Following the principles of the template organizing 
style of data interpretation,70 Dr. Simon and the research staff will develop a code book based on the previous 
analysis activities, and then code the transcripts and observation guides using NVivo 10, a qualitative data 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 15 of 23 
 
management software tool.71 Team members will meet regularly to review the coding strategies and themes 
that emerge from further analysis of the thematic and categorical reports that will be generated following 
coding. We will continue analysis until no new major themes emerge. We will address recognized criteria for 
qualitative research: credibility, fittingness, auditability, and confirmability.72 
 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 16 of 23 
 
(5 ) HUMAN SUBJECTS 
 
 
The overall goal of this project is to evaluate the Secure Messaging for Medication Reconciliation Tool 
(SMMRT) for the purposes of post-discharge medication reconciliation. This project proposes two types of 
research: Aim 1 is quantitative: an RCT to compare the effect of an intervention on the medication 
reconciliation process following discharge from the hospital.  Aim 2 is qualitative, gathering the preferences, 
attitudes, and behaviors of Veterans and VA pharmacists and nurses regarding the SMMRT tool and its 
implementation.  
 
Aim 1: Risk to Subjects 
 
Human Subjects Involvement and Characteristics 
The goal of Aim 1 is to conduct a two-arm RCT to evaluate the effects of SMMRT. This trial will 
compare 1) Usual Care (UC) with 2) UC plus pharmacist-mediated medication reconciliation via SM using 
SMMRT among Veterans discharged from the hospital. A total of 240 Veterans will be recruited for Aim 1 once 
deemed appropriate for the study. The study will be explained to Veterans who express interest, and informed 
consent will be obtained. Veterans will be informed that their care will not be affected if they choose not to 
participate in the study. Next, Veterans will be randomized to condition, and the baseline interview will be 
administered. If the Veteran is randomized to UC plus SMMRT, the research associate will instruct the Veteran 
on how the tool works. When the Veteran has completed the training with the SMMRT tool, the research 
associate will ensure no other questions remain. Veterans in the UC plus SMMRT Intervention condition will be 
monitored during the length of the study. A total of 20 Veterans from the experimental arm of Aim 1 will be 
contacted for a post- intervention in-depth interview (See Aim 2 below). 
 
Sources of Materials 
Data to assess the outcomes of the RCT will be derived from review of the medical record (CPRS and 
VistAWeb), the Corporate Data Warehouse (CDW), and telephone-based interviews with participants. 
 
Potential Risks 
The risks to subjects from our study are minimal. The primary risk to Veterans is disclosure of sensitive 
information. In general, as in virtually any human research study, there is a risk of data security breach and 
resulting loss of confidential study data. Our procedures are designed to prevent any unauthorized disclosure; 
see Adequacy of Protection from Risk, below. We do not anticipate any adverse effects of the research to 
require any medical or professional intervention. 
 
Adequacy of Protection from Risk 
 
Recruitment and Informed Consent 
All research staff will be trained on protection of human subjects, and regularly supervised to ensure 
respect for potential participants, integrity of data collection, and appropriate interactions with staff. Veterans 
will be informed that their participation in the study is voluntary, and refusal to participate will not change the 
care they receive. 
 
Protection against Risk 
As described above, the potential risks to Veterans in Aim 1 are minimal. The main risk is loss of 
confidentiality of sensitive information. Recruitment will take place on inpatient units within VA Boston, as will 
the consent process, baseline interview and SMMRT tool training. If randomized to the SMMRT intervention, 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 17 of 23 
 
all medication data will remain in My HealtheVet, never leaving the VA firewall. When conducting the follow- 
up telephone interviews, the research associates will confirm that they are speaking with the proper Veteran 
participant. The research associate will only leave telephone messages saying that the Veteran is 
participating in a health study. 
We will only collect identifiable data when absolutely necessary and scientifically justified. The SMMRT 
will be sent through the secure messaging feature of My HealtheVet. Only the Veteran and study pharmacist 
will have access to the medication data. Data transmissions between study staff will be minimized, but when 
necessary, will meet or exceed all standards for encryption and security that are in place for the electronic 
transmission of Veterans’ clinical information, with which the Principal Investigator is familiar. 
 
Aims 2: Risk to Subjects 
Human Subject Involvement and Characteristics 
 
Aim 2 will involve telephone interviews with 20 Veterans who were allocated to the experimental 
arm of the RCT, with the use of semi-structured, in-depth interviewing techniques to assess Veterans’ 
experience with the SMMRT tool. In addition, Aim 2 will include brief, focused interviews with 10-15 
PACT nurses and 5-10 PACT pharmacists from VISN 1 facilities regarding their perceptions of whether and how 
the SMMRT intervention may have affected their post-discharge follow-up calls. The interviews with nurses and 
pharmacists will also explore unanticipated consequences of the SMMRT intervention. 
 
Sources of Materials 
During the course of conducting study Aim 2, we will collect data through interviews with 
Veterans, VA pharmacists, and nurses. Thus, all of the research material for this study will be 
data obtained specifically for research purposes. 
 
Potential Risks 
This study involves a low level of risk to all human subjects. There are no apparent physical risks for 
any of the subjects. In general, as in virtually any research study, there is a risk of data security breach and 
resulting loss of confidential study data. There is potential risk that the questions and conversations in the 
interviews could be psychologically upsetting to the Veterans. There is also the theoretical possibility that 
participating in the interviews could be physically taxing. Additionally, VA staff members are considered a 
vulnerable group, due protections exceeding their Veteran counterparts. The likelihood of any of these risks 
is low. None of these potential risks is serious. 
 
Adequacy of Protection from Risk 
 
Recruitment and Informed Consent 
Veterans: For Aim 2, we will identify Veterans allocated to the experimental condition (Usual Care + 
SMMRT Intervention) at the time of the 30-day follow-up telephone call. These Veterans will already 
have provided informed consent at the time of their enrollment in Aim 1. However, they will be reminded 
of the key elements of informed consent at the time of invitation to participate in Aim 2. Veteran 
participants will be informed that the collection of data will be strictly confidential and that the 
participation of all human subjects will be completely voluntary. The “script” that research staff will use in 
approaching Veterans for recruitment in the interviews will explicitly indicate the voluntary nature of 
participation and the assurance of confidentiality of respondents. 
VA Pharmacists and Nurses: A member of the study team will reach out to potential staff participants 
for Aim 2, as described in the Protocol. Those interested in participating will be educated on the study, made 
aware that participation will not impact employment status, and be provided a forum to address questions and 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 18 of 23 
 
concerns. VA employee participants will be informed that their data collection from interviews will be strictly 
confidential and their participation will be strictly voluntary. Please refer to the e-mail and telephone script 
submitted separately, which explicitly indicate the voluntary nature of participation and the assurance of 
confidentiality of respondents, as well as special rights and protections afforded for VA staff participating as 
research subjects. 
 
Protection against Risk 
As described above, the potential risks to subjects in this study are of low likelihood and generally not 
serious. Nevertheless, we have given these risks careful consideration and have developed a plan to ensure 
that research subjects are protected against them. With respect to ensuring confidentiality and data security, 
we will take the following precautions. We will only collect identifiable data when absolutely necessary and 
scientifically justified. Data will only be stored on VA servers, never leaving the VA system. We note that our 
servers for data storage are located on a separate research-specific password-protected server, providing an 
additional layer of security. Data transmissions between study staff will be minimized, but when necessary, 
will meet or exceed all standards for encryption and security that are in place for the electronic transmission of 
Veterans’ clinical information, with which the Principal Investigator is familiar. Recordings of participants 
during laboratory observations and in-depth interviews will be treated with similar care. The audio recordings 
will be made with a portable digital recording device that will always be maintained under lock-and-key of 
study staff (either in locked file cabinets within a locked office. or, when in transport between facilities, within a 
locked box). The digital recordings will be downloaded to password- protected computer files on the VA 
network server (behind the research firewall), and transcriptions from these recordings will be similarly stored 
and handled. Prior to processing the transcriptions for qualitative analysis, study staff will remove all names 
and other personal identifiers from the transcripts. Transcripts will not be printed routinely. Rather, study 
team members will be instructed to review them as electronic files, to minimize the risk of PHI disclosure. 
We expect these processes, which our team members have employed in a variety of studies both 
within and outside the VA, will be highly effective in protecting subjects from the risks of data security and 
confidentiality breaches. With respect to protecting participants from the risk of psychological distress and 
physical taxation as a result of participating in the interviews, we will take the following precautions.  First, 
we will ensure that VA facilities be accessible to all participants, with special attention to needs of those 
with physical disabilities. Second, we will train the research associate to be sensitive to any discomfort that 
the subject may be experiencing; Dr. Simon has extensive clinical and research experience related to 
interviewing and will ensure that the research associates have sufficient training in this regard. Third, we 
will ensure that sufficient research staff members are present to assist participants in locating restrooms 
and other necessary facilities. 
We do not anticipate any adverse effects of the research to require any medical or 
professional intervention. 
 
 
Aims 1 and 2: Potential benefits of research to subjects and others 
 
This application proposes to develop a secure messaging medication reconciliation tool Intervention to 
improve the post-discharge medication reconciliation process, and compare its effectiveness with treatment as 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 19 of 23 
 
usual. The potential benefits of this research to Veterans include greater accuracy in the post-discharge 
medication reconciliation process, resulting in improved health of Veterans. We also anticipate that this 
intervention, if proven effective and, ultimately, integrated successfully into the post-discharge medication 
reconciliation process, will lead to reduced medication errors, fewer hospitalizations and adverse drug events. 
Finally, this line of research can lead to the demonstration of the utility of an eHealth tool that Veterans with 
less comfort with technology can use, leading the way toward additional behavioral interventions that do not 
add to the clinic's workload. We believe that these benefits do outweigh the minimal risk of harm to research 
subjects, thus justifying the research. 
 
Importance of knowledge to be gained 
 
This project is unique in that it brings together three high priority areas for VA; improving medication 
reconciliation interventions, specifically, secure message based medication reconciliation tools; transforming 
care through the use of healthcare informatics, specifically, the SMMRT tool; and reducing racial and ethnic 
minority health care disparities, through the use of an eHealth tool that has few barriers to use by Veterans 
who may have poor health and computer literacy. If ultimately proven effective, the use of this eHealth tool 
for multiple behavioral issues can be integrated into the interface between primary care and behavioral 
health, and can fit well into the model of Patient Aligned Care Teams. 
The knowledge to be gained from this study will be significant in addressing three high priority areas 
at once, and point to the pathway for using patient-facing technology to meet the 21st century requirements 
for all of our Veterans to be more involved in their own care. It will provide information on integrating eHealth 
tools into primary care as well as the effectiveness of this tool, at least for medication reconciliation. The 
potential knowledge to be gained from this project outweighs the minimal risks to participants. 
 
Data and Safety Monitoring Plan 
 
Dr. Simon, as principal investigator, and the VA Boston IRB will be responsible for monitoring the safety 
of participants. The Data and Safety Monitoring Plan will be carried out by Dr. Simon. Research staff will be 
trained to understand and document adverse and serious adverse events, and to immediately call Dr. Simon in 
the case of such an event. Dr. Simon will immediately notify the IRB of the adverse event and appropriate 
clinical action will be taken. A Case Report Form will be created that will be used to report all adverse 
events. This form will include the participant’s study number and all relevant information about the adverse 
event. Case Report Forms will be filled out immediately by the staff person who is responding to the problem. 
Serious adverse events will be reported to the VA Boston IRB within 48 hours of its occurrence. All other 
adverse events will be reported annually to the IRB and HSR&D. 
 
Inclusion of women, minorities and/or children 
 
Both men and women Veterans will be included in this study. Minorities will be included to the 
greatest extent possible. All Veterans and staff, regardless of race, ethnicity, or sexual orientation will be 
included in this study. This research will not include any children. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 20 of 23 
 
(6) REFERENCES 
 
1. 
Russ, AL, Zillich, AJ, McManus, MS, Doebbeling, BN, Saleem, JJ. Prescribers’ interactions with medication alerts at 
the point of prescribing: A multi-method, in situ investigation of the human–computer interaction. International 
Journal of Medical Informatics. 2012;81(4):232-243. 
2. 
Russ, AL, Zillich, AJ, Melton, BL, Russell, SA, Chen, S, Spina, JR, Weiner, M, Johnson, EG, Daggy, JK, McManus, MS. 
Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-
based simulation. Journal of the American Medical Informatics Association. 2014:amiajnl-2013-002045. 
3. 
Lesselroth, BJ, Felder, RS, Adams, SM, Cauthers, PD, Dorr, DA, Wong, GJ, Douglas, DM. Design and implementation 
of a medication reconciliation kiosk: the Automated Patient History Intake Device (APHID). Journal of the American 
Medical Informatics Association. 2009;16(3):300-304. 
4. 
Jonk, Y, O'Connor, H, Schult, T, Cutting, A, Feldman, R, Ripley, DC, Dowd, B. Using the Medicare Current Beneficiary 
Survey to conduct research on Medicare-eligible veterans. Journal of Rehabilitation Research & Development. 
2010;47(8). 
5. 
Shimizu, E, Glaspy, K, Witt, MD, Poon, K, Black, S, Schwartz, S, Bholat, T, Diaz, N, Kuo, A, Spellberg, B. Readmissions 
at a Public Safety Net Hospital. PloS one. 2014;9(3):e91244. 
6. 
Joynt, KE, Jha, AK. Thirty-day readmissions—truth and consequences. New England Journal of Medicine. 
2012;366(15):1366-1369. 
7. 
VHA Directive 2011-2012: Medication Reconciliation.  Department of Veterans Affairs, Veterans Health 
Administration, Washington, DC. 2011. Available at: 
http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=2390  
8. 
Nebeker, JR, Barach, P, Samore, MH. Clarifying adverse drug events: a clinician's guide to terminology, 
documentation, and reporting. Annals of Internal Medicine. 2004;140(10):795-801. 
9. 
Gandhi, TK, Weingart, SN, Borus, J, Seger, AC, Peterson, J, Burdick, E, Seger, DL, Shu, K, Federico, F, Leape, LL. 
Adverse drug events in ambulatory care. New England Journal of Medicine. 2003;348(16):1556-1564. 
10. 
Gandhi, TK, Burstin, HR, Cook, EF, Puopolo, AL, Haas, JS, Brennan, TA, Bates, DW. Drug complications in 
outpatients. Journal of General Internal Medicine. 2000;15(3):149-154. 
11. 
Ernst, ME, Brown, GL, Klepser, TB, Kelly, MW. Medication discrepancies in an outpatient electronic medical record. 
American journal of health-system pharmacy. 2001;58(21):2072-2075. 
12. 
Manley, HJ, Drayer, DK, McClaran, M, Bender, W, Muther, RS. Drug record discrepancies in an outpatient 
electronic medical record: frequency, type, and potential impact on patient care at a hemodialysis center. 
Pharmacotherapy. 2003;23(2):231-239. 
13. 
Linsky, A, Simon, SR. Medication discrepancies in integrated electronic health records. BMJ quality & safety. 
2012;22(2):103-109. 
14. 
Forster, AJ, Murff, HJ, Peterson, JF, Gandhi, TK, Bates, DW. Adverse drug events occurring following hospital 
discharge. J Gen Intern Med. Apr 2005;20(4):317-323. 
15. 
Calkins, DR, Davis, RB, Reiley, P, Phillips, RS, Pineo, KL, Delbanco, TL, Iezzoni, LI. Patient-physician communication 
at hospital discharge and patients' understanding of the postdischarge treatment plan. Arch Intern Med. May 12 
1997;157(9):1026-1030. 
16. 
Gray, SL, Mahoney, JE, Blough, DK. Adverse drug events in elderly patients receiving home health services 
following hospital discharge. Ann Pharmacother. Nov 1999;33(11):1147-1153. 
17. 
Coleman, EA, Smith, JD, Raha, D, Min, S-j. Posthospital medication discrepancies: prevalence and contributing 
factors. Archives of Internal Medicine. 2005;165(16):1842. 
18. 
Jencks, SF, Williams, MV, Coleman, EA. Rehospitalizations among patients in the Medicare fee-for-service 
program. New England Journal of Medicine. 2009;360(14):1418-1428. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 21 of 23 
 
19. 
Kaboli, PJ, Go, JT, Hockenberry, J, Glasgow, JM, Johnson, SR, Rosenthal, GE, Jones, MP, Vaughan-Sarrazin, M. 
Associations between reduced hospital length of stay and 30-day readmission rate and mortality: 14-year 
experience in 129 Veterans Affairs hospitals. Annals of Internal Medicine. 2012;157(12):837-845. 
20. 
Jack, BW, Chetty, VK, Anthony, D, Greenwald, JL, Sanchez, GM, Johnson, AE, Forsythe, SR, O'Donnell, JK, Paasche-
Orlow, MK, Manasseh, C. A Reengineered Hospital Discharge Program to Decrease Rehospitalization: A 
Randomized Trial. Annals of internal medicine. 2009;150(3):178-187. 
21. 
Resources for Reducing Unnecessary Hospital Readmissions. Agency for Healthcare Research and Quality 
(AHRQ);2014. Available at http://www.pso.ahrq.gov/Topics#resources. Accessed: 1 June 2014. 
22. 
Forster, AJ, Clark, HD, Menard, A, Dupuis, N, Chernish, R, Chandok, N, Khan, A, van Walraven, C. Adverse events 
among medical patients after discharge from hospital. Canadian Medical Association Journal. 2004;170(3):345-
349. 
23. 
National Patient Safety Goals for 2014. 2013. Available at: 
http://www.jointcommission.org/assets/1/6/2014_HAP_NPSG_E.pdf.  
24. 
Kwan, JL, Lo, L, Sampson, M, Shojania, KG. Medication Reconciliation During Transitions of Care as a Patient Safety 
StrategyA Systematic Review. Annals of internal medicine. 2013;158(5_Part_2):397-403. 
25. 
Mueller, SK, Sponsler, KC, Kripalani, S, Schnipper, JL. Hospital-Based Medication Reconciliation Practices: A 
Systematic Review. Archives of Internal Medicine. 2012;172(14):1057-1069. 
26. 
Boockvar, KS, Santos, SL, Kushniruk, A, Johnson, C, Nebeker, JR. Medication reconciliation: barriers and facilitators 
from the perspectives of resident physicians and pharmacists. Journal of Hospital Medicine. 2011;6(6):329-337. 
27. 
Lesselroth, B, Adams, S, Felder, R, Dorr, DA, Cauthers, P, Church, V, Douglas, D. Using consumer-based kiosk 
technology to improve and standardize medication reconciliation in a specialty care setting. Joint Commission 
Journal on Quality and Patient Safety. 2009;35(5):264-271. 
28. 
Lesselroth, B, Adams, K, Tallett, S, Wood, S, Keeling, A, Cheng, K, Church, V, Felder, R, Tran, H. Design of admission 
medication reconciliation technology: a human factors approach to requirements and prototyping. HERD. 
2012;6(3):30-48. 
29. 
Nazi, KM, Weaver, F. The My HelatheVet Peronal Health Record Portal in 2013: New Features, Study Findings, and 
Opportunities. VIReC Cyberseminar. 2013. Veterans Health Administration. Available at. 
http://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/632-notes.pdf Accessed 2 December 
2013. 
30. 
Nazi, KM, Hogan, TP, McInnes, DK, Woods, SS, Graham, G. Evaluating patient access to electronic health records: 
results from a survey of veterans. Medical Care. 2013;51:S52-S56. 
31. 
Houston, TK, Volkman, JE, Feng, H, Nazi, KM, Shimada, SL, Fox, S. Veteran Internet Use and Engagement With 
Health Information Online. Military medicine. 2013;178(4):394-400. 
32. 
McInnes, DK, Solomon, J, Bokhour, B, Asch, S, Ross, D, Nazi, K, Gifford, A. Use of electronic personal health record 
systems to encourage HIV screening: an exploratory study of patient and provider perspectives. BMC research 
notes. 2011;4(1):295. 
33. 
Shimada, SL, Hogan, TP, Rao, SR, Allison, JJ, Quill, AL, Feng, H, Phillips, B, Nazi, KM, Haidary, ST, Houston, TK. 
Patient-provider secure messaging in VA: variations in adoption and association with urgent care utilization. 
Medical Care. 2013;51(3 Suppl 1):21-28. 
34. 
Goldzweig, CL, Orshansky, G, Paige, NM, Towfigh, AA, Haggstrom, DA, Miake-Lye, I, Beroes, JM, Shekelle, PG. 
Electronic Patient Portals: Evidence on Health Outcomes, Satisfaction, Efficiency, and AttitudesA Systematic 
Review. Annals of Internal Medicine. 2013;159(10):677-687. 
35. 
Wagner, EH. Chronic disease management: what will it take to improve care for chronic illness? Effective Clinical 
Practice. 1997;1(1):2-4. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 22 of 23 
 
36. 
Wagner, EH, Austin, BT, Von Korff, M. Organizing care for patients with chronic illness. The Milbank Quarterly. 
1996:511-544. 
37. 
Coleman, K, Austin, BT, Brach, C, Wagner, EH. Evidence on the Chronic Care Model in the new millennium. Health 
Affairs. 2009;28(1):75-85. 
38. 
Heyworth, L, Paquin, AM, Clark, J, Kamenker, V, Stewart, M, Martin, T, Simon, SR. Engaging patients in medication 
reconciliation via a patient portal following hospital discharge. Journal of the American Medical Informatics 
Association. 2013:amiajnl-2013-001995. 
39. 
Heyworth, L, Clark, J, Marcello, TB, Paquin, AM, Stewart, M, Archambeault, C, Simon, SR. Aligning Medication 
Reconciliation and Secure Messaging: Qualitative Study of Primary Care Providers' Perspectives Journal of Medical 
Internet Research. 2013;15(12). 
40. 
Bergman AA, JH, Frankel RM, Kobylinski AL, Gonzalvo JD, Hudmon KS, Zillich AJ. Pharmaceutical Role Expansion 
and Developments in Pharmacist-Physician Communication. Health Communication In press 2014. 
41. 
Haggstrom, DA, Saleem, JJ, Russ, AL, Jones, J, Russell, SA, Chumbler, NR. Lessons learned from usability testing of 
the VA's personal health record. Journal of the American Medical Informatics Association. 2011;18(Suppl 1):i13-
i17. 
42. 
Russ, AL, Weiner, M, Russell, SA, Baker, DA, Fahner, WJ, Saleem, JJ. Design and Implementation of a Hospital-
Based Usability Laboratory: Insights from a Department of Veterans Affairs Laboratory for Health Information 
Technology. Joint Commission Journal on Quality and Patient Safety. 2012;38(12):531. 
43. 
Schnipper, JL, Gandhi, TK, Wald, JS, Grant, RW, Poon, EG, Volk, LA, Businger, A, Williams, DH, Siteman, E, Buckel, L. 
Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial. Journal 
of the American Medical Informatics Association. 2012;19(5):728-734. 
44. 
Gadd, CS, Ho, Y-X, Cala, CM, Blakemore, D, Chen, Q, Frisse, ME, Johnson, KB. User perspectives on the usability of 
a regional health information exchange. Journal of the American Medical Informatics Association. 2011;18(5):711-
716. 
45. 
Payne, TH, Detmer, DE, Wyatt, JC, Buchan, IE. National-scale clinical information exchange in the United Kingdom: 
lessons for the United States. Journal of the American Medical Informatics Association. 2011;18(1):91-98. 
46. 
Jaspers, MW. A comparison of usability methods for testing interactive health technologies: Methodological 
aspects and empirical evidence. International Journal of Medical Informatics. 2009;78(5):340-353. 
47. 
Smelcer, J, Miller-Jacobs, H, Kantrovich, L. Usability of electronic medical records. Journal of usability studies. 
2009;4(2):70-84. 
48. 
Johnson, C. Why did that happen? Exploring the proliferation of barely usable software in healthcare systems. 
Quality and Safety in Health Care. 2006;15(suppl 1):i76-i81. 
49. 
Callahan, CM, Unverzagt, FW, Hui, SL, Perkins, AJ, Hendrie, HC. Six-item screener to identify cognitive impairment 
among potential subjects for clinical research. Medical Care. 2002;40(9):771-781. 
50. 
Lewis, CL, Golin, CE, DeLeon, C, Griffith, JM, Ivey, J, Trevena, L, Pignone, M. A targeted decision aid for the elderly 
to decide whether to undergo colorectal cancer screening: development and results of an uncontrolled trial. BMC 
medical informatics and decision making. 2010;10(1):54. 
51. 
Nielsen, J. Estimating the number of subjects needed for a thinking aloud test. International journal of human-
computer studies. 1994;41(3):385-397. 
52. 
Kushniruk, AW, Patel, VL. Cognitive and usability engineering methods for the evaluation of clinical information 
systems. Journal of biomedical informatics. 2004;37(1):56-76. 
53. 
VHA Directive 1164: Essential Medication Information Standards.  Department of Veterans Affairs, Veterans 
Health Administration, Washington, DC. 2013. Available at: In concurrence, pending approval of the Under 
Secretary for Health. 

SMMRT Trial, IRB#3156, Protocol Version 2.0, August 31, 2018 
 
Page 23 of 23 
 
54. 
Hix, D, Hartson, HR. Developing user interfaces: ensuring usability through product & process. New York: John 
Wiley & Sons, Inc.; 1993. 
55. 
Lewis, JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. 
International Journal of Human-Computer Interaction. 1995;7(1):57-78. 
56. 
Russ, AL, Baker, DA, Fahner, WJ, Milligan, BS, Cox, L, Hagg, HK, Saleem, JJ. A rapid usability evaluation (RUE) 
method for health information technology. Paper presented at: AMIA Annual Symposium Proceedings2010. 
57. 
Roland, M, Torgerson, DJ. Understanding controlled trials: What are pragmatic trials? BMJ. 1998;316(7127):285. 
58. 
Clifford, PR, Maisto, SA. Subject reactivity effects and alcohol treatment outcome research. Journal of Studies on 
Alcohol and Drugs. 2000;61(6):787. 
59. 
Ogborne, AC, Annis, HM. The reactive effects of follow-up assessment procedures: An experimental study. 
Addictive behaviors. 1988;13(2):123-129. 
60. 
Schulz, KF, Grimes, DA. Generation of allocation sequences in randomised trials: chance, not choice. The Lancet. 
2002;359(9305):515-519. 
61. 
McInnes, DK, Solomon, JL, Shimada, SL, Petrakis, BA, Bokhour, BG, Asch, SM, Nazi, KM, Houston, TK, Gifford, AL. 
Development and evaluation of an internet and personal health record training program for low-income patients 
with HIV or hepatitis C. Medical Care. 2013;51(3 Suppl 1):S62-66. 
62. 
Cameron, KA, Ross, EL, Clayman, ML, Bergeron, AR, Federman, AD, Bailey, SC, Davis, TC, Wolf, MS. Measuring 
patients’ self-efficacy in understanding and using prescription medication. Patient education and counseling. 
2010;80(3):372-376. 
63. 
Coleman, EA, Mahoney, E, Parry, C. Assessing the quality of preparation for posthospital care from the patient's 
perspective: the care transitions measure. Medical Care. 2005;43(3):246-255. 
64. 
Bass, PF, Wilson, JF, Griffith, CH. A shortened instrument for literacy screening. Journal of General Internal 
Medicine. 2003;18(12):1036-1038. 
65. 
Kripalani, S, Roumie, CL, Dalal, AK, Cawthon, C, Businger, A, Eden, SK, Shintani, A, Kelly Cunningham Sponsler, M, 
Harris, LJ, Theobald, C. Effect of a Pharmacist Intervention on Clinically Important Medication Errors After Hospital 
Discharge. Ann Intern Med. 2012;157:1-10. 
66. 
Goldman, RE, Soran, CS, Hayward, GL, Simon, SR. Doctors' perceptions of laboratory monitoring in office practice. 
Journal of evaluation in clinical practice. 2010;16(6):1136-1141. 
67. 
Patton, MQ. Qualitative evaluation and research methods. SAGE Publications, inc; 1990. 
68. 
Kuzel, A. Sampling in qualitative Inquiry. In: Crabtree, BF, Miller, W, eds. Doing Qualitative Research. Vol 2. 
Thousand Oaks, CA: Sage Publications; 1999:33-45. 
69. 
Borkan, J. Immersion/crystallization. Doing Qualitative Research. Vol 2. Thousand Oaks, CA: Sage Publishers; 
1999:179-194. 
70. 
Crabtree, BF, Miller, W. Using Codes and Code Manuals: A Template Organizing Style of Interpretation. In: 
Crabtree, BF, Miller, W, eds. Doing Qualitative Research. Vol 2. Thousand Oaks, CA: Sage Publications; 1999:163-
177. 
71. 
NVivo qualitative data analysis sofware [computer program]. Version 10: QSR International Pty Ltd; 2013. 
72. 
Altheide, DL, Johnson, JM. Criteria for assessing interpretive validity in qualitative research. Handbook of 
Qualitative Research. London: Sage Publishers; 1994:485-499.