A Mobile Personal Health Record for Behavioral Health Homes
Phase: NA
Conditions: Hypertension, Hyperlipidemia, Diabetes
Sponsor: Emory University

 
 
 
A Mobile Personal Health Record for Behavioral Health Homes 
IRB00067447  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
A Mobile Personal Health Record for Behavioral Health Homes 
 
Protocol Dated June 13th, 2017 
 
NCT01890226 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
Project Summary/Abstract 
 
Poor quality of medical care is a major contributor to excess medical morbidity and premature mortality 
in persons with serious mental illnesses (SMI). To address this problem, community mental health providers 
are increasingly partnering with safety net medical providers to develop behavioral health homes, integrated 
clinics in which persons with SMI receive coordinated medical and mental health care. 
However, behavioral 
health homes have faced logistical and privacy challenges in integrating electronic medical records across 
organizations. This application proposes to develop and test a mobile Personal Health Record (mPHR) to 
overcome this problem while more fully engaging patients in their health care. The mPHR will have the 
capability to access medical and mental health medication and lab data in real time; to help clients set and 
maintain health and lifestyle goals; to provide medication and appointment prompts and reminders; and to 
facilitate communication with providers via asynchronous communication with the EHRs. 
This three-phase study will develop, test, and disseminate the mPHR. During the first phase, we will 
develop the app building on experience and preliminary data from a PC-based PHR project, and link it to the 
medical and mental health EHR in a behavioral health home. During the second phase, we will conduct a 
randomized trial of the mPHR in 300 subjects randomized to the mPHR or usual care. The study will use an 
adaptive design, which will implement the app in five 6-month cohorts of 60 subjects. After each cohort, the 
app will be updated based on feasibility and acceptability findings, new research and commercial 
developments in PHR apps, and changes in usual care in the clinic and community practice. Analyses will 
examine overall effects of the intervention, and whether there is a change in its benefits across study cohorts. 
Study outcomes will include quality of preventive and cardiovascular care; patient activation and provider 
management of chronic illness; and service use including medication compliance, missed appointments, 
emergency department use, and costs of care from a societal and managerial perspective. During the final 
phase of the intervention, the app will be disseminated to behavioral health homes across the country through 
a national technical assistance center. 
The proposed study will apply a novel technology that has not previously been studied in mental 
health settings, will be conducted in a new organizational model of care that is anticipated to become 
widespread in community mental health in the coming years, and will use an innovative adaptive design that 
will ensure that the intervention remains relevant and updated throughout the study period. A multidisciplinary 
team including mental health services researchers, experts in information technology and app design, and 
policymakers with expertise in public sector integration will help ensure that the study results will help 
transform care delivery in mental health safety net settings. 

Project Narrative 
 
This study will be the first to test a mobile PHR for use in behavioral health homes.  If successful, it will 
provide a vital tool for empowering patients and improving quality of care in these emerging models of service 
delivery. 

II. SPECIFIC AIMS 
A long literature has demonstrated that persons with serious mental illnesses have poor quality of 
medical care, 1 which is a major contributor to excess medical morbidity and premature mortality.2  In response 
to this public health problem, an emerging model of care has developed - the behavioral health home – a clinic 
based in a community mental health center, typically developed in partnership with a safety net medical 
provider, where persons with serious mental illnesses receive coordinated medical and mental health care.3 
While these are promising models of service delivery, they have faced challenges reaching their full potential to 
improve quality and outcomes of health care. 4  A 2011 survey of 56 behavioral health homes in a national 
demonstration program found difficulties in integrating electronic medical records across facilities to be the 
single largest challenge in successfully implementing these clinics.5 Furthermore, there have also been 
limitations in fully engaging consumers in their care delivery,4 one of the most important features of organized 
models of care delivery. 6 
In general medical populations, electronic Personal Health Records (PHRs) have shown considerable 
promise in reshaping healthcare by shifting the ownership and locus of health records from being scattered 
across multiple providers to an approach that is longitudinal and person-centered. With the growing use and 
capabilities of smartphones even amongst disadvantaged sectors of the population, there is an opportunity to 
develop PHRs for mobile platforms, or mobile Personal Health Records (mPHRs).7    By communicating with 
patients’ medical and mental health Electronic Health Records (EHRs), mPHRs can deliver real-time 
information, assist in adherence to medication regimens and appointments, and facilitate communication 
between patients and providers.8  As such, mPHRs have the potential both to help foster better coordination of 
services and consumer engagement in care and hold great promise for individuals with serious mental illness. 
This application proposes to develop and test a mPHR designed to improve quality of care in behavioral 
health homes. The mPHR will have the capability to access medical and mental health medication and           
lab data in real time; to provide medication and appointment prompts and reminders; to help clients set and 
maintain health and lifestyle goals; and to facilitate communication with providers. An adaptive randomized 
design,9 incorporating updates to the intervention reflecting developments in mobile health technologies and in 
the organization of behavioral health home services, will be used to optimize relevance and effectiveness of the 
intervention throughout the study period. 
The proposed project will address three specific aims: 
Specific Aim 1: Develop a smartphone Android application (app) that will link the medical and mental health 
EHRs in a behavioral health home. 
Specific Aim 2: Evaluate, using an adaptive, randomized controlled study design, the effects of the mPHR on 
patient and provider indicators in a sample of 5 sequential cohorts of 60 subjects, half of whom will receive the 
mPHR (300 subjects total). 
H1: As compared with the treatment as usual group, subjects with an mPHR will have higher performance on 
indices of cardiometabolic and preventive quality of care. 
H2: As compared with the treatment as usual group, subjects with an mPHR will have greater improvement in 
patient activation and provider management of chronic illness. Patient activation and provider management of 
chronic illness will partly mediate the relationship between the intervention and quality of care. 
Specific Aim 3: Evaluate the impact of the mPHR on service use and costs of care. 
H3: As compared with the treatment as usual group, subjects with a mPHR will have better medication 
compliance, fewer missed appointments, fewer ED visits, and have a modest increase in costs. Medication 
compliance, missed appointments, reduced ED use, and regular mPHR use will, in part, mediate the 
relationship between the intervention and costs of care. 
The proposed study is highly innovative in the intervention, the setting, and the study design: 1) This 
study applies a novel technology that has not previously been studied in mental health settings; 2) The 
research will be conducted in a new organizational model of care, the behavioral health home, that is 
anticipated to become widespread in community mental health in the coming years4; and 3) This research uses 
an adaptive research design that will ensure that the intervention remains relevant using periodic updates 
throughout the study period.9  The potential impact of the intervention is supported by pilot data suggesting 
feasibility, acceptability, and potential benefits of PC-based PHRs, and a dissemination plan that will allow for 
widespread use of the mPHR in behavioral health homes and, more broadly, in community mental health 
centers nationwide. The multidisciplinary team of researchers, leaders from the community behavioral health 
field, and technology experts is uniquely qualified to conduct the proposed project to maximize its potential to 
transform care in safety net behavioral health settings. 

III. RESEARCH STRATEGY 
IIIA. Background and Significance 
A literature extending back more than 70 years has demonstrated the high rates of medical morbidity 
and excess mortality in persons with serious mental illness (SMI).10 Mortality due to general medical conditions 
is elevated by two to three times in persons with severe mental disorders,11, 12  and this differential mortality gap 
appears to be increasing over time.13, 14 These troubling statistics have led to efforts among federal and state 
policymakers, clinicians, and mental health consumers to seek to identify opportunities to improve health and 
reduce mortality in this population.15, 16 Improving quality of medical care is a first critical step in reducing 
cardiovascular mortality in particular and extending lifespans of persons with serious mental illnesses.17, 18 
Because of the specialized needs and challenges in accessing primary care clinics, there has been a 
recent growth of interest by policymakers and clinical leaders in developing behavioral health homes --clinics 
based in mental health settings where persons with serious mental illnesses receive coordinated medical and 
mental health care.3  In 2009, SAMHSA began funding a series of demonstration projects through its Primary 
Care Behavioral Health Integration (PBHCI) initiative.19 By the end of 2012, 96 such homes will be funded 
nationwide through the PBHCI program. More than three-quarters of these specialty homes have been 
developed as partnerships between community mental health centers (CMHCs) and primary care provider 
organizations, the majority of which (67%) are Federally Qualified Health Centers (FQHCs).5 Outside of this 
demonstration project, a growing number of CMHCs are developing partnerships with safety net medical 
providers as a means of caring for their patients’ medical needs. In Georgia alone, 16 out of the state’s 27 
CMHCs have developed partnerships with public sector medical providers with the goal of jointly caring for 
patients’ medical and mental health needs. Missouri was recently awarded one of the nation’s first Medicaid 
demonstration health home waivers, which will develop health homes for patients with comorbid conditions 
through partnerships between CMHCs and FQHCs.20 
While specialty mental health homes present a unique opportunity to improve physical health care for 
persons with serious mental illnesses, they have also faced challenges in implementation. While all CMHC and 
FQHC partners in these homes are required to have EHRs, CMHCs and FQHCs rarely use the same health 
record vendors; thus, communication across clinics, even when they are in the same building, is limited by both 
technical and medico-legal barriers and ultimately translates to obstacles in serving the consumer. These 
organizations typically have different procedures for order entry and charting (e.g. note sign offs) and separate 
privacy policies governing access to, and use of, the records.  A 2011 RAND survey of the first two cohorts of 
PBHCI grantees found that difficulties in integrating electronic medical records across facilities was the single 
largest challenge in implementing behavioral health homes.5 Finally, these sites have generally used a medical 
model with a limited focus on recovery and self-management. It is therefore a priority to look for opportunities to 
more fully engage consumers in their health care.4 
PHRs hold promise for linking services in fragmented and under-resourced public sector settings.21 
The Markle Foundation has defined a personal health record as “An electronic application through which 
individuals can access, manage, and share their health information, and that of others for whom they are 
authorized, in a private, secure, and confidential environment.”22  Although the empirical literature on PHRs is 
limited, existing studies show promise for PHRs as tools for improving care. In a qualitative study by Tang et al, 
patients reported that providing printed summary information to patients at the end of a clinic visit improved their 
understanding of their care, enhanced their relationships with providers, improved their satisfaction with care, 
and motivated them to adhere to treatment plans.23  Jerden and Weinehall found that 25% of patients who 
received a PHR booklet reported making one or more lifestyle change, most commonly by increasing physical 
activity or improving diet.24 PHRs that have been developed as extensions of EHRs within particular health 
systems have had higher rates of uptake than freestanding PHRs that do not automatically provide consumers 
access to health data. 32  Similarly, patient portals, extensions of EHRs that provide patients with access           
to the most current laboratory and medication information, have been shown to increase patient and provider 
interaction leading to better management of chronic illness including chronic depression. 33, 34  While no   
existing studies have been conducted with mobile PHRs via smartphone, these would be expected to be 
particularly beneficial for poor populations who may have limited access to personal computers with high speed 
internet access.25 
Mobile PHRs might be expected to help improve quality of care for patients with serious mental 
illnesses by overcoming patient and provider barriers to care seen in public sector mental health settings. 17 
(see Figure 1). Patient factors include lack of familiarity with current treatments, cognitive limitations that may 
lead to missed appointments or medication dosages, and difficulty in effectively communicating with 
providers.1, 17  These constructs can be summarized in Hibbard’s concept of patient activation, which 

comprises an individual’s capacity to manage his or her own illness and work effectively within the formal 
health system. 26-28 PHRs are hypothesized to help enhance activation by improving knowledge and 
engagement with care and allowing the patient to set and track health goals. Reminders and prompts can help 
patients improve adherence to medications and upcoming appointments. 
At the provider level, while most behavioral health homes have co-located services, they are not fully 
integrated, and providers face challenges in accessing EHRs from partner organizations largely due to 
technical incompatibilities in the EHR systems 4, 5  Mobile PHRs can help address these provider barriers by 
providing a single, consumer-controlled record that spans multiple systems of care, and facilitating 
communication across organizations and between patients and providers. 
 
Figure 1: Conceptual model for mPHR 
 
 
A 2011 survey found that 72% of persons with SMI from a large rehabilitation facility owned and 
regularly used a cell phone or smartphone, suggesting the appropriateness of the use of mobile apps to 
improve care in these populations.29 However, we are not aware of any PHRs in that have been developed for 
this population in the research literature or commercial space. 
 
IIIB. Approach 
IIIB1. Overview of Proposed Research Project and Potential Innovation 
IIIB1a: Overview: The proposed project will develop and test a mobile PHR to improve quality of medical care in 
behavioral health homes. In the initial stage, a mobile PHR will be developed for the Android platform, building 
on pilot data from a PC-based PHR developed by the study team. Subsequently, the app will be studied     
using an adaptive randomized controlled study among patients treated in a behavioral health home, updating 
the app every 6 months based on data on feasibility and acceptability. In the final phase of the project,           
we will work through the National Council for Community Behavioral healthcare to disseminate the app to 
behavioral health homes and other CMHCs nationwide. 
IIIB1b. Innovation: The proposed study is innovative in at least three ways: 1. It will apply a novel technology 
that has not previously been used in behavioral health homes or other mental health settings. 2. It will be 
conducted in a new organizational model of care, the behavioral health home, that is anticipated to become 
widespread in community mental health settings in coming years and 3. It uses an innovative, adaptive 
research design that will ensure that the intervention remains relevant throughout the study period. 
IIIB2: Preliminary Data 
IIIB2a. Study team experience and expertise: The study team includes a unique blend of researchers, 
information technology experts, and policymakers that will optimize the potential both for successful completion 
and maximal public health impact. Benjamin Druss MD, the principal investigator, is a psychiatrist and professor 
of health policy at the Rollins School of Public Health at Emory University whose work focuses on improving 
health and healthcare for persons with serious mental illnesses. He is the principal investigator on the       
HOME study, a randomized study of a partnership-based behavioral health home for persons with serious 
mental illness, and the AHRQ-funded My Health Record study, which developed and pilot tested the first PC- 
based PHR for patients with serious mental illness. He was a member of the planning group for the 
NIMH/AHRQ sponsored conference “Mental Health & Health IT Research: The Way Forward,” which helped 
define gaps in research in information technology in mental health. Silke von Esenwein, PhD has worked on all 
of these projects as a co-investigator; she has presented extensively about the use of health information 
technology among persons with serious mental illnesses. Paul Weiss, MS is a biostatistician with extensive 
experience in analyzing clinical trials as well as in mobile app design. Ellen Zegura, PhD is professor and Chair 
in the Department of Computer Science at the Georgia Institute of Technology. She is an expert in developing 
and testing technology interventions including mobile applications in underserved populations. Elaine Warren,  
M.S.C., is an IT consultant with over ten years of experience in development and operation of information 
Mobile 
PHR 
Patient Factors 
⇑Activation 
⇑Adherence 
⇓Missed appts 
⇑Quality of Care 
⇑Outcomes 
⇔ Cost 
Provider factors 
⇑Coordination of care 

systems including mobile application design. Michael Lardiere MSW is the Vice President of Health Information 
Technology and Strategic Development for the National Council for Community Behavioral Healthcare and one 
of the nation’s experts in the use of IT in safety net health and mental health settings, as well as in newly 
emerging behavioral health home models. 
IIIB2b. Developing and pilot-testing a PC-based Personal Health Record for persons with SMI: “An Electronic 
Personal Health Record for Mental Health Consumers” developed and pilot-tested the first PHR for persons 
with SMI and medical comorbidity, called My Health Record. This PHR was developed as a collaborative effort 
between Emory investigators, experts in development and implementation of medical PHRs, and mental health 
consumer leaders. 
Because it was 
implemented in a CMHC 
without an EHR, it was 
developed as a 
standalone program, in 
which patients entered all 
data into a PC-based 
PHR with the help of their 
care manager. 
IIIB2c. Computer literacy  
curriculum: At baseline, 
approximately 1/3 of 
clients reported low levels 
of computer literacy. 
Therefore we developed a 
curriculum to increase 
computer literacy for 
persons with serious 
mental illness, including 
those with cognitive 
impairment. In a pre-post 
test of the curriculum, a 
majority of participants 
greatly increased their knowledge of the internet, ability to access health information and community 
resources, needing an average of 2.2 hours to complete the training modules. 
IIIB2d. Feasibility: A total of 170 subjects were successfully randomized, and there has been a greater than 
90% retention rate in the study through the 1-year follow up. Using online tracking of accessing the PHR, 71/85 
(83.6%) subjects in the intervention group have accessed the PHR at least monthly to obtain health information 
using computers at public libraries, in their own home, and at internet cafes. 
IIIB2e. Potential effectiveness in improving patient activation and quality of care: The two primary outcomes for 
this pilot study were patient activation and use of preventive services (as recommended by the US Preventive 
Services Taskforce). Each was balanced at baseline, but showed clinically and statistically significant 
improvements at 12-month follow-up. At 12-month follow up, patients in the PHR intervention had an average 
patient activation score of 59.9 (out of a possible score of 100) compared to 55.2 for the control group. The  
total proportion of recommended preventive medical services was double that in the case versus the control 
group (40% versus 20%, p<0.001) (See Table 1). 
 
Table 1: Outcomes of PC-Based Personal Health Record Pilot 
Wave 
Control (n=85) 
Case (n=85) 
p value 
Patient Activation 
Baseline 
55.6 ± 14.4 
55.1 ± 14.3 
0.83 
12-Month 
55.2 ± 14.6 
59.9 ± 12.5 
0.043 
Total Preventive Services 
Baseline 
0.27 ± 0.09 
0.26 ± 0.10 
0.22 
12-Month 
0.20 ± 0.11 
0.40 ± 0.13 
<0.001 

IIIB2f. Focus Groups: We conducted two focus groups with participants in the PHR study, one with participants 
who used the PHR monthly throughout the study (83.7% of the intervention group), to assess acceptability and 
experience in using it. We conducted a second focus group among non-users of the PHR (16.3% of the 
intervention group). The groups were convened and conducted using standard procedures as described by 
Krueger et al.30 31-34  Acceptability among PHR Users: Among regular PHR users in the first focus group (n=8), 
clients expressed a high degree of enthusiasm for the PC-based PHR. They described using it to track 
treatments and appointments: “It’s great, I use it to stay on top of my meds and appointments with my doctor.” 
They described how it made them more aware of their self management needs. “It helps me remember to take 
my blood sugar and work on keeping it down.” Finally, several described it as supporting their broader recovery 
goals. “I feel like it lets me make progress on my whole heath, helps me keep the physical together with the 
mental and spiritual.” Reasons for not using the PHR among non-users: In the focus group of participants who 
did not use the PHR at least once per month (n=7), we examined the reasons for non-use. The two most 
common reasons for non-use were 1. Difficulty/time spent in entering PHR data “It’s a good idea, but it takes too 
much time trying to put everything in about my meds and all.” and 2. Lack of easy access to a computer “I use 
the one at the library, but the library closes at 7. How am I supposed to look at it after that?” 
Acceptability of using mobile phone interventions and potential interest in using a mobile PHR in the  
population: In both of the groups, participants were asked about potential interest in a smartphone-based 
mobile PHR as an alternative to the PC-based PHR. Participants who did not own smartphones (11/15 total) 
generally found it easy to use the demonstration smartphone in the focus group and found the interface 
relatively easy to use. Participants reported, “This is like a computer but it’s easier to use.” “It’s easy to see 
since it’s all lit up”. Those who used PHRs felt that a mobile platform could provide added ease of accessibility 
“I’d like that since I could always have it in my pocket.” Those who did not use their PHR felt that it could help 
overcome the barriers to use. “Yeah, if I didn’t have to write all that in there it would help….I wouldn’t have to 
find a computer to use it…” 
IIIB2g. Summary: Preliminary qualitative and quantitative data using a PC-based PHR suggest high potential 
feasibility and acceptability, along with potential benefits on patient activation and quality of primary medical 
care. The primary barriers to use of the PHR – time and effort needed in entering data, and common lack of 
access to a computer – could both be overcome by using a mobile PHR platform. 
 
IIIB3 Study Site 
The Four Corners clinic in Austell, Georgia was established in 2010 as one of the first behavioral health 
homes in the country funded under SAMHSA’s Primary Care Behavioral Health Integration (PBHCI) grants. 
The clinic is a satellite clinic of the West End Medical Center Federally Qualified Health Center (FQHC), co- 
located within the Cobb County Community Mental Health Center (CMHC). The Four Corners clinic is staffed 
by a full time nurse practitioner and RN from the FQHC, and two care coordinators who are paid for by the 
CMHC. It provides comprehensive primary and preventive care to patients with one or more cardiometabolic 
risk factor (high blood pressure, diabetes, elevated cholesterol). This partnership model between an FQHC 
and a CMHC is the most prevalent organizational model used in behavioral health homes. 5 
Table 2 presents the demographic and clinical characteristics of patients who are currently seen in the 
Four Corners behavioral health home. A total of 420 patients have been enrolled since 2010, with an 
enrollment rate of approximately 200 new patients per year. The 
Table 2: Characteristics of Four Corners Clinic 
target enrollment goal will be 1,000 patients. A majority of the 
sample have elevated BMI (60%), blood pressure (77%), and 
elevated LDL cholesterol (64%). 
Characteristics of the EHRs: Both the West End Medical FQHC 
and the Cobb County CMHC have fully functional EHR; 
however, these EHRs do not currently have the ability to share 
data or information across the two organizations. Clinicians in 
the Four Corners clinic enter notes into the West End Medical 
Clinic record, then make electronic copies of the notes, and 
then, cut and paste these into the Cobb County EHR. 
The Cobb County CMHC uses Carelogic, a behavioral 
health EHR that is in use in over 100 CMHCs in 26 states. West End Medical Center uses NextGen 
Ambulatory EHR, one of the most widely used ambulatory EHRs in FQHCs nationwide.35 Each of the two 
EHRs has a patient portal feature that allows extraction of selected fields from the record to a website or 
server. Fields that can be selected include laboratory values, medications, and upcoming appointments. Each 
Va riable 
% 
Demographics 
Femal e 
65% 
African-American 
22% 
Age 25 to 34 years old 
17% 
Age 35 to 44 years old 
43% 
Age 45 to 54 years old 
16% 
Cardiometabolic Risks 
Borderli ne or High BP 
77% 
Borderli ne or High HgB A1C 
41% 
High cholesterol 
50% 

of the patient portal features also has the capability to provide asynchronous email communication with the 
provider team. The portals are protected by firewalls and allow for HIPAA-compliant, secure data transfer. 
 
IIIB4: Specific Aim 1: Develop a smartphone Android application (app) and link it to the medical and 
mental health EHR in a behavioral health home. The application will be developed during the first 9 months 
of the study. After hiring staff and establishing the study team, a beta version of the app will be constructed, 
incorporating and building on features developed for the PC-based PHR, then subsequently refined using 
qualitative techniques focusing both on design and content of the mPHR. 
IIIB4a. App Development:The app will be programmed in Android, an open-source, Linux-based operating 
system for smartphones with a Java programming interface. The mobile Android app will be developed using 
the Android Software Development Kit. 36 The app will be developed to link with the mobile version of Microsoft 
HealthVault, a web-based platform developed by Microsoft to store health information for PHR. HealthVault 
supports a number of exchange formats including all major industry standards, making it possible to integrate 
with most electronic PHRs. The HealthVault security system works with cryptographic constructs available 
more widely on smartphones to preserve user privacy. HealthVault supports mobile platforms by providing 
open-source, community-supported client libraries.37 An app developer supervised by Elaine Warren M.S.C. will 
develop this app. 
Data fields from the EHR from each of the participating sites will be uploaded to the HealthVault server 
using the patient portal feature on each of the EHRs. Data are stored in XML format; with user consent, multiple 
user records can be downloaded from the HealthVault servers for aggregate analysis.37  Each enrolled patient 
will be invited to register in the system and given a unique identification and an encrypted password. 
Passwords are a minimum of eight characters and are required to use mixed case letters, numbers, and 
punctuation. This information will be used to populate fields on the handheld device from data in the CMHC 
EHR and the FQHC EHR using a user-friendly interface tailored to the needs and potential cognitive limitations 
of the population. To ensure privacy and safety of data, no health data will be stored on the mobile device – all 
will be maintained on the HealthVault server. 
IIIB4b Mobile PHR structure:  
The mPHR will comprise four sections: 1. Information downloaded directly from the EHRs; 2. 
Information entered by consumers; 3. Prompts and reminders for medications and upcoming appointments, 
and self-management; and 4. Communication with the care team. 
To address limitations in computer 
literacy and cognitive deficits in some of the 
population, information will be presented 
using a simple user interface and graphics, 
suitable for a 6th grade reading level. A 
“traffic light” system38 will be used to 
represent laboratory and anthropomorphic 
findings, indicating whether values are 
normal (green), borderline (yellow) or 
abnormal (red). 
i. Information directly downloaded from the  
EHR: A PHR can help improve patient 
activation by helping provide patients with 
the health information they need to better 
manage their own illnesses.39 Providing this information in a single, patient-controlled record may also help 
providers to better coordinate the care they deliver and make it more patient-centered.40 Using the patient 
portal feature on the two EHRs, the mPHR will download to HealthVault and then display in the mPHR: 
1. Current medications will be presented, including separate categories for medical and psychotropic 
medications, along with the indications. 
2. Any known allergies.(food, medication, or otherwise) and contraindications will be displayed. 
3. The PHR will present lab values including glucose, cholesterol, and hemoglobin A1c, along with a traffic 
light indicator of whether the values are at target or abnormal. 
4. Anthropomorphic measures will be displayed, including blood pressure, weight, BMI and waist 
circumference, with a traffic light indicating whether values are abnormal, borderline, or normal. 
5. Dates and upcoming appointments (medical and psychiatric) will be provided. 
6. Name and contact information for all members of the provider team will be included. 
Figure 2: Mobile PHR structure 

ii. Subjects will have the ability to enter data on personal goals either directly into the app or on a PC, via the 
secured HealthVault website. Fields entered by the patient will include: 
1. Current resources and health care needs: This section includes details that the patient identifies as 
potentially relevant for healthcare professionals, including: disabilities; challenges in transportation; need for a 
translator, role of spirituality in health care, and living situation. 
2. Goals and Action Steps: This is a worksheet developed for the PC-based PHR that asks the respondent 
to identify longer term health goals, and small, short-term steps and specific actions to be taken in order to 
pursue those goals.  Action plans, which involve setting and tracking short-term health goals, has been shown 
to be a highly effective strategy for increasing activation and promoting health behavior change.41 
iii.Electronic Prompts and Reminders: Prompts and reminders will be provided 1. At the time that each 
medication dosage is due; 2. Two weeks before each medication refill; and 3. One day before each medical 
and mental health visit. 
iv.Communication with providers: The HealthVault Message Center allows patients to send and receive 
encrypted email to and from healthcare providers including the care coordinator and any other providers who 
opt-into the subjects’ mPHR. All messages are stored in the Microsoft HealthVault record. 
IIIB4c. Access controls: Participants can choose to share data with providers, family members, and friends, 
and will be able to assign an access level to each person listed on the plan to determine the level of access 
each member has to the specific fields. Access may include the ability to fully edit the plan, view only, or to 
have no access to the plan. 
IIIB4d: Role of the care coordinator: PHRs are most effective if they are used in conjunction with providers who 
use them as tools to engage patients in their care. 42, 43 The Four Corners clinic employs two full-time care 
coordinators, bachelor-level care managers who provide education and help coordinate patient care. For the 
study, we will train the care coordinators to use the mPHR as a tool for patients in the mPHR care condition. 
IIIB4e: Training 
i.Training the care coordinator in use of the app and ongoing supervision: After the app is developed each of 
the CMHC care coordinators will receive an interactive, 3-hour training program by the app developer, and by 
Warren, Zegura and Druss in use of the app and the computer literacy training program. Topics will include: 1. 
Use of smartphones by persons with SMI. 2. An overview of the computer literacy curriculum 3. A review of 
each of the each of the components of the app (medications; laboratory values; consumer-derived data on 
health goals; prompts and reminders for medications and upcoming appointments; communicating with 
providers. Monthly meetings with the care coordinators will discuss current experiences and troubleshoot 
problems. 
ii. Training clients in use of the app and ongoing support: The training of clients by care coordinators in use of 
the app will parallel the process of training the care coordinators. The care coordinator will provide a two- 
session, one-on-one overview of use of smartphones and the mPHR. Topics will include: 1) Overview of 
smartphones, including basic features from the training curriculum (turning on; charging; using apps), based on 
the computer literacy curriculum for persons with SMI developed by the study team; 2) An overview of each of 
the components of the app; and 3) Troubleshooting (what do to if an app is not working; what to do if a phone is 
lost or stolen). Initial demonstration will be followed by subjects being asked to show that they can use the key 
smartphone functions and app components. This process will be continued until the subject demonstrates 
proficiency in using the smartphone and each component of the intervention app. Development and refinement 
of the training in Aim 1 will guide the standardization of the training we will use in Aim 2 for the RCT. 
iii.App maintenance and troubleshooting: The app developer will maintain the app and be available by 
telephone and email to participants, care managers, and study staff to troubleshoot any problems that arise. 
IIIB4f. Refine the intervention using semi-structured consumer interviewers: After the beta version of the app is 
developed, one-on-one semi-structured interviews will be conducted with 10 participants from the Four Corners 
Behavioral Health Home. The clinic director will solicit 10 volunteers with SMI who do not have their own 
smartphone. For this formative work in Aim 1, each participant will be shown a smartphone with the app and 
asked questions about the usability and content of each of the four components of the app (lab and medication 
information downloaded from the EHR; personal details and goals entered by the participant; electronic prompts 
and reminders; and secured communication with the provider). Participants will be asked: How easy is it         
for you to use this program? What is hardest for you about using it? What changes would make it more useful 
to you? All interviews will be tape recorded and transcribed. Analysis of these transcripts will include both 
deductive and inductive analysis, and proceed in three phases:44, 45  1) preliminary analysis relying 
predominantly on a set of deductive codes representing the initial objectives of the interviews (e.g. ascertaining 
participants’ experiences in using each of the three components of the phone application.); 2) in-depth analysis 

based on the emergence of key inductive themes, or concepts identified by participants themselves as being 
important; and 3) interpretation and synthesis of key concepts in the data based on a thorough and systematic 
exploration of both inductive and deductive themes. 
After these interviews are completed and analyzed, the app developer, IT experts (Warren, Zegura), 
and PI will review the qualitative data. Based on the findings and recommendations from this steering group, 
the app designer will make further refinements of the intervention prior to the full randomized trial. 
 
IIIB5: Specific Aim 2: Evaluate, using an adaptive, randomized controlled study design, the effects of 
the mPHR on patient and provider indicators in a sample of 5 sequential cohorts of 60 subjects, half of 
whom will receive the mPHR. 
IIIB5a. Recruitment and Randomization: Subjects will be recruited from a roster of clients from the Four 
Corners Behavioral Health Home. Characteristics of the clinic are described in IIIB3. The inclusion criteria for  
participation in the clinic include having a serious mental illness and at least one cardiometabolic risk factor. 
The only exclusion criterion will be inability to consent based on a 6-item, validated screener. 46 These 
relatively inclusive criteria were chosen to optimize generalizability to other behavioral health homes; 
preplanned analyses will assess whether the intervention is effective across putative moderators such as 
baseline health and computer literacy. 
Subjects who meet eligibility criteria and who provide informed consent to participate will be randomly 
assigned in blocks of 60 (30 case and 30 controls), to either receive the mPHR or continue in usual care in the 
clinic. Subjects who do not own an Android-based smartphone will be provided with one loaded with the study 
app for the 6-month duration of the intervention. Those who own an Android-based smartphone will be offered 
the option of having the Android app loaded onto their own smartphone for the duration of the intervention. All 
participants will participate in the baseline assessment and those assigned to the mPHR group will receive the 
two-session, one-on-one app training described above in IIIB4e. Randomization will be stratified by whether 
subjects own and will use their own Android-based smartphone for the study. A computer-generated algorithm 
and concealment of allocation techniques will be used to minimize assignment bias. 
IIIB5b. Adaptive randomized design: The landscape of public sector mental health service delivery is evolving 
in the context of health reform and grassroots development of new models of care in communities.19 
Simultaneously, new health technologies, particularly in the mobile space, are rapidly developing. 47    More 
flexible study designs are need to ensure that study findings are relevant.48 
To address these issues, the study will use an adaptive randomized design. An adaptive design uses 
the cumulative knowledge of current treatment successes and failures to change qualities of the ongoing trial.9  
A total of 5 cohorts of 60 subjects each will be sequentially enrolled in the trial for 6 months each, with half 
randomized to receive the mPHR, after which 2 months will be spent updating and modifying the intervention 
prior to the next cohort, while recruiting the new cohort of subjects. Modifications to the app will be made  
based on 1. Feasibility and acceptability of the app within the current cohort 2. New developments in health 
and PHR apps 3. Changes in usual care both in the clinic and other behavioral health homes. Analyses will 
examine both overall effects of the intervention, and whether there is a change in its benefits over time. 
i. Assessing feasibility and acceptability: At the end the 6-month intervention, participants will return for a follow 
up assessment of feasibility and acceptability, assessment of quality of care and health outcomes. Feasibility 
will be assessed by asking clients to demonstrate their ability to access and use the features of the app. A 1-5 
Likert scale will be used to rate degree of mastery of each of the components of the application, with a score of 
3 in each category representing a “passing grade.” Acceptability will be tested by direct tracking of app use. 
With informed consent, information from the app will be downloaded to the study server automatically on a 
weekly basis; information will include the number of times each component of the app is accessed and the 
particular component that is used. 
ii. Reviewing technological advances in mobile health technology: Using methodology used in other 
systematic app review studies,49 Elaine Warren, M.C.S. will conduct searches every 6 months in the official 
application stores for five major smartphones: iPhone (App Store), Android (Android Market), Blackberry (App 
World), Nokia/Symbian (Ovi) and Windows Mobile (Marketplace), within the health/lifestyle and medical 
categories, for products related to personal health records or health management. Any new products will be 
downloaded to examine new developments or features that could be incorporated into the mPHR. 
iii. Reviewing changes in usual care: Changes in the usual care condition at both the clinic and patient level will 
be reviewed every six months. Deborah Strotz, MSW, is the director of the Four Corners clinic. Each year, she 
will present updates on any changes in staffing or structure of the clinic. At the subject level, questions will 

review whether subjects in each group own smartphones, and whether and how they have used any health 
apps in the past 6 months. 
iv. Reviewing national changes in care delivery in community mental health. Behavioral health homes are a 
relatively recent innovation, and their structure is continuing to evolve. More generally, models of service 
delivery in the public specialty mental health sector are changing at a rapid pace. In his role as Vice President 
of Health Information Technology for the National Council for Community Behavioral Healthcare, Michael 
Lardiere conducts an annual survey of all community behavioral providers on changes in organizational 
structure and use of technology. He also provides consultation to providers including all SAMHSA PBHCI 
grantees on topics including purchasing of EHRs and integration of medical and mental health EHRs in 
behavioral health homes. For each cycle, he will provide an update on new organizational developments in 
community mental health and use of health technology. 
Every 6 months, the study team will meet to review these developments in health technology and in 
mental health systems. At the meeting, the team will develop recommendations for updates to the mPHR app 
which the app developer will implement in the next cohort. This process will seek to balance fidelity the core 
features of the PHR while allowing flexibility to ensure relevance to real-world settings, paralleling strategies 
used in mental health dissemination and implementation studies. 50, 51 Analyses will account for potential 
differences across cohorts due to these modifications (IIIB5c). 
IIIB5b: Outcome Measures for Specific Aim 2: 
i.Overview: The conceptual model predicts that an mPHR can improve quality of medical care by improving 
patient activation and coordination of provider care. Quality of cardiometabolic and preventive services will be 
the main study outcomes, and these will be drawn directly from the West End EHR. Patient activation, provider 
management of chronic illness, and medication adherence will be studied as intermediate (mediating) 
outcomes, and cardiometabolic quality of care and health related quality of life will also be studied as key distal 
outcomes resulting from better quality of care. Data from each of the two EHRs will be used for generating the 
quality indicators. Outcome assessments will be tracked at 6 months, and then an additional six months after 
the intervention is complete to assess whether any benefits persist over time. 
ii.Qualilty of Care Measures: 
1. Quality indicators for the cardiometabolic risk factors will be drawn from the two electronic health records 
using indicators from the Community Quality Index (CQI) study, the largest and most comprehensive study of 
quality of care ever conducted in the United States.52 53, 54  There are 9 indicators for hyperlipidemia, 13 for 
hypertension, and 13 for diabetes; each covering domains of screening, diagnosis, treatment, and follow-up. 55 
A quality score for each condition is generated by dividing all instances in which recommended care is 
delivered by the number of times a participant is eligible for indicator. 
2. Quality of Preventive Medical Services:  Quality indicators, as well as the populations eligible for each 
preventive measure, will be obtained from the U.S. Preventive Services Task Force (USPSTF) guidelines. 56 
For the study, we will include items with strong research evidence (evidence grade “A” or “B” to support their 
use. 
iii.Intermediate Outcomes: 
These outcomes are postulated to be both important measures of the intervention, and pathways through 
which the intervention may improve quality of medical care. 
1. Patient Activation:. The study will assess patient activation, as measured by the patient by Patient Activation 
Measure (PAM).26  This is a 22-item measure of patient knowledge, skills and confidence in self-management 
behaviors that has been shown to have high validity reliability (infit values of 0.71-1.44) and criterion validity 
based on concordance with related constructs including self-management behaviors.26 27 medication 
adherence, and outcomes including quality of life.28 
2. Provider Management of Chronic Illness: The Patient Assessment of Chronic Illness Care (PACIC) is a 20- 
item, patient self-report instrument developed by Glasgow, Wagner et al.that assesses the extent to which 
patients with chronic illness report receiving care that aligns with the Chronic Care Model. 57, 58 59 It has been 
found to have good test-retest reliability (r = 0.58 during the course of 3 months) and to be associated with 
improved health behaviors,58  global patient ratings of their health care, and quality of life. 60 
iv.Clinical Outcomes: 
Better quality of care is a first step to improving cardiometabolic outcomes, and improvement in process 
measures have been shown to predict better clinical outcomes. 61, 62  We therefore hypothesize that the mPHR 
will result in improved cardiometabolic and quality of life outcomes. For the purposes of the study, these will be 
treated as secondary outcomes. 

1. .The Framingham Cardiovascular Risk Index provides the 10-year risk of developing incident coronary heart 
disease. It is calculated based on a weighted score based on age, blood pressure (4 levels), smoking status 
(yes/no), diabetes (yes/no), total cholesterol (3 categories), and HDL cholesterol (3 categories). The index has 
been used successfully to assess cardiovascular risk in populations with severe mental illnesses. 63-66 
Secondary analyses will examine changes in individual cardiometabolic risk. 
2. Health Related Quality of Life: The SF-36 is a measure of Health Related Quality of Life constructed for use 
in the Medical Outcomes Study. 67, 68 A summary Physical Component Summary (PCS) and Mental 
Component Summary (MCS) scores can be constructed from the survey, scored between 0 (poor health) to 
100 (perfect health). 69    For patients with severe mental disorders, these measures have been found to 
demonstrate good internal consistency, test-retest reliability, and concurrent and discriminative validity.70, 71 
v. Potential Confounding/Moderating Variables: It is essential to assess balance across key variables that 
could confound results. It is also important to examine key moderators such as baseline health and computer 
literacy; groups with higher levels of literacy at baseline might be expected to have better study outcomes. 
1. Sociodemographic characteristics: Demographic factors such as age, ethnicity, health insurance status, and 
income will be assessed at baseline, and included in all multivariate models. 
2. Health Literacy and computer literacy may be moderators of the intervention effect –that is, individuals with 
higher levels of literacy may be more able to fully engage with the mPHR and derive maximal benefit from the 
intervention. Health literacy will be assessed using the STOFHLA, a brief instrument that ranks respondants’ 
reading and numeracy health literacy on a scale of 1-100.72 Computer literacy will be assessed using a 27-item 
measure developed by Hargitti et al.73 Each of these surveys has been conducted in nationally representative 
samples, allowing for benchmarking findings derived from the current study. 
IIIB5c:Data Analytic Strategy for Specific Aim 2: 
i.Overall data analytic strategy: All analyses will be conducted as intent-to-treat. The primary analytic 
technique for assessing differences between the study groups will be random regression.74  This method 
makes it possible to compare the difference in change between groups over time, and to conduct intention-to- 
treat analyses that include subjects with missing data at one or more follow-up periods. These will be 
conducted using the SAS MIXED procedure for continuous variables and PROC NLMIXED, which is the 
preferred procedure for binary and ordinal variables when there are relatively few observations per subject.75 
Each equation will model the process or outcome measure of interest as a function of 1) randomization group 
2) time since randomization 3) group*time interaction. The group*time interaction will be the primary 
parameter estimate of interest, since it represents the difference in change between the two randomization 
arms over time. All models will also include any parameters that vary at baseline at the p=0.05 level. 
ii. Analytic issues for the adaptive design: Because the intervention will be updated every 6 months, results will 
be initially be examined stratified by each of the cohorts. In the final model, cohort will be included as a fixed 
effect, and group*time*cohort will also be studied to assess whether the effects of the intervention change as it 
is updated. Because the core features of the intervention will not change over time, we hypothesize that these 
differences will not be statistically significant. 
iii. Sample Size Calculation: The study sample size was calculated to produce a statistical power of 0.8 for 
detecting a significant change in RAND composite CQI measure for cardiometabolic quality of care. The 
sample size calculations for random regression models for the primary study outcomes follow the method 
outlined by Diggle et al.76  We are conservatively estimating 10-20% attrition. Using this approach, we estimate 
that an initial sample size of 300 will provide adequate power to detect a 10% difference in the composite 
quality index with an intraclass correlation of 0.3, which would be considered a clinically significant 
improvement.52 
iv: Mediation analysis for intermediate outcomes: In addition to examining patient activation and provider 
chronic care as intermediate endpoints in the study, we will also conduct analyses to see whether they partly 
mediate the association between the intervention group and quality of care. Mediation analyses will use 
adaptations of structural mean models developed for use in mediation analysis in complex randomized trials 
developed by Ten Have. 77, 78 Using the M Plus package, 79  separate models will be constructed to examine the 
relationship between the independent variable (study arm), the outcome (quality of care) and each mediating 
variable. Models will determine whether 1. Intervention status is associated with improved quality of care          
at 12 months. 2. Intervention status is associated with the patient activation and provider effectiveness in 
managing chronic illness at 6 months. 3. The 6-month measure of patient activation or provider effectiveness in 
managing chronic illness is associated with 12-month quality score and 4. Adding the mediator to a multivariate 
model significantly reduces the magnitude of the association between intervention status and 12-month quality 
score, using the Sobel test. 80 will be used to test the statistical significance of the separate impact of each 

mediator on the direct effect of the baseline intervention by comparing adjusted and unadjusted effects of the 
baseline intervention relative to their standard errors. 
v. Moderator analysis for health and computer literacy: Moderator analyses make it possible to identify what 
types of clients benefit most from a given intervention. The approach will follow Kenny et al’s method for 
analyzing moderator effects in randomized trials using random regression. 81 For each analyses assessing 
heath and computer literacy as potential moderators, we will add a group*time*moderator interaction term, the 
statistical significance of which will allow us to assess the presence of moderation. All appropriate lower-order 
interactions will also be included in the model. 
IIIB6: Specific Aim 3: Evaluate the impact of the mPHR on service use and costs of care. Handheld 
devices can help improve adherence to medically-indicated treatments by providing prompts and reminders; 82, 
83 better adherence may improve efficiency and quality of care. We hypothesize that the prompts and reminders 
for the mPHR will help improve efficiency of service use, including improving adherence to medications         
and appointments, and that better adherence to indicated treatments will reduce unnecessary ED visits         
and improve quality of preventive services. Coupled with the relatively low cost of deploying the app once it      
is developed, we anticipate a favorable cost profile for the intervention from both a societal and managerial 
perspective. 
IIIB6a. Outcome measures for Specific Aim 3. 
i. Medical and mental health service use: Health service use will be assessed via chart review for all services 
provided by each of the two partner organizations, supplemented by a validated self-report instrument 
developed for the CATIE trial 84 to capture any service use outside of the two organizations. The following 
categories of service use will be calculated: 1. Primary care medical visits 2. Specialty medical visits 3. Medical 
Emergency Department (ED) visits 4. Medical Inpatient days 5. Mental Health outpatient visits 6. Mental health 
ED visits 7. Mental health inpatient days. 
ii. Missed appointments: A separate field is available in each of the EHRs for missed appointments. This will be 
modeled as a two-part model (any missed visit, number of missed visits among those with a missed visit). 
iii. Medication Adherence: Adherence will be assessed using chart records, by assessing a medication 
possession ratio (MPR), defined as the number of days of treatment dispensed divided by the number of days 
between prescription refills (excluding the last prescription) 85, 86 for those patients with each of the target study 
conditions (diabetes, hypertension, and hyperlipidemia). Separate indicators will be calculated for each class of 
medications used to treat diabetes, hyperlipidemia, and hypertension, with an average value across the three 
cardiometabolic conditions used as an aggregate measure of adherence. 
iv. mPHR use: The study will capture mPHR use 1. To assess feasibility of the intervention by study 
participants; 2. To assess fidelity to the study intervention and 3. To conduct exploratory mediation analyses to 
assess whether greater app use is associated with better study outcomes. Each instance of mPHR use will be 
logged along with time and date information indicating the specific fields of the mPHR used, and amount of 
time spent with the PHR. Event auditing will categorize whether 1. Users view fields without changing data or 2, 
Users change fields in the mPHR. 
IIIB6b: Data Analytic Strategy for Specific Aim 3: 
After conducting bivariate analyses, random regression will be used to assess service use outcomes at 
follow up, using the group*time interactions as the primary variable of interest. See IIIB5c for more detail on the 
analytic models. The study team has extensive experience in conducting these service use analyses.87 
i. Service Use analyses: Separate models will be developed for each service use category as a dependent 
variable. These analyses will focus on the intervention’s impact on medication adherence, no shows, and ED 
use. Additionally, a series of mediation analyses will test the conceptual model postulating that changes in 
service use will moderate clinical and cost outcomes. In particular, we will test whether improved medication 
adherence , reduced ED service use, and reduction in missed appointments predicts improved quality of 
cardiometabolic care, increased use of preventive services, and reduced costs. A second set of mediation 
analyses will examine whether there is a dose-response association between use of the mPHR and quality of 
care. These will be exploratory, recognizing that regular mPHR users may be more engaged in their care at 
baseline than other enrollees. Mediation analysis will follow the methods described in IIIB5c. 
ii. Costs from the Health System Perspective: Unit costs for all services will be assigned to each service type 
based on median national expenditures for each type of service from the Medical Expenditure Panel Survey.88 
This survey is well suited for assessing unit costs from a health system perspective because it uses direct 
payments, not charges, and captures costs across all insurance groups. 
Costs of the intervention will be estimated using micro-costing approaches that calculate direct 
measures of resources consumed in establishing and maintaining it.89 90.91 Technology costs will include costs 

of the smartphone and monthly data plan, including total data use associated with the mPHR, which could be 
borne by the patient or a provider organizaiton. Care coordinators will keep logs of all mPHR visits to use for 
assessing the portion of their total hours per week spent on working with the mobile PHR in intervention 
clients. Analysis methods: Cost analyses will follow standard approaches from the cost-effectiveness 
literature. 92-94    For the health system perspective, all expenditures will be counted as positive costs. Analyses 
will compare change across total costs between the mPHR and control groups, using the group*time variable 
as the parameter of interest. 
iii. Costs from the Managerial Perspective (CMHC, FQHC): Conditions under which a program can be 
financially viable is among the most pressing and salient questions underlying managers’ decisions to adopt 
and maintain those programs.95  The methods proposed for the budget impact analysis in the current 
application follow the approach put forth by the ISPOR Task Force on Good Research Practices.96 In contrast 
to the health system perspective, reimbursements will be treated as negative costs. 
Billing rates will be obtained for Medicaid, Medicare, private insurance, and for uninsured clients (state 
or federal block grant rates) for each type of visit. Returns will be assessed varying key parameters that could 
exist in implementing the program under real-world conditions: 1. Insurance case mix (Medicaid versus 
uninsured); 2. Provider type working with the mPHR (bachelor’s level, MSW, RN); and 3. Patient flow (number 
of patients using the mPHR). Financial sustainability, defined as conditions in which reimbursement payments 
break even, or exceed intervention costs,97 will be assessed at 6 and 12 months after implementing the 
intervention, examining whether, or at what point, potential reimbursement for services would exceed staff 
costs for administering the intervention. 
 
Table: Summary of Study Measures 
Baseline 
6 months 
12 months 
Source 
Quality Measures 
Medical quality measures (RAND) 
X 
X 
X 
Chart Review (EHRs) 
Quality of preventive services (USPTF) 
X 
X 
X 
Chart Review (EHRs) 
Clinical Outcome Measures 
Framingham cardiovascular risk score 
X 
X 
X 
Chart Review (EHRs) 
Individual measures of cardiometabolic risk 
X 
X 
X 
Chart Review (EHRs) 
Physical and Mental Health Related Quality of Life 
X 
X 
X 
Interview 
Intermediate Outcome /Mediator Variables 
Patient activation (PAM) 
X 
X 
X 
Interview 
Provider management of chronic illness (PACIC) 
X 
X 
X 
Interview 
Potential Confounders/Moderator Variable 
Sociodemographics 
X 
X 
X 
Chart review (EHRs) 
Health and computer literacy 
X 
X 
X 
Interview 
Service Use and Cost 
Use of medical and MH services 
X 
X 
X 
Chart review (EHRs), interview 
Missed appointments 
X 
X 
X 
Chart Review (EHRs) 
Medication adherence (MPR) 
X 
X 
X 
Chart review (EHRs) 
Use of PHR 
X 
X 
X 
Chart review (mPHR) 
Costs/payments 
X 
X 
X 
Chart review 
 
IV. DISSEMNATION PLAN 
Passive diffusion of findings through the research literature is a slow and unreliable means of achieving 
uptake of scientific innovations.98 To optimize dissemination of the app, we will include a formal dissemination 
plan for the app in partnership with the National Council for Community Behavioral Healthcare, led by Michael 
Lardiere. The app will be disseminated to behavioral health homes funded through SAMHSA’s PBHCI program 
(96 sites anticipated by the end of FY 2012) through the Center for Integrated Health Solutions, funded by 
SAMHSA and HRSA through the National Council for Community Behavioral Healthcare. That site offers a 
range of technical assistance to PBHCI grantees, including regional meetings, webinars, and online tools. In his 
role as a consultant to that Technical Center, Dr. Druss regularly conducts webinars and provides technical 
assistance to sites in developing and evaluating their programs. The mPHR app will be made freely available 
and Dr. Druss and Michael Lardiere will present at the annual PBHCI grantee meeting explaining how to use it 
in behavioral health homes. Beyond its role for PBHCI grantees, the National Council for Community Behavioral 
Healthcare also provides a broad range technical assistance to its 1,900 member organizations. To expand   
the reach of the mPHR to this broader national platform, Dr. Druss and Michael Lardiere will present annual 
workshops at the National Council for Community Behavioral Healthcare’s annual meeting, and will present      
a webinar to reach community providers who are unable to attend that meeting. 

V. ADDRESSING POTENTIAL LIMITATIONS 
We believe the proposed study has a number of strengths, including addressing an issue of high public 
health importance to a vulnerable population, the use of a novel health information technology innovation, an 
innovative study design, high potential for generalizability, and rich opportunities for dissemination. 
Nonetheless, we also recognize several potential issues. 
A.Challenges in using technology inter entions in public sector populations: We recognize that rates of 
smartphone use in persons in poor populations is likely to be lower than usage in the general population. 
However, extrapolating from the history of cell phone diffusion, competition is likely to drive prices down and 
encourage widespread use in coming years, with particularly extensive uptake in poor and minority 
communities. 99-101 Thus while we will provide smartphones to the study participants, we anticipate that by the 
time the study is completed, most persons treated in the public mental health sectors will own their own 
smartphones, adding to ease of dissemination. 
B.Challenges specific to persons with SMI: We recognize based on our existing work that about a third of 
persons with SMI may have gaps in health and computer literacy and cognitive symptoms associated with 
poverty, lower educational attainment, and underlying symptomatology. We are addressing these challenges 
by 1.ensuring that it is designed to fit the cognitive and literacy needs of the population, 2. using a mobile 
platform to ensure internet connectivity, and 3. providing all participants in the intervention group with a 
modified computer literacy training program developed as part of the pilot work. While all members of the 
behavioral health home will be eligible to enroll in the study, analyses will examine whether persons with lower 
rates of health or computer literacy at baseline derive the same benefit as higher literacy participants. 
C.Challenges in ensuring privacy: Several steps will be taken to ensure that these data are protected and 
deleted if a pho e is lost or stolen. All smartphones will be password protected, with only the client and study 
administrator informed about the password. All patient data will be stored on the secured HealthVault server, 
none will be stored on the smartphone. If a phone is lost or stolen, Emory’s IT Department will remotely disabl 
the smartphone, and wipe all data including the study app. 
C. Challenges in addressing potential contamination: Because the program will be conducted in a single 
behavioral health home, there is the potential for contamination if providers’ practices are changed by use of 
the mPHR. Such contamin tion would end to reduce the effect size of the intervention, increasing the 
possibility of type II error.102 To investigate this possibility, we will examine pre-post changes in process of care 
measures for the usual care group to assess for potential effects on control subjects. 
 
VI. TIMELINE: 
During the first 9 months, we will we will develop a beta version of the app and refine it using qualitative 
techniques. Next, 5 cohorts of 60 subjects each will be sequentially enrolled in the trial for 6 months each; 
after each cohort, 2 months will be spent updating and modifying the intervention prior to the next cohort. The 
intervention will be conducted from year 2 through the first part of year 5. The cohorts will be completed by the 
final 6 months of year 4, with 12-month follow-ups completed by the 6 months mark at year 5. Data analyses 
and dissemination will occur during the final 6 months of the study. 
 
Study Timeline 
 

VIII. HUMAN SUBJECTS 
The study proposes to develop and test a smartphone-based, mobile personal health record (mPHR) for 
persons with serious mental disorders treated in behavioral health homes.  In the first phase of the study, the 
study team will develop and refine the app based on qualitative interviews. In the second phase, 5 cohorts of 
60 subjects will be randomized to receive either the mPHR or usual care. During the final phase, the program 
will be disseminated through a national technical assistance center to behavioral health homes nationwide. 
 
VIIA: Protection of Human Subjects 
VIIA1: Sources of research material: Sources of research material will include interviews with subjects and 
administrative data from the Four Corners Behavioral Health Home in Cobb County, GA. Interviews will be 
audiotaped and transcribed. Audiotapes will be destroyed once interviews have been transcribed. 
 
VIIA2: Plans for Recruitment of Subjects: 
Subjects will be recruited from a roster of clients from the Four Corners Behavioral Health Home. All of these 
patients have a diagnosis of a serious mental illness and one or more cardiometabolic risk factor. The only 
exclusion criterion will be inability to consent based on a 6-item, validated screener. 46 
 
VIIB: Informed Consent:  
VIIB1. Qualitative interviews: All eligible participants in interviews will be offered written informed consent. The 
informed consent process will describe the risks and benefits of participating in the interviews, focusing on the 
potential for breach of confidentiality. 
 
VIIB2. Randomized Trial: All participants will be offered written informed consent. The procedures of the study 
and the informed consent document will be thoroughly reviewed with the subject in both verbal and written form 
by a trained research assistant. Only those subjects demonstrating adequate comprehension will be consented 
and enrolled in the study. Study personnel will also obtain permission to recontact the subjects and ask      
them to complete follow-up assessments. 
The informed consent process will include the following: notification that participants will be interviewed 
three times (baseline, six-month, and 1-year follow-up); that half of participants (selected at random) will 
complete and maintain a smartphone-based, mobile personal health record; that agreement to participate in the 
study does not obligate participants to accept any particular treatments; and that participants are free to 
withdraw from any part of the study (either the intervention program or the follow-up assessments) at any time. 
 
VIIC: Potential Risks 
VIIC1. Breach of confidentiality: It is essential to monitor and minimize the possibility of a breach in 
confidentiality either via the mobile personal health record, care coordinator, research staff (interviewer), or 
through inappropriate access to the smartphone-based mobile PHR. These issues are particularly salient in the 
use of information technology, whose goal is to consolidate and improve access to health information. Section 
VIID1 describes the mechanisms for addressing these potential privacy concerns. 
 
VIIC2: Acute medical or mental health crisis: It is possible that patients will be found to be in a medical or 
mental health crisis at the time of screening or that a new problem will emerge sometime during the study. 
Section VIID2 describes mechanisms for safely addressing these potential problems. 
 
VIIC3.Subject Burden: Study assessments will always result in an expenditure of time and inconvenience for 
subjects, as well as the potential for discomfort or embarrassment related to the content of the interview. 
 
VIID Adequacy of Protection against Risks:  
VIID1. Protecting against breach of confidentiality: 
VIID1a: Privacy Protections for the mPHR:  
1. Patient’s role in selecting who has access to health information: Patients assign an Access Level to each 
person listed on the plan to determine the level of access each member has to the specific health record. 
Access may include the ability to fully edit the plan, view only, or to have no access to the plan. 
2. Password features: Upon registration, all mPHR users (patients, providers, and other care team members) 
obtain a unique identification and an encrypted password. Passwords are a minimum of eight characters and 
are required to use mixed case letters, numbers, and punctuation. 

3. Data storage: Data will be stored on HealthVault servers, which are password protected and HIPAA 
compliant. No patient data will be stored locally on the smartphones. All data transfers to and from the server 
use 128 mb encryption. If a smartphone is lost or stolen, Emory IT will remotely wipe all apps and data off of 
the smartphone. 
4. Auditing Capabilities: Full-content auditing makes it possible to give users as much access to the 
application as needed to easily perform their tasks, but with the understanding that a full report of 
their activities is always available. Participants will be encouraged to review their own audit 
trails to see who has accessed their mPHR. 
There are two different types of auditing that occur within the mPHR. Event auditing occurs 
when information is logged when users take certain actions (but do not change data). Data 
auditing occurs when the results of data operations (inserts, updates, and deletes) are logged. 
Each table in the mPHR schema that holds dynamic user-generated content has an associated audit table. 
Separate logs are kept for logins, searching for a mPHR, viewing the mPHR, viewing particular screens, and 
viewing a user’s registration information. 
The project manager will print weekly logs of auditing activities for all patients in the study. If there is 
any evidence of a breach (e.g. inappropriate attempts to view or change the record by unauthorized 
personnel), the principal investigator and data safety monitoring board will be notified, and appropriate action 
will be taken. 
 
VIID1b: Protecting privacy for the study and study data:  
1. For the qualitative interviews, no identifying information will be included in the transcripts of the interviews or 
in any publications or reports that emerge from those reports. 
2. For the randomized trial, all interviews will be held in a private area away from the clinical care area. 
3. For study data, unique identifiers will be generated for all study subjects; these will be the only identifiers 
used in files used for data entry and analysis. The file linking these identifiers to patient names and addresses 
will be stored in a separate locked file that will be available only to the principal investigator and project 
manager. All research files will be stored in password-protected computer files or locked file cabinets. 
Identifying information such as name, address, and telephone number will be available only to those study 
personnel responsible for contacting study participants. 
 
VIID2: Protocol for addressing acute medical or mental health crisis: 
If patients will have a medical or mental health crisis either at the time of an interview by the research 
assistant, the research assistant will immediately contact the principal investigator. Based on the clinical 
situation, acuity, and patient preferences, a decision will be made whether to a. Contact the patient’s primary 
care provider or mental health clinician or b. Escort the patient to the Emergency Room for a full evaluation. 
 
VIID3. Minimizing subject burden: To minimize the interview burden for the randomized trial, interviews will be 
scheduled at a time and location that is as convenient as possible for patients –when possible, prior to or 
directly after a Four Corners Clinic appointment. Interviews will be conducted in a private, comfortable setting. 
If a subject appears tired or anxious, the subject will be asked if he or she wishes to rest, take a break, or 
terminate the interview. Monetary reimbursement for each interview and focus group meeting will help 
compensate subjects for the time and effort expended. Monetary reimbursement for attendance at each 
training session will help defray any needed transportation or childcare expenses. 
 
VIIE Potential Benefits of the Proposed Research Project to the Subjects and Others: 
VIIE1. For subjects in the usual care group of the trial: For those in the usual care arm of the randomized trial, 
patients will not receive any direct health benefits from participation. 
 
VIIE2. For subjects in the intervention group: Subjects participating in the intervention group are expected to 
directly benefit from the mPHR. As described in the body of the proposal, hypothesized benefits include 
improved quality of care and better clinical outcomes. 
 
VIIE3. For other persons treated in public safety settings: The study’s results are hoped to inform the 
development of other programs to improve care for clients with serious mental Illness. 

VIIE4 Risk Benefit Ratio: Given the protections to ensure confidentiality, ensure high quality of the intervention, 
and minimize subject burden, and the potential benefits of improved care for patients in the active treatment 
group of the randomized trial, we anticipate a favorable risk-benefit ratio for the projects. 
 
VIIF. Data Safety and Monitoring Plan 
VIIF1. Responsibility, Frequency, and Content of Data Safety Monitoring: The PI will be involved in all aspects 
of the research and will be responsible for monitoring all data. The PI will review safety data on a monthly basis 
during the course of the study. A three-member DSMB committee will be assembled who will review study 
recruitment and any potential adverse events twice each year. A semi-annual report by the PI to the Emory IRB 
will include a brief description of the randomized trial; baseline demographic characteristics, and retention and 
disposition of study participants. 
 
VIIF2: Adverse Event Reporting: Serious adverse events are expected to be unlikely. If one occurs, the PI will 
immediately report the event to the Emory IRB. The PI, in consultation with co-investigators and others as 
needed, will review the adverse event report and determine the need for subsequent action. Any subsequent 
action will also be documented and reported to the Emory IRB. 
 
VIIF3: Confidentiality and Integrity of Data:  
VIIF3a. Qualitative interviews: No identifying information will be included in the transcripts of interviews or in 
any publications or reports that emerge from those reports. All audiotapes will be destroyed after transcripts 
have been made, and the transcripts will be kept in a locked file. 
VIIF3b. Randomized trial: PHR data will be stored on HIPAA compliant HealthVault servers; no patient data will 
be stored on study smartphones. Unique identifiers will be generated for all study subjects; these will be the 
only identifiers used in files used for data entry and analysis. The file linking these identifiers to patient names 
and addresses will be stored in a separate locked file that will be available only to the principal investigator and 
project manager. All research files will be stored in password-protected computer files or locked file cabinets. 
Identifying information such as name, address, and telephone number will be available only to those study 
personnel responsible for contacting study participants. 
VIIF3c HIPAA: The Emory University Institutional Review Board has developed a series of guidelines and 
amendments to their review process to ensure that all research protocols are compliant with the privacy 
provisions of the Health Insurance Portability and Accountability Act. We will work closely with that office to 
ensure that any final IRB protocol is in full compliance with this act. 

VIIG1. Inclusion of Women Women will be included in the enrollment process and we expect them to be 
well-represented based on the characteristics of the CMHCs from which they are recruited. 
 
VIIG2. Inclusion of Minorities Minority groups are well-represented in the mental health centers, and will be 
included based on their representation in that population. 

 
TARGETED/PLANNED ENROLLMENT: Number of Subjects* 
Ethnic Category 
Sex/Gender 
Females 
Males 
Total 
Hispanic or Latino 
8 
4 
12 
Not Hispanic or Latino 
187 
101 
288 
Ethnic Category Total of all Subjects 
195 
105 
300 
Racial Categories 
American Indian/Alaska Native 
0 
0 
0 
Asian 
4 
2 
6 
Native Hawaiian or Other Pacific Islander 
0 
0 
0 
Black or African American 
43 
23 
66 
Hispanic or Latino 
8 
4 
12 
White 
140 
76 
216 
Other 
0 
0 
0 
Racial Categories: Total of All Subjects 
195 
105 
300 

VIIG3. Inclusion of Children 
Because children under age 18 have different health problems and needs from adults, they would not 
easily be integrated into the broader intervention, and hence will not be included in intervention. Approximately 
5% of patients in the four corners behavioral health home is between the ages of 18 and 20, and thus are 
considered children under NIH guidelines. There is no a priori reason to assume that they would derive less 
benefit from a mobile PHR than adults over age 21. Therefore, individuals age 18-20 will be included in the 
study, and there is no reason to expect that special accommodations will be needed for the group. 

REFERENCES 
 
1. 
Mitchell AJ, Malone D, Doebbeling CC. Quality of medical care for people with and without comorbid 
mental illness and substance misuse: systematic review of comparative studies. Br J Psychiatry. 
2009;194(6):491-9. 
2. 
Druss BG, Bradford WD, Rosenheck RA, Radford MJ, Krumholz HM. Quality of medical care and 
excess mortality in older patients with mental disorders. Arch Gen Psychiatry. 2001;58(6):565-72. 
3. 
Alakeson V, Frank RG, Katz RE. Specialty care medical homes for people with severe, persistent 
mental disorders. Health Aff (Millwood). 2010;29(5):867-73. 
4. 
Alexander L, Druss B. Behavioral health homes for persons with mental and substance use conditions. 
Washington, DC; 2012 
5. 
Scharf D, Vaughan C, Eberhart N, Schmidt N, Dutta T, Burnham A. Inegrating primary care services in 
to behavioral health settings for persons with serious mental illness: first lessons learned from SAMHSA's 
Primary and Behaviroal Health Care Integration (PBHCI) grants program. PBHCI annual grantee presentation; 
May; San Diego, CA2011. 
6. 
Carter EL, Nunlee-Bland G, Callender C. A patient-centric, provider-assisted diabetes telehealth self- 
management intervention for urban minorities. Perspectives in health information management / AHIMA, 
American Health Information Management Association. 2011;8:1b. PMCID: 3035826. 
7. 
Lee J, editor. Smart health: Concepts and status of ubiquitous health with smartphone ICT Convergence 
(ICTC); 2011 Sept 28-30, 2011. 
8. 
Chong C-C, Watanabe F. From Mobile Phones to Personal Wellness Dashboards. Pulse, IEEE. 2010:57- 
63. 
9. 
Brown CH, Ten Have TR, Jo B, Dagne G, Wyman PA, Muthen B, Gibbons RD. Adaptive designs for 
randomized trials in public health. Annu Rev Public Health. 2009;30:1-25. PMCID: 2778326. 
10. 
Maltzberg B. Mortality Among Patients with Mental Disease. Utica: New York State Hospital Press; 
1934. 
11. 
Osborn DP, Levy G, Nazareth I, Petersen I, Islam A, King MB. Relative risk of cardiovascular and 
cancer mortality in people with severe mental illness from the United Kingdom's general practice research 
database. Arch Gen Psychiatry. 2007;64(2):242-9. 
12. 
Felker B YJ, Short D. Mortality and medical comorbidity among psychiatric patients: a review. 
Psychiatr Serv. 1996;47(12):1356-63. 
13. 
Saha S, Chant D, McGrath J. A systematic review of mortality in schizophrenia: is the differential 
mortality gap worsening over time? Arch Gen Psychiatry. 2007;64(10):1123-31. 
14. 
Hoang U, Stewart R, Goldacre MJ. Mortality after hospital discharge for people with schizophrenia or 
bipolar disorder: retrospective study of linked English hospital episode statistics, 1999-2006. BMJ. 
2011;343:d5422. PMCID: 3172324. 
15. 
Druss BG, Bornemann TH. Improving health and health care for persons with serious mental illness: the 
window for US federal policy change. JAMA : the journal of the American Medical Association. 
2010;303(19):1972-3. 
16. 
Manderscheid RW. Premature death among state mental health agency consumers: assessing progress in 
addressing a quiet tragedy. Int J Public Health. 2009;54 Suppl 1:7-8. 
17. 
Druss BG, Newcomer JW. Challenges and solutions to integrating mental and physical health care. J 
Clin Psychiatry. 2007;68(4):e09. 
18. 
Kilbourne AM, Morden NE, Austin K, Ilgen M, McCarthy JF, Dalack G, Blow FC. Excess heart- 
disease-related mortality in a national study of patients with mental disorders: identifying modifiable risk 
factors. Gen Hosp Psychiatry. 2009;31(6):555-63. 
19. 
Druss BG, Mauer BJ. Health care reform and care at the behavioral health--primary care interface. 
Psychiatr Serv. 2010;61(11):1087-92. 
20. 
Missouri Department of Mental Health. Health Care Homes. 2012 [updated 2012; cited]; Available 
from: http://dmh.mo.gov/about/chiefclinicalofficer/healthcarehome.htm. 
21. 
Tang PC, Lansky D. The missing link: bridging the patient-provider health information gap. Health Aff 
(Millwood). 2005;24(5):1290-5. 

22. 
Markle Foundation. Connecting for Health. The personal health working group final report. : Markle 
Foundation; 2003 July 1st, 2003 
23. 
Tang PC, Newcomb C. Informing patients: a guide for providing patient health information. J Am Med 
Inform Assoc. 1998;5(6):563-70. 
24. 
Jerden L, Weinehall L. Does a patient-held health record give rise to lifestyle changes? A study in 
clinical practice. Family practice. 2004;21(6):651-3. 
25. 
Crawford S. The New Digital Divide. New York Times. 2011 December 3, 2011. 
26. 
Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure 
(PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39(4 Pt 
1):1005-26. 
27. 
Hibbard JH, Mahoney ER, Stock R, Tusler M. Do increases in patient activation result in improved self- 
management behaviors? Health Serv Res. 2007;42(4):1443-63. PMCID: 1955271. 
28. 
Mosen DM, Schmittdiel J, Hibbard J, Sobel D, Remmers C, Bellows J. Is patient activation associated 
with outcomes of care for adults with chronic conditions? The Journal of ambulatory care management. 
2007;30(1):21-9. 
29. 
Ben-Zeev D, Davis KE, Kaiser S, Krzsos I, Drake RE. Mobile Technologies Among People with 
Serious Mental Illness: Opportunities for Future Services. Adm Policy Ment Health. 2012. 
30. 
Morgan D. Planning Focus Groups. In: Morgan DaK, RA., editor. The Focus Group Kit. Thousand 
Oaks, CA: Sage Publications; 1998. 
31. 
Berkman LF, Blumenthal J, Burg M, Carney RM, Catellier D, Cowan MJ, Czajkowski SM, DeBusk R, 
Hosking J, Jaffe A, Kaufmann PG, Mitchell P, Norman J, Powell LH, Raczynski JM, Schneiderman N. Effects 
of treating depression and low perceived social support on clinical events after myocardial infarction: the 
Enhancing Recovery in Coronary Heart Disease Patients (ENRICHD) Randomized Trial. JAMA : the journal of 
the American Medical Association. 2003;289(23):3106-16. 
32. 
Brown J. The use of focus groups in clinical research. In: Miller BCaW, editor. Doing Qualitative 
Research. Thousand Oaks, CA.: Sage. p. 109-24. 
33. 
Krueger R. Analyzing and Reporting Focus Group Results. Thousand Oaks, CA: Sage Publications.; 
1997. 
34. 
Morgan DaK, RA. The Focus Group Kit. London: Sage; 1998. 
35. 
Lardiere M. A National Survey of Health Information Technology (HIT) Adoption in Federally 
Qualified Health Centers. Bethesda, MD: National Association of Community Health Centers; 2009 
36. 
Android Developers Software Kit,. [cited]; Available from:  
http://developer.android.com/sdk/index.html. 
37. 
;  [cited]; Available from: http://developer.healthvault.com/. 
38. 
Hawley KL, Roberto CA, Bragg MA, Liu PJ, Schwartz MB, Brownell KD. The science on front-of- 
package food labels. Public Health Nutr. 2012:1-10. 
39. 
Kim J, Bates DW. Analysis of the definition and utility of personal health records using Q methodology. 
Journal of medical Internet research. 2011;13(4):e105. PMCID: 3278091. 
40. 
Reti SR, Feldman HJ, Ross SE, Safran C. Improving personal health records for patient-centered care. J 
Am Med Inform Assoc. 2010;17(2):192-5. PMCID: 3000780. 
41. 
Lorig K. Action planning: a call to action. Journal of the American Board of Family Medicine : JABFM. 
2006;19(3):324-5. 
42. 
Tobacman JK, Kissinger P, Wells M, Prokuski J, Hoyer M, McPherson P, Wheeler J, Kron-Chalupa J, 
Parsons C, Weller P, Zimmerman B. Implementation of personal health records by case managers in a VAMC 
general medicine clinic. Patient Education and Counseling. 2004;54(1):27-33. 
43. 
Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A Research Agenda for Personal Health 
Records (PHRs). J Am Med Inform Assn. 2008;15(6):729-36. 
44. 
Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107-15. 
45. 
Richards L. Handling Qualitative Data: A Practical Guide. Thousand Oaks, CA: Sage 
Publications; 2009. 
46. 
Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item screener to identify cognitive 
impairment among potential subjects for clinical research. Med Care. 2002;40(9):771-81. 

47. 
Ehrenreich B, Righter B, Rocke DA, Dixon L, Himelhoch S. Are mobile phones and handheld 
computers being used to enhance delivery of psychiatric treatment? A systematic review. J Nerv Ment Dis. 
2011;199(11):886-91. 
48. 
Luce BR, Kramer JM, Goodman SN, Connor JT, Tunis S, Whicher D, Schwartz JS. Rethinking 
randomized clinical trials for comparative effectiveness research: the need for transformational change. Ann 
Intern Med. 2009;151(3):206-9. 
49. 
Rosser BA, Eccleston C. Smartphone applications for pain management. Journal of telemedicine and 
telecare. 2011;17(6):308-12. 
50. 
Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based 
interventions in health care: application of the replicating effective programs framework. Implement Sci. 
2007;2:42. PMCID: 2248206. 
51. 
Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in 
mental health services: an emerging science with conceptual, methodological, and training challenges. Adm 
Policy Ment Health. 2009;36(1):24-34. 
52. 
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA. The quality of health 
care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635-45. 
53. 
Brook R. The RAND/UCLA appropriateness method. In: McCormick K, Moore S, Siegel R, editors. 
Clinical practice guideline development: methodology perspectives. Rockville, Md, November: Agency for 
Health Care Policy and Research; 1994. p. 59-70. 
54. 
Shekelle PG, Kahan JP, Bernstein SJ, Leape LL, Kamberg CJ, Park RE. The reproducibility of a method 
to identify the overuse and underuse of medical procedures. N Engl J Med. 1998;338(26):1888-95. 
55. 
Asch S, Clark K. Hypertension. Santa Monica, CA: RAND; 2000 
56. 
Thorpe JM, Kalinowski CT, Patterson ME, Sleath BL. Psychological distress as a barrier to preventive 
care in community-dwelling elderly in the United States. Med Care. 2006;44(2):187-91. 
57. 
Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation 
of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005;43(5):436-44. 
58. 
Glasgow RE, Whitesides H, Nelson CC, King DK. Use of the Patient Assessment of Chronic Illness 
Care (PACIC) with diabetic patients: relationship to patient characteristics, receipt of care, and self- 
management. Diabetes care. 2005;28(11):2655-61. 
59. 
Tsai AC, Morton SC, Mangione CM, Keeler EB. A meta-analysis of interventions to improve care for 
chronic illnesses. Am J Manag Care. 2005;11(8):478-88. 
60. 
Schmittdiel J, Mosen DM, Glasgow RE, Hibbard J, Remmers C, Bellows J. Patient Assessment of 
Chronic Illness Care (PACIC) and improved patient-centered outcomes for chronic conditions. J Gen Intern 
Med. 2008;23(1):77-80. 
61. 
Donabedian A. The quality of medical care. Science. 1978;200(4344):856-64. 
62. 
Rossi MC, Lucisano G, Comaschi M, Coscelli C, Cucinotta D, Di Blasi P, Bader G, Pellegrini F, 
Valentini U, Vespasiani G, Nicolucci A. Quality of diabetes care predicts the development of cardiovascular 
events: results of the AMD-QUASAR study. Diabetes Care. 2011;34(2):347-52. PMCID: 3024347. 
63. 
Cohn T, Prud'homme D, Streiner D, Kameh H, Remington G. Characterizing coronary heart disease risk 
in chronic schizophrenia: high prevalence of the metabolic syndrome. Can J Psychiatry. 2004;49(11):753-60. 
64. 
Goff DC, Cather C, Evins AE, Henderson DC, Freudenreich O, Copeland PM, Bierer M, Duckworth K, 
Sacks FM. Medical morbidity and mortality in schizophrenia: guidelines for psychiatrists. J Clin Psychiatry. 
2005;66(2):183-94; quiz 47, 273-4. 
65. 
Correll CU, Frederickson AM, Kane JM, Manu P. Metabolic syndrome and the risk of coronary heart 
disease in 367 patients treated with second-generation antipsychotic drugs. J Clin Psychiatry. 2006;67(4):575- 
83. 
66. 
Del Valle MC, Loebel AD, Murray S, Yang R, Harrison DJ, Cuffel BJ. Change in framingham risk 
score in patients with schizophrenia: a post hoc analysis of a randomized, double-blind, 6-week trial of 
ziprasidone and olanzapine. Prim Care Companion J Clin Psychiatry. 2006;8(6):329-33. 
67. 
McHorney CA, Ware JE, Jr., Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. 
Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 
1993;31(3):247-63. 

68. 
McHorney CA, Ware JE, Jr., Lu JF, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF- 
36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 
1994;32(1):40-66. 
69. 
Ware JE, Jr., Kosinski M, Bayliss MS, McHorney CA, Rogers WH, Raczek A. Comparison of methods 
for the scoring and statistical analysis of SF-36 health profile and summary measures: summary of results from 
the Medical Outcomes Study. Med Care. 1995;33(4 Suppl):AS264-79. 
70. 
Tunis SL, Croghan TW, Heilman DK, Johnstone BM, Obenchain RL. Reliability, validity, and 
application of the medical outcomes study 36-item short-form health survey (SF-36) in schizophrenic patients 
treated with olanzapine versus haloperidol. Med Care. 1999;37(7):678-91. 
71. 
Russo J, Trujillo CA, Wingerson D, Decker K, Ries R, Wetzler H, Roy-Byrne P. The MOS 36-Item 
Short Form Health Survey: reliability, validity, and preliminary findings in schizophrenic outpatients. Med 
Care. 1998;36(5):752-6. 
72. 
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure 
functional health literacy. Patient Educ Couns. 1999;38(1):33-42. 
73. 
Hargittai E. An Update on Survey Measures of Web-Oriented Digital Literacy. Soc Sci Comput Rev. 
2009;27(1):130-7. 
74. 
Bryk A. Hierarchical linear models: applications and data analysis methods. Newbury Park, CA.: Sage; 
1992. 
75. 
Ten Have TR, Localio AR. Empirical Bayes estimation of random effects parameters in mixed effects 
logistic regression models. Biometrics. 1999;55(4):1022-9. 
76. 
Diggle P, Liang K, Zeger S. Analysis of longitudinal data Oxford: Clarendon Press.; 1994. 
77. 
Ten Have T, Elliot M, Joffe M, Datto C. Causal Models for Randomized Physician Encouragement 
Trials in Treating Primary Care Depression. Journal of the American Statistical Association. 2004;99(485):16- 
25. 
78. 
Ten Have T, Joffe M, Lynch K, Maisto S, Brown G, Beck A. Causal mediation analyses with rank 
preserving models. Biometrics. 2007;in press. 
79. 
Muthen L, Muthen B. Mplus users guide: the comprehensive modeling program for applied researchers 
(Version 2.14). . Los Angeles: CA.; 1998 
80. 
MacKinnon D, Dwyer J. Estimating mediated effects in prevention studies. Evaluation Review. 
1993;17(2):144-58. 
81. 
Kenny DA, Calsyn RJ, Morse GA, Klinkenberg WD, Winter JP, Trusty ML. Evaluation of treatment 
programs for persons with severe mental illness: moderator and mediator effects. Eval Rev. 2004;28(4):294- 
324. 
82. 
Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text 
messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a 
randomized controlled trial. JAMA. 2012;307(16):1702-8. 
83. 
Szilagyi PG, Adams WG. Text messaging: a new tool for improving preventive services. JAMA. 
2012;307(16):1748-9. 
84. 
Lieberman JA, Stroup TS, McEvoy JP, Swartz MS, Rosenheck RA, Perkins DO, Keefe RS, Davis SM, 
Davis CE, Lebowitz BD, Severe J, Hsiao JK. Effectiveness of antipsychotic drugs in patients with chronic 
schizophrenia. N Engl J Med. 2005;353(12):1209-23. 
85. 
Steiner JF, Prochazka AV. The assessment of refill compliance using pharmacy records: methods, 
validity, and applications. J Clin Epidemiol. 1997;50(1):105-16. 
86. 
Cramer JA, Benedict A, Muszbek N, Keskinaslan A, Khan ZM. The significance of compliance and 
persistence in the treatment of diabetes, hypertension and dyslipidaemia: a review. Int J Clin Pract. 
2008;62(1):76-87. 
87. 
Druss BG, von Esenwein SA, Compton MT, Zhao L, Leslie DL. Budget impact and sustainability of 
medical care management for persons with serious mental illnesses. Am J Psychiatry. 2011;168(11):1171-8. 
88. 
Cohen JW, Cohen SB, Banthin JS. The medical expenditure panel survey: a national information 
resource to support healthcare cost research and inform policy and practice. Med Care. 2009;47(7 Suppl 1):S44- 
50. 
89. 
Hlatky MA. Economic endpoints in clinical trials. Epidemiol Rev. 2002;24(1):80-4. 

90. 
Rosenheck RA, Druss B, Stolar M, Leslie D, Sledge W. Effect of declining mental health service use on 
employees of a large corporation. Health Aff (Millwood). 1999;18(5):193-203. 
91. 
Wolff N, Helminiak TW, Tebes JK. Getting the cost right in cost-effectiveness analyses. Am J 
Psychiatry. 1997;154(6):736-43. 
92. 
Gold MR. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996. 
93. 
Manning WG, Jr. Panel on cost-effectiveness in health and medicine recommendations: identifying 
costs. J Clin Psychiatry. 1999;60 Suppl 3:54-6; discussion 7-8. 
94. 
Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the Panel on Cost- 
effectiveness in Health and Medicine. JAMA : the journal of the American Medical Association. 
1996;276(15):1253-8. 
95. 
Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L. Toward evidence-based 
quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and 
implementation strategies 1966-1998. J Gen Intern Med. 2006;21 Suppl 2:S14-20. 
96. 
Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, Orlewska E, Watkins J, 
Trueman P. Principles of good practice for budget impact analysis: report of the ISPOR Task Force on good 
research practices--budget impact analysis. Value Health. 2007;10(5):336-47. 
97. 
McBryde-Foster MJ. Break-even analysis in a nurse-managed center. Nurs Econ. 2005;23(1):31-4, 3. 
98. 
Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in 
translational research. Journal of the Royal Society of Medicine. 2011;104(12):510-20. 
99. 
Washington J. For minorities, new "digital divide" seen.; 2011 January 10, 2011 
100. 
Sarashohn-Kahn J. How smartphones are changing helath care for consumers and providers: California 
Helath Care Foundation; 2010 April 2010 
101. 
Ettner R. Smartphones to Overtake Feature Phones in U.S. by 2011; 2010 
102. 
Cuzick R, Edwards R, Segnan N. Adjusting for non-compliance and contamination in randomized 
clinical trials. Stat Med. 1996;16(9):1017-29.