Impact of High Deductible Health Plans on Patients With Bipolar Disorder
Conditions: Bipolar Disorder
Sponsor: Harvard Pilgrim Health Care

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
1 
 
 
 
 
 
 
 
Study Title: Impact of High Deductible Health Plans on Patients With 
Bipolar Disorder    
Identifier: NCT02560701 
Unique Protocol ID: IHS-1408-20393 
Date of Document: 03/31/2016 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
2 
 BACKGROUND [Relevant PCORI Methodology Standard RQ-1] 
A.1.  Bipolar Disorder is a Major Cause of Disability, Morbidity and Mortality [RQ-3]: Bipolar disorder is a 
severe mental illness characterized by early onset, high risk of recurrent mania or hypomania and depressive 
episodes, persistent risk of suicide, and low rate of fully sustained recovery. It is the sixth leading cause of 
adult disability in industrialized countries.2 In the US, the 12-month prevalence of bipolar spectrum disorders is 
2.8% and the lifetime prevalence is 4.4%.3 These estimates include bipolar-I and –II disorder as well as 
subthreshold bipolar disorders. Subthreshold cases account for half of bipolar spectrum disorders;3 although 
less intense, they cause significant morbidity and functional disability.4 Patients with bipolar disorder have a 
~10-15% lifetime suicide risk,5 and, compared to the general public, greater risk of substance abuse,3 
cardiovascular disease, diabetes, and obesity.6 Further, bipolar disorder causes poor social and psychological 
functioning, work impairment and absenteeism, and a low health-related quality of life.7  
A.2.  Efficacious Treatments Exist for Bipolar Disorder: Although specific psychotherapies have 
demonstrated efficacy in improving bipolar disorder outcomes,8-13 pharmacotherapy remains the foundation of 
acute and long-term management.14,15 Acute treatment aims to stabilize mood and relieve mania or depression 
symptoms. Maintenance treatment is required to decrease mortality, and prevent relapse and recurrence.14 
Effective medications for bipolar disorder include lithium, anticonvulsants, and antipsychotics.14,16,17 While 
patients with bipolar disorder can have asymptomatic periods, episodes of clinical instability are common;18,19 
thus office visits to general practitioners and mental health specialists are also a fundamental component of 
high quality pharmacological and interpersonal care.14  
A.3.  High-Deductible Health Plans in the US are Expanding at Unprecedented Rates [RQ-5]: The US has 
been grappling for decades with rising health spending, which outpaces inflation and is often cited as a threat 
to the economic sustainability of governments, employers, and families.20 In response, payers21 and 
employers22 have increasingly turned to high-deductible health plans (HDHPs) with high levels of patient cost-
sharing. Compared with traditional plans, HDHPs have lower monthly premiums, but they subject most 
services to annual deductibles of $2,600 to $12,900 per family.23 Families might therefore pay the entire 
deductible amount out-of-pocket before more generous coverage, such as full coverage or coinsurance, begins.  
Two types of HDHPs predominate in the US: Health Savings Accounts (HSA) eligible and HSA-ineligible 
HDHPs. The 2003 Medicare Modernization Act created HSAs that allow employers and employees to 
contribute tax-free funds into accounts for medical services payments.24 Such accounts must be connected to 
HDHPs that have a regulated structure including minimum deductible amounts, maximum annual account 
contributions, and defined coverage requirements. HSA-eligible HDHPs with "value-based" out-of-pocket 
exclusions often fully cover a limited number of preventive services (e.g., cancer screening), have a moderate 
copayment (e.g., $20) for annual primary care physician preventive visits, but require full cost-sharing up to the 
annual deductible for all other services (e.g., mental health visits, medications, tests, hospitalizations). In 
contrast, HSA-ineligible HDHPs (i.e., Health Reimbursement Arrangement plans or those without an 
associated account) are also designed to be "value-based," but generally include more generous drug 
coverage (i.e., relatively low medication copayments as in traditional plans) and specialist visits might be 
exempt from the deductible; most other services including mental health visits and hospitalizations are subject 
to the deductible.  
"Value-based" HDHPs can be defined as those with low or no out-of-pocket costs for certain preventive 
services or medications with evidence of effectiveness.25 Proponents theorize that these financial incentives 
will steer patients toward testing and treatments that have a high benefit-to-cost ratio, ultimately driving down 
healthcare costs. However, this definition of "value" is generally a top-down conception from policymakers, 
payers, and insurers hoping to reduce long-term healthcare spending. The voice of patients in articulating what 
care is "valuable" to them is largely excluded. In addition, evidence that value-based HDHPs improve 
outcomes and lower costs is limited at best.26,27 HDHP critics are concerned that patients facing higher out-of-
pocket cost burdens, especially vulnerable populations such as those with chronic mental illness, might defer 
or avoid needed care.28,29  

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
3 
Figure 1. Growth of HDHPs  
(from The Kaiser Family Foundation, Employer Health Benefits Annual Survey 20141)
Both HSA-eligible and -ineligible 
HDHPs have shown unprecedented 
growth;21 national enrollment 
quadrupled between 2006 and 2014 
and 41% of workers now have 
HDHPs (Figure 1).1,30 The 
Affordable Care Act, which went into 
full effect in 2014 is likely to cause 
an “explosion”31,32 in HDHPs 
because of their lower premiums, 
coverage mandates, and an 
imminent "Cadillac tax" on more 
generous plans.  
A.4.  Insurance Coverage Affects 
Treatment and Outcomes for 
Patients with Serious Mental 
Illness: Although most prior studies 
of medication coverage among the severely mentally ill have not included commercially-insured individuals, 
their results are nonetheless illuminating. Patients with mental illness are particularly vulnerable to restrictions 
in coverage, especially medication coverage, due to either financial barriers (e.g., cost-sharing, caps on 
reimbursable prescriptions) or administrative barriers (e.g., prior authorization requirements). Financial barriers 
are commonly used in private health plans to manage psychotropic medication utilization.33 Increased patient 
cost-sharing could worsen already high rates of non-adherence among the mentally ill.34,35 In previous studies, 
we found that imposing caps on the number of reimbursable prescriptions or increasing drug copayments 
reduced adherence to psychotropic medications and increased rates of emergency services, nursing home 
admissions, and partial hospitalizations.36-38 Even a modest increase in copayments from $2 to $7 caused a 
25% decrease in psychiatric medication refills among patients with schizophrenia.39 Higher patient cost-sharing 
is also a barrier to antipsychotic medication adherence in commercially insured patients.40  
Administrative barriers are designed to encourage physicians and patients to substitute lower cost for 
higher cost services. Our studies in Medicaid/Medicare populations suggest that these policies can create 
unintended adverse effects among patients with mental illness (e.g., reductions in treatment initiation, 
premature discontinuation of therapy, unintended switching among medications) without appreciable cost 
savings to the program.41-45  
In addition, certain patient subgroups are at a higher risk of non-adherence. Factors such as comorbidity 
burden, race/ethnicity, and income may influence adherence. Co-occurring somatic conditions put patients with 
mental illness at a greater risk of adverse health events and high health care costs.46,47 Black Medicaid patients 
with schizophrenia are less likely to have adequate medication adherence in either the acute or maintenance 
treatment phase.48 Similar racial/ethnic disparities in both treatment and adherence exist among patients with 
bipolar disorder.49,50 Although the effect of income on adherence among the mentally ill is not clear, it has been 
established that low-income patients with a chronic somatic illness are more sensitive to copayment changes 
than comparable high-income patients.51  
A.5.  Modern HDHPs Might Reduce both Appropriate and Inappropriate Health Care [RQ-5]: The 
landmark RAND Health Insurance Experiment of the 1970-80s found that cost sharing reduces use of both 
appropriate and inappropriate health care, including diagnostic testing, treatment, and hospitalizations.52 The 
poorest and sickest individuals subject to high cost-sharing had a 10% projected increase in mortality.52 The 
RAND study suggested that individuals with poorer mental health status had a relatively more favorable 
response to free mental health care.53 Nevertheless, this study, with a total sample size of only ~2000 adults, 
could not assess outcomes for individuals with less common conditions such as bipolar disorder. In addition, 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
4 
Figure 2. Study conceptual model 
the RAND experiment occurred in an era with a vastly different range of medications, tests, and services. 
There are few well-controlled longitudinal studies of the impacts of modern HDHPs and, to our knowledge, 
no controlled studies of HDHPs among the chronically mentally ill. Our research group has used rigorous 
research designs to study patients’ utilization of health care after transition to HDHPs (see Section F below). 
We found that HDHPs reduced hospitalizations and emergency department (ED) visits, primarily for low 
severity medical problems.54 We also identified a trend toward reduced colorectal cancer screening, with 
HDHP patients more likely to receive a lower-cost, less sensitive test.55 With respect to medication use, other 
investigators have found that subjecting medications to full cost-sharing under HDHPs reduced use of chronic 
illness medications including cardiovascular and asthma drugs.56,57 Our study found that adherence is largely 
preserved when HDHP designs include more generous medication copayments as in traditional plans.58 This is 
important from a policy perspective because it suggests that tailored health plans could preserve access to 
essential treatments. Difficulties affording medications could be followed by complications such as 
hospitalizations, representing increased suffering from the patient perspective. Among patients with mental 
illness, it is not known how HDHPs impact quality of care, patient outcomes, and cost burden. Figure 2 shows 
the theoretical basis for our research, illustrating how considerable increases in patient cost-sharing and 
complex benefit structures under HDHPs may affect mental health care. 
A.6.  Patients Perspectives on Access to Treatment for Bipolar Disorder [RQ-3; RQ-6]: Few published 
studies explore patients’ perspectives on either the impact of HDHPs or the role of patient cost-sharing in 
treatment for bipolar illness. No studies, 
quantitative or qualitative, specifically examine 
the impact of HDHPs on bipolar patients. A 
general survey of new HDHP enrollees found 
low awareness of the deductible, and over half 
of those who were aware of their deductible 
anticipated forgoing medical care in response.59 
Our previous survey found that lower-income 
families in HDHPs were more likely to report 
forgone care, compared to higher-income 
families;60 among all families with a chronic 
condition, half had difficulty paying for care – 
more than twice the rate among families in 
traditional plans, even after controlling for other 
factors.61 A focus group study of persons with 
disabling physical or mental health conditions 
found that HDHPs limited patients’ ability to 
afford basic health services and medications; 
participants reported forgoing care, taking medical risks, and experiencing anxiety about healthcare choices.62 
A qualitative study found that bipolar patients rarely mentioned medication costs as “concerns or fears;”63 
however, in a follow-up study of those with poor adherence, 60% reported problems accessing care including 
inability to pay.64 Our patient partners at the Depression and Bipolar Support Alliance consider access and cost 
issues to be leading concerns, as did a large majority of respondents to a recent DBSA web survey.65  
A.7.  Need for the Proposed Research [RQ-3]: The proposed study addresses several important gaps in 
current evidence about mental illness and HDHPs. First, decision-makers at all levels (consumers, employer-
sponsors, and private and public sector policymakers) lack reliable information about the quality and continuity 
of care for the hundreds of thousands of individuals living with mental illness in commercial insurance plans 
and about their out-of-pocket costs.66-68 Moreover, there is essentially no information about how commercially-
insured mentally ill patients respond to different cost-sharing arrangements, let alone the much higher levels in 
HDHPs. Third, prior research suggests that HDHPs reduce careseeking and treatment adherence (see above). 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
5 
Patients with mental illness might be at particular risk for disruptions in treatment because expensive 
psychotropic medications and mental health visits are subject to full cost-sharing in many HDHPs. Adherence 
is already low among patients with bipolar disorder (~35%34) and further financial pressures could exacerbate 
nonadherence, resulting in adverse outcomes such as hospitalization. Finally, many previous studies of the 
impact of HDHPs have included subjects with a choice of health plans, thereby increasing individual-level 
selection bias, a major threat to validity.69-71  
Our research will address these important gaps in understanding, while overcoming common research 
limitations. We will include a longitudinal, national, and socioeconomically diverse population, allowing us to 
draw solid and generalizable conclusions about the impact of HDHPs on patients with bipolar disorder. 
Through previously funded studies, we have already constructed many complex variables and algorithms 
needed for the proposed research, and have obtained 9 of the 11 years of insurance claims data from a large 
US insurer. We have also developed cutting-edge analytical methods to better understand the intended and 
unintended consequences of transitioning to HDHPs. Employer-mandated switches to an HDHP provide an 
ideal natural experiment for comparing modern health insurance designs on quality of care, access and 
adherence to medications, burden of out-of-pocket costs, and adverse outcomes such as preventable 
hospitalizations. Moreover, ongoing involvement of patients and patient advocates in our proposed study, 
including incorporation of patients’ perspectives through a range of mechanisms, will ensure a richness that 
has been lacking in previous research.  
B.  SIGNIFICANCE  
The rapid expansion of "value-based" HDHPs that include major cost sharing could have very detrimental 
effects on patients with bipolar disorder. We will take advantage of a massive insurance claims dataset that 
captures the increasingly common "natural experiment" of employers mandating that all employees enter 
HDHPs. We will compare three distinct insurance designs: traditional low-deductible insurance plans and 
HDHPs with and without medications subject to the deductible. We will evaluate (1) changes in medication 
adherence, health services utilization, and quality of care; (2) changes in adverse events; and (3) changes in 
patient out-of-pocket costs after the switch from traditional to HDHP insurance. We will assess these effects in 
the overall population of patients with bipolar illness and in specific vulnerable subgroups, including 
racial/ethnic minorities, poorer patients, rural patients, and patients with high comorbidity burdens.  
The voices of patients -- especially vulnerable patients with mental illness -- have largely been excluded 
when determining which "valuable" services should be exempt from high cost sharing under HDHPs. We will 
address this critical gap by conducting in-depth interviews with a sample of patients and holding ongoing 
reciprocal dialogue with stakeholders, including a community of patients with bipolar disorder. Stakeholders will 
help to refine our quantitative analyses and complement them with a deeper understanding -- one that cannot 
be readily derived from claims data alone -- of key issues surrounding access to mental health treatment. 
B.1.  Our Proposed Study Addresses Major Research Gaps [RQ-1; RQ-6]: Our proposed study aligns 
closely with recently published PCORI research priorities. PCORI has prioritized research about bipolar illness 
because of its prevalence and severity, treatment complexity, and evidence gaps in treatment approaches.72 
PCORI’s 2012 “National Priorities for Research and Research Agenda”73 calls for studies on "Improving Health 
Systems," especially on “new system-level strategies ... that have not been rigorously evaluated”, “comparative 
studies on the use of incentives,” and “alternative system-level approaches to improving patient access to 
care.” Our comparison of traditional plans and HDHPs, which include benefit features intended to influence 
patient utilization, responds to this call. Our interviews (Aim 4) exploring patient views on the value of different 
services directly respond to the PCORI agenda’s emphasis on outcomes that “patients experience and think 
are important”. Finally, our analyses of vulnerable patients addresses the PCORI agenda by examining 
“differences in patient response” across “socioeconomic, demographic, and other patient characteristics.” 
In April 2013, PCORI’s Advisory Panel on Improving Healthcare Systems identified five topics for special 
consideration, including studies comparing “different insurance features on chronically ill patients’ access to 
care and quality of care;” the PCORI brief specifically noted that the impact of HDHPs “remains unknown for 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
6 
access, quality, and outcomes.” Another brief, “Mental Health and Primary Care Co-Location,” made particular 
note of the high risks of medical comorbidity, nonadherence, treatment complications, and hospitalization 
among patients with mental illness, especially those in rural areas. Our proposal includes a special focus on 
patients with bipolar disorder who live in rural areas and who have high comorbidity levels. Similarly, the 
Institute of Medicine’s (IOM) priority list74 calls specifically for studies that: (1) compare strategies for enhancing 
medication adherence, (2) compare effects of alternate benefit designs and cost-sharing arrangements on 
chronically ill patients, and, (3) delineate barriers to care for populations that experience health disparities. Our 
research plan aligns closely with these IOM priorities. 
  The two most prominent clinical guidelines for treating bipolar disorder (from the American Psychological 
Association14 and the VA/Department of Defense75) both emphasize medication adherence to prevent adverse 
outcomes (such as relapse, hospitalization, and suicide), and consistent clinical follow-up to ensure optimal 
medication adherence. Patient advocacy organizations frequently voice concerns about financial barriers to 
access and the importance of expanded insurance coverage.76 In a 2010 survey65 conducted by our study 
partner, the Depression and Bipolar Support Alliance (DBSA), 71% of respondents reported that psychiatric 
medication costs were a barrier to their treatment. DBSA’s report concluded that the data did not support “the 
common (mis)conception that people with bipolar disorder or depression are in denial and think they don't 
need their medication;” rather, side effects and out-of-pocket costs play very prominent roles. The priorities laid 
out by both the professional guidelines and advocacy organizations point to the significance of our study, which 
will examine important potential barriers to medication adherence and other mental health services. 
B.2.  Our Proposal Has Several Innovations and Major Potential to Influence System Policies [RQ-3]: 
The proposed project features several innovations in the areas of mental health systems research that promise 
to inform future policies and insurance designs. We will construct the largest and most geographically diverse 
observational dataset of commercially-insured patients with serious mental illness, including plan members 
from all 50 states over 11 years. The national setting is more generalizable than typical settings such as 
regional health systems. Our study would also be the first controlled, longitudinal study comparing the impacts 
of HDHPs and traditional plans on patients with a serious mental illness. No current literature exists on 
changes in medication use, outpatient visits, quality of care, or adverse outcomes associated with HDHPs in 
patients with bipolar disorder.  
The large sample size (~109,000 patients with bipolar disorder aged 12-64 years who match our study 
criteria, described below) in our claims data will permit us to compare major contemporary insurance designs. 
This will be the first rigorous study to compare the impacts of HDHPs with and without full drug cost-sharing 
among the mentally ill; subjecting chronic medications to deductibles might profoundly impact treatment 
continuity and quality. Our study will also be the first large enough to examine the HDHP effects on subgroups 
of patients with bipolar disorder who might be particularly vulnerable, including racial minorities, patients with 
high comorbidity, and those in rural areas or with low income. Our unique dataset includes individual-level 
socioeconomic status (SES) measures, enabling more precise inference of SES effects on outcomes. HDHPs 
may exacerbate the health inequalities faced by these vulnerable groups, who may be more likely to forego 
essential care under HDHPs.  
Our quasi-experimental, longitudinal study design will be uniquely rigorous in addressing the question of 
HDHP effects on patients with bipolar disorder. We will use aggregated and patient-level interrupted time 
series regression modeling, adjusted difference-in-differences, and a two-stage propensity matching approach 
to create a closely matched control group. Accumulating evidence77 suggests that our matching approach 
leads to effect estimates that closely approximate randomized controlled trials. Another distinct advantage will 
be our inclusion of individuals offered no choice of insurance plans, minimizing member-level selection bias, a 
major limitation of research in this field.  
 
Our thorough patient engagement component, which includes a study co-investigator based at DBSA, in-
depth interviews with patients currently enrolled in insurance plans comparable to those in our claims dataset, 
and regular feedback from both our Patient/Stakeholder Advisory Panel and the broader DBSA community, will 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
7 
provide our study with energy, ideas for protocol enhancements, a wealth of lived experience for interpreting 
results, and myriad opportunities for disseminating results as they emerge. 
B.3.  Pathways toward Improvements in Care [RQ-3]: HDHPs could cause disruptions in treatment of 
bipolar illness due to cost-related nonadherence or confusion about coverage details. HDHPs could also cause 
reductions in outpatient and inpatient care, or increased psychiatric hospitalizations due to treatment 
disruptions. Because of the rapid growth in HDHPs, our study is highly relevant to policymakers at all levels. 
Both types of HDHPs (with and without medications subject to the deductible) as well as traditional plans with 
no deductible are all highly prevalent in the US. This wide “practice variation” in insurance benefits design 
implies a crucial need to understand the impacts of design differences in order to optimize patient health. Prior 
research indicates that even small differences in insurance design can have powerful consequences for 
vulnerable patients. 
 
Opportunities to implement policy recommendations would be readily available. Insurers redesign benefit 
offerings every year and respond quickly to shifting pressures from employers. Our study potentially includes 
an immediately-available "policy intervention;" if HDHPs with generous medication coverage promote bipolar 
medication adherence and better health outcomes, employers and health insurers could quickly shift to such 
designs. Evidence that exempting mood stabilizing drugs from full cost-sharing preserves appropriate 
utilization could also contribute to amendments to the 2003 Medicare Modernization Act that requires HSA-
eligible HDHPs to subject all medications to the deductible.24 State and federal regulators closely examine 
insurer practices and set standards for benefits packages. We expect that policy makers will use our results to 
introduce health plan designs that promote high quality care among patients with mental illness.  
Given the timely and policy-relevant nature of our research, we expect to share preliminary results at 
scientific conferences, through reports and meetings with stakeholders (PCORI, DHHS, CMS, AHRQ, NIMH, 
MHRN/HMORN, NAMI, major health insurers), and with the DBSA community through news updates and other 
communications. We anticipate 7 published reports in medical and health policy journals with broad circulation. 
Our research group has had considerable success informing health policy decisions. For example, our past 
studies of drug benefit limits among the chronically ill elderly and those with schizophrenia have informed 
policies to improve drug coverage in Medicaid populations, provide Medicare Part D drug coverage subsidies 
for the near poor, close the Medicare drug coverage gap, and include benzodiazepines in the Medicare drug 
benefit.36-38,78,79 This research will continue our tradition of innovative academic/policymaker partnership. 
Our collaborator, DBSA, enhances our capacity to translate study findings to practice. DBSA is a national 
mental health advocacy organization created by and for people who live with mood disorders. DBSA has 
multiple avenues for disseminating research findings and mobilizing patients and caregivers to press for 
needed changes, including 800 local support groups, a popular internet site, a Facebook page with >100,000 
followers, and the Care For Your Mind blog with ~5,000 user sessions per month. DBSA is also soon to launch 
its Parity Campaign, informing patients of new policies (including the Affordable Care Act) that seek to increase 
access to mental health care, and working to ensure that these policy goals are realized. DBSA leaders have 
expressed confidence that the goals of our proposed research align well with their Parity Campaign, and will 
disseminate our study results in tandem with the Campaign as appropriate (see Letters of Support). 
C.  STUDY APPROACH [RQ-2] 
Our study objective is to determine the effects of two types of HDHPs (with and without medications 
subject to the deductible), compared with traditional commercial insurance, on adolescents and adults with 
bipolar disorder. To accomplish this, we will conduct quantitative analyses using a very large, geographically 
diverse dataset of insurer claims complemented by in-depth interviews with individual patients. Insights gained 
from these interviews, from our Patient/Stakeholder Advisory Panel, and from our ongoing partnership with a 
major patient advocacy organization (DBSA) will shape and inform the quantitative analyses.  
Aims 1-3 will use insurance claims data to examine the impacts of ongoing natural experiments whereby 
some employers shift their entire employee populations from traditional insurance to HDHPs with two distinct 
benefit structures. In Aim 1, we will evaluate changes in medication adherence, and in the intensity and quality 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
8 
of other health care following this shift. In Aim 2, we will assess changes in adverse events such as 
preventable psychiatric hospitalizations. In Aim 3, we will evaluate changes in patient out-of-pocket cost 
burden. In each of these first three aims, we will first examine HDHP effects in the overall population of 
patients with bipolar illness, then compare differences in effects between the two types of HDHPs, and finally 
examine changes in specific vulnerable subgroups, including racial/ethnic minorities, poorer patients, rural 
patients, and patients who have high comorbidity burdens. In Aim 4, using interviews, we will explore patient 
experiences coping with complex insurance benefits and gather their views on higher- and lower-value care, 
contrasting these to standard definitions and conceptions of healthcare “value." 
Throughout the three years of study, we will rely on our Patient/Stakeholder Advisory Panel and our 
investigative partnership with DBSA to monitor our progress, to continuously bring in fresh community 
perspectives on our work and its relevance, and to help us refine our study approach. 
C.1.  Overview of Design and Methods: In Aims 1-3, we will use 11 years of insurance claims data on more 
than 109,000 patients with bipolar disorder. Outcomes will include adherence to bipolar medications, rates of 
laboratory-based medication monitoring, psychotherapy visits, use of acute services such as emergency 
department visits and hospitalizations, and out-of-pocket costs (total, by type of service, and per service unit). 
We will first examine trends over calendar time among the overall population and vulnerable subgroups 
defined by income, race/ethnicity, rural residence, and comorbidity burden. We will then rigorously evaluate 
HDHP effects by examining changes in the above outcomes before and after the mandated switch from 
traditional plans to HDHPs. Our studies will generate the first empirical evidence regarding the comparative 
effectiveness of alternative insurance designs in preserving quality and continuity of care among the mentally ill.  
We will use a range of analytical methods tailored to our study outcomes. Specifically, we will use two 
strong quasi-experimental, longitudinal research designs: an interrupted time series with comparison 
series design and a pre-post with propensity-matched comparison group difference-in-differences 
design. The Cochrane Collaboration (an independent organization that produces and disseminates systematic 
reviews of health care interventions) considers both to be strong research designs that permit causal 
inference.80,81 Moreover, we will use a range of statistical methods including generalized linear mixed 
models,82,83 and interrupted aggregate and patient-level time series models58,84 as appropriate for each of our 
study outcomes. Study design features that strengthen the internal validity of the proposed research and 
reduce member- and employer-level selection biases include (i) selecting only employers that offer only one 
health plan type to employees, (ii) using a validated two-level (employer and member) propensity score 
matching approach,77,85 and (iii) propensity score matching on baseline population characteristics, including 
the functional form of the baseline outcome trends.77 Cutting-edge research has shown that this matching 
approach leads to effect estimates that closely approximate randomized controlled trials.77  These approaches 
will reduce baseline differences between employers choosing among the three possible types of insurance and 
control for potential patient-level differences in employee populations.  
In Aim 4, we will conduct in-depth interviews with approximately 40 commercially insured individuals with 
bipolar disorder or their family caregivers to explore how they navigate deductibles, copayments, and other 
complex insurance features. We will also determine the health care services that patients most value and 
assess how they prioritize difficult health care cost tradeoffs.  
C.2  Sources of Data and Access [IR-1]: Our OptumInsight health insurance claims data, spanning 11 
years (2004-2014), is from one of the largest and most geographically diverse commercial health plan in the 
US. This insurer has membership in all 50 states and annual enrollment of ~33 million members; this insurer 
was among the first to offer modern HDHPs. In addition to pharmacy and medical service claims, the de-
identified datasets contain enrollment information and Credit Report data linked to the member file, including 
indicators of individual-level SES rarely used in health care research (household income, net worth, and 
educational attainment). To address the concern of the adequacy of this data source, we note this standard 
health insurance claims dataset is directly from a large national health insurer.  Health insurance claims 
datasets have been used for decades in health services research and their use has been validated for 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
9 
capturing cohorts of patients with bipolar disorder.  Our group86-88and others have published multiple studies 
using these data.  
To assess the efficiency of data linkage between claims data and credit report data and implications for bias 
based on proportion of total sample that may not have these data [IR-2], data vendor Optum will link patient 
addresses to geocoded 2000 and 2010 census data, a process that has previously yielded <1% unlinked data. 
Information on member race and ethnicity is also included, based on geocodes, US census data, and an 
imputation strategy incorporating member surname.[MD-2] We will also have detailed benefit information for 
~80% of employers, including individual and family deductible levels, HDHP type (HSA, Health Reimbursement 
Arrangement, or non-account), and copay and coinsurance amount by service type. For employers with 
missing deductible levels,[MD-2] we have developed an imputation algorithm that has 97.0% sensitivity and 
96.2% specificity. To determine employer deductible levels, we will use a benefits type variable that we have 
for smaller employers (with approximately 100 or fewer employees).  For larger employers, we will take 
advantage of the fact that health insurance claims data are the most accurate source for assessing out-of-
pocket obligations among patients who utilize health services.  Our claims data contain an in-network/out-of-
network deductible payment field.  For patients who use expensive or frequent services, the sum of their yearly 
deductible payments will add up to clearly identifiable exact amounts such as $500.00, $1000.00, $2000.00, 
etc.  When several members have these same amounts, it provides strong evidence that the employer offered 
such an annual deductible level.  It will also be possible to detect employers that offer choices of deductible 
levels when multiple employees have deductibles at two or more levels, such as 20 employees with an annual 
amount of $1000.00 and 12 employees with $500.00.  For employers with at least 10 workers, we therefore will 
sum each employee's in-network deductible payments and number of claims over the enrollment year and plan 
to assess other key characteristics such as percentage with Health Savings Accounts.  On a randomly 
selected half of the employer data set that contained our calculated employer characteristics (such as the 
percentage of patients with deductible levels between $1000-$2500) as well as actual deductible amounts, we 
will use a logistic model that predicts the 3-level outcome of deductible <=$500/$500-$999/>$1000 based on 
multiple aggregate employer characteristics such as the first and second most common whole number 
deductible value, the percentage with Health Savings Accounts or Health Reimbursement Arrangements, the 
median deductible payment, the percentage of employees using services, the employer size, the percentage of 
employees with deductible levels between $100-$500/ $500-$1000/ $1000-$2500/ >$2500, etc.  This 
predictive model will output the probability that employers had deductibles in the three categories (summing to 
1) and we will assign the employer to the level that had the highest probability.  If we detect employers that had 
10 or more employees with whole number deductible levels both above and below $500 (e.g. $250.00 and 
$1500.00), we will assign the employers' category as "choice."  If 100% of employees had Health Savings 
Accounts, we will overwrite any previous assignment to classify the employer as a high-deductible 
employer.  We will test the predictive model on the other half of the sample for which we have actual deductible 
levels.  At employers with 75-100 workers, we anticipate to find sensitivity and a specificity of over 96%.  
We will have all study years (2004-2014) available in-house. The sample size (see section C.10) will be 
sufficient to investigate HDHPs' effects within carefully selected employer groups, as well as within vulnerable 
patient subgroups. 
Our patient partner, DBSA, will help recruit Aim 4 interview respondents through local chapter meetings 
and social media platforms. Interested individuals will contact DBSA, who will screen for eligibility. Participants 
will be non-elderly adult patients with bipolar illness enrolled in commercial plans or caretaking family members, 
able to converse in English; respondents will give informed consent for recorded telephone interviews of 
approximately 1 hour, each conducted using a semi-structured interview guide.  
C.3  Interventions and Comparators [RQ-5; HT-1 to HT-4]: Our analyses in Aims 1-3 will include members 
of traditional health plans (including low-deductible Health Maintenance Organization, Preferred Provider 
Organization, and Point of Service plans) and HDHPs. We define traditional health plans as those having 
annual deductibles of $500 or less for individuals and copayments of less than $50 for most services. We 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
10 
Figure 3. Study design of HDHP members and controls 
define HDHPs as those requiring individuals to spend at least $1000 out-of-pocket annually for most clinical 
services before more comprehensive coverage begins, but which typically have exemptions for preventive 
care.  
We will further classify HDHPs as HSA-eligible and HSA-ineligible. HSA-eligible HDHPs subject most 
medical services (including psychotropic medications and all mental health visits) to the annual deductible; only 
“preventive” health care (a limited set of services such as check-ups and screenings) are under first-dollar 
coverage, either for free or with a modest copayment. By contrast, HSA-ineligible HDHPs also subject many 
services to the deductible, but they generally include more generous coverage for medications (i.e., tiered 
copayments, as in traditional plans) and primary care visits might require only a modest copayment. These 
coverage differences for key services create a natural experiment that will permit us to answer crucial 
questions regarding optimal insurance designs for patients with bipolar illness.  
For Aim 4 in-depth interviews, we will sample respondents with either traditional insurance or HDHPs in 
order to explore this contrast. However, we 
recognize that many respondents may not fully 
understand the design of their insurance plan, and 
we expect new themes to emerge regarding lack 
of benefits clarity. 
C.4  Enrollment Period and Follow-up Duration 
for Aims 1-3: Our study period will extend 11 
years (January 2004 to December 2014), a period 
of rapid growth of HDHPs,22 capturing an 
unprecedented sample size of patients with 
bipolar disorder transitioning to HDHPs. The 
intervention cohorts will include individuals who 
transitioned to HDHPs between January 2005 
(after a minimum one year baseline period in a 
traditional plan) and December 2013 (to allow one 
year of follow-up). We will follow all subjects for 1 
year before and 1 year after the index month of switching (Figure 3). Matched controls will be drawn from 
employers who chose to remain in a traditional plan in the same calendar year, and our matching approach will 
ensure calendar month of switch is also balanced across the study groups. We will require continuous 
enrollment for the 24 month study period for both groups. 
 
C.5  Study Population for Aims 1-3 [RQ-3; CI-1 to CI-4; HT-1 to HT-4]: Inclusion Criteria: We will include 
health plan members aged 12 to 64 years at the beginning of the baseline period. Consistent with prior 
research,89-91 we will use medical claims data and a validated algorithm36,41,43-45,92,93 to identify members who 
had at least two ambulatory encounters or one hospital encounter with a diagnosis of bipolar disorder (ICD-9 
diagnostic codes: bipolar-I: 296.0, 296.1, 296.4-296.7; bipolar-II: 296.89; and bipolar-other: 301.11 and 
301.1343-45,89-91). The first advisory panel committee meeting in October 2015 directly shaped our cohort 
definition stream of work; advisors agreed that bipolar inclusion criteria should include requiring at least 1 
inpatient or 2 outpatient diagnoses, to better ensure that the diagnosis is valid and relevant to the planned 
analyses. Despite the tightening of our diagnostic criteria, our newest cohort totals remain on target with the 
estimates in our original power analysis. Eligibility for the study is predicated on the principal diagnosis of 
bipolar disorder (any diagnosis) not just diagnosis at the first position, as advised by our stakeholder panel.  
The study cohort was derived through applying the following inclusion rules, therefore defining individuals as 
eligible for our study:  
 
 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
11 
Study cohort:  Of 41,638,597 members enrolled in the national insurer during 2004-2012, we found 18,184,556 
have a 2-year enrollment span, a study inclusion criterion. Among these, 148,706 have at least 1 inpatient or 2 
outpatient diagnoses of bipolar disorder; this reflects a prevalence of 1% members with bipolar disorder, which 
is consistent with expectation among the commercially insured population. Among 148,706 members with 
bipolar disorder, 57,028 matched our age criterion (12 to 64 years of age. (We will conduct sensitivity analyses 
of our main results varying our bipolar diagnosis criteria [e.g., requiring diagnosis prior to index date, requiring 
only one diagnosis; CI-5]) To minimize selection bias, our study cohorts will include individuals whose 
employers offer only a single benefit type with no choice. Our intervention cohorts will include traditional plan 
members with bipolar illness who experience an employer-mandated switch to HSA-eligible HDHPs with full 
drug cost-sharing (i.e., out-of-pocket medications payments until the deductible limit is reached), or to HSA-
ineligible HDHPs that subject medications only to copayments as in traditional health plans. The control cohort 
will include members with bipolar illness whose employers offered only a traditional plan for the follow-up year. 
We estimate that 7,227 have HDHP insurance and 49,801 have traditional plan insurance.Exclusion Criteria: 
We will exclude members age 65 years or older who could be eligible for Medicare benefits, including drug 
coverage through Medicare Part D. We will also exclude members whose employer offered a choice of health 
plan. This strengthens our design by reducing potential selection bias due to, for example, healthier members 
selecting HDHPs or individuals anticipating discretionary utilization.  . Input from our panel advisory meetings 
led to the decision to exclude from our cohort those individuals with a diagnosis of schizoaffective disorder or 
schizophrenia in order to identify a clinically homogenous group for our study, as advised by our clinical 
psychiatrist co-investigator Alisa Busch, MD. This criterion was further supported by our advisors at DBSA.  
According to sources at DBSA, schizophrenia and schizoaffective disorder are not comorbidities seen in their 
peer population.  In their judgment, people living with schizophrenia and schizoaffective disorder are unlikely to 
come to DBSA meetings and would have their needs better met in other settings.  The ineligibility criteria 
therefore included the 3 categories:  1) patients with only 1 outpatient bipolar diagnosis, 2) 1 inpatient or 2 
outpatient diagnosis of schizophrenia diagnosis, and 3) 1 inpatient or 2 outpatient diagnosis of schizoaffective 
disorder.  We estimate 3,731 individuals were excluded from the study cohort due to evidence of 
schizoaffective disorder and schizophrenia. There have been no changes in identifying eligible individuals to 
enroll in our approved research plan beyond the stakeholder advisory panel decision to exclude schizoaffective 
disorder and schizophrenia, calculate their overlap with the bipolar population, and exclude them from the final 
cohort.   
 
Subgroups [RQ-4]: We will stratify our analyses by additional characteristics to permit evaluation of vulnerable 
subgroups: comorbidity level (e.g., highest tertile versus intermediate and lowest tertiles of validated Adjusted 
Clinical Groups score94,95), SES (e.g., highest versus lowest tertile household income), rural versus urban 
residency, and race/ethnicity group152-153(non-Hispanic white versus black, Asian, Hispanic, and other). These 
variables are described in section C.8.   For Rural versus non-rural areas; non-rural areas will be the reference 
group. We will use Rural-Urban Commuting Area (RUCA) codes that reflect travel and shopping patterns at the 
county level. Using an established probabilistic ZIP code-to-county file,148 we will assign patients by their 
residential ZIP codes to RUCA groups. Similar to previous research,149-151 we will categorize the 10-point 
RUCA classification into: “rural” (RUCA codes 7-10) and “non-rural”, comprising large rural cities/towns (RUCA 
codes 2-6) and urban areas (RUCA code 1). 
 
C.6  Matching Strategy for Aims 1-3 [CI-5]: For comparing effects among insurance designs, we will use 
propensity score matching based on both employer- and member-level characteristics to balance the study 
and control groups on observed characteristics as in our prior work.96-98  We will first match similar employers 
(to minimize employer selection effects) and then match individuals within groups of similar employers to 
ensure high comparability between study groups. The individual match will include matching on the baseline 
outcome measure, an approach crucial to generating estimates comparable to randomized controlled trials.77,85 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
12 
We will use the sample selected with this method for interrupted time-series analyses and for pre-post with 
comparison group difference-in-differences (Section C.9). Use of comparison groups matched at both the 
employer- and individual-level will increase comparability between HDHP and traditional plan cohorts in 
baseline employer characteristics, member demographic characteristics, and baseline health care utilization. 
This offers a further degree of control for any potential contemporaneous changes that may influence specific 
study outcomes.41-44,54,55,80,99  
Employer-level Propensity Matching: The study population will include employees of firms that mandated a 
switch to HDHPs and others that kept traditional plans as their sole insurance option. While this minimizes 
member-level selection into health plans, our previous work indicates that some employers might select 
HDHPs based on employee characteristics.100 To create strata of employers with similar baseline 
characteristics, we will generate scores predicting the propensity of an employer switching to an HDHP in a 
given year among all employers making such a decision. Propensity scoring is a well-established method that 
assists in generating a comparison group with a similar likelihood of being exposed to a given “intervention” (in 
this case, shifting to HDHP coverage) based on measured characteristics when subjects have not been 
randomly allocated into study groups.101-104 We will use logistic regression to predict an employer’s likelihood of 
switching to an HDHP (versus remaining in a traditional plan). Candidate predictors in our propensity score 
models will include baseline employer-level covariates (Section C.8). We will use results from this model to 
divide employers into four strata according to propensity score, then match HDHP members with bipolar illness 
to traditional plan members with bipolar illness within each stratum (see immediately below).104  
Individual-level Propensity Matching: To identify closely matched patient controls for each of our HDHP study 
members with bipolar illness, we will use member-level propensity score matching101 to select members within 
the eligible pool of employees from each stratum of matched employers. Based on individual-level covariates 
(Section C.8) and similar enrollment dates, we will match four contemporaneous controls with bipolar disorder 
to each HDHP member with bipolar disorder within a pre-defined propensity score caliper (0.6 of the standard 
deviation of the pooled baseline propensity score).105  
C.7  Outcome Measures for Aims 1-3 [RQ-6]: We will examine a range of validated outcomes58,89,90,106-108 that, 
based on the literature, are regarded by clinical experts and policymakers as important. Each outcome domain 
has also been identified as important to patients and caregivers, based on the literature63-65 and advice from 
our Patient/Stakeholder Advisory Panel. Aim 4 interviews and community feedback throughout the study will 
provide further information about the importance to patients of these and potential alternative measures.  
(Aim 1) Access to Appropriate Outpatient Treatment: We will measure several domains of outpatient care: 
(i) 
Use of outpatient services: We will measure overall access to outpatient services with monthly 
prevalence and counts of different categories of outpatient visits (total, primary care non-mental health, primary 
care mental health, and mental health provider/substance abuse specific [defined by Current Procedural 
Terminology psychiatric codes (908xx series) and ICD-9 codes 290-310109]). DBSA advisors on our advisory 
panel have also contributed to the decision to include and examine use of psychotherapy visits. 
(ii) 
Regular mental health follow-up visits: While patients with bipolar disorder may have asymptomatic 
periods, episodes of clinical instability are common18,19 thus periodic monitoring is recommended even for 
those who are euthymic. As a measure of treatment quality, we will examine changes in receiving at least one 
outpatient mental health/substance abuse visit per quarter. 
(iii) 
Medication adherence for bipolar disorder: As a measure of overall access to medications, we will 
examine overall monthly and annual prevalence rates of treatment with medications that are either guideline 
recommended or indicated by the Food and Drug Administration for bipolar disorder, including 1st and 2nd 
generation antipsychotics, anticonvulsants, and lithium14,16,17 Our measures of quality of care will include two 
patient-level monthly measures of medication availability58,106,107: the average standardized dose of bipolar 
medications available per day, and the proportion of days with each type of bipolar medication available (i.e., 
proportion of days covered106,107). We can use these to assess the association between switching to an HDHP 
and changes in the average dose of medications available, or a change in the likelihood of skipping days of 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
13 
treatment with a particular medication (e.g., proportion of days covered by a mood-stabilizing agent).89,90,110  
(iv) 
Guideline-recommended clinical monitoring: The American Psychiatric Association14 recommends 
routine drug level monitoring for a number of bipolar disorder medications. Our quality measures will include: 
for lithium, carbamazepine and valproate users: testing for drug serum level at least once in a year;111 for 
lithium users: renal and thyroid function tests every 6 months; and for valproate and carbamazepine users: 
blood counts and hepatic function tests every 6 months.14,111 In addition, use of 2nd generation antipsychotics 
may be associated with cardiometabolic risk through effects on body weight, and lipid and glucose 
metabolism.112,113 Consensus guidelines recommend that patients receiving 2nd generation antipsychotics be 
monitored for lipid and glucose levels every 3 months to ascertain risk of diabetes and cardiovascular 
disease.114 Thus, we will also assess lipid (total cholesterol and triglycerides) and serum glucose testing every 
6 months in users of these medications, as in previous research.108  
(Aim 2) Adverse Patient Clinical Outcomes: We will examine changes in rates of: (i) psychiatric and non-
psychiatric ED visits and day hospitalizations to determine if cost-sharing under HDHPs resulted in adverse 
short-term health events among patients with bipolar disorder, and (ii) psychiatric and non-psychiatric 
hospitalizations and hospitalization days as evidence of the impact of HDHPs on overall rates of serious 
morbidity. As in our prior work, we will define psychiatric-related events as those associated with a primary 
diagnosis of bipolar disorder (ICD-9: 296.0, 296.1, 296.4-296.8, 301.11 and 301.1343-45,90), depression (ICD-9: 
296.2, 296.3, 298.0, 300.4, 309.1, and 31142), or schizophrenia (ICD-9: 29536,41), substance abuse (ICD-9: 291, 
292, 303-305.0, 305.2-305.7, 305.9109), or events in psychiatric hospitals.45  
(Aim 3) Patient Out-of-Pocket Cost Burden: We will examine annual patient out-of-pocket costs (paid 
deductible, coinsurance, and copayment amounts), summing these discretely for prescription fills, outpatient 
visits, and other major service categories. We will calculate both the overall annual burden of costs per patient, 
and measures such as average cost per fill or per month of medication, and per behavioral health visit, which 
may be easier for lay consumers of our research results to consider and compare. To control for price inflation, 
we will convert all patient costs to current year US dollars using the Consumer Price Index.115 
C.8  Covariates for Aims 1-3: We will use (i) employer-level and (ii) individual-level covariates to propensity 
match, adjust, or stratify our analyses.  
Employer-level Covariates include: employer size, health plan expenditure quintile, employee out-of-pocket 
cost quintile, median employee age, median employee comorbidity score, and proportions of employees who 
are women, have family coverage, and reside in high-poverty or low-education neighborhoods. Our ongoing 
research has found that health plan and employee out-of-pocket costs independently predict employer HDHP 
enrollment. We will calculate baseline expenditures by summing standardized amounts paid by the health plan 
for all services utilized by all employees; the data vendor provides these standardized cost amounts. We will 
use median employee Adjusted Clinical Groups scores94,95 to estimate comorbidity; the ACG algorithm uses 
age, sex, and ICD-9-CM codes to calculate a morbidity score relative to a reference population average of 
1.0.94 Researchers have validated the index against premature mortality.95 
Member-level Covariates include: bipolar disorder type, age, sex, SES, race/ethnicity, state of residence, 
rurality, Dartmouth Atlas Hospital Referral Region of residence, baseline out-of-pocket and health plan 
expenditures, number and month (relative to index date) of outpatient/ED/hospital visits, and enrollment in 
individual versus family coverage. To derive indicators of race/ethnicity, we will first use a variable supplied by 
the data vendor that applies surname analysis to categorize members as Hispanic or Asian. We will then use 
US Census block group data116 (geocoding) to indicate predominantly black or white neighborhoods. We will 
combine both techniques because geocoding is not sensitive in detecting ethnicity117 and surname analysis is 
non-informative regarding race.[MD-2] This validated approach has a high positive predictive value.118-122 We 
will have access to individual-level Credit Report-derived variables for income and net worth, which is a major 
advance over previous studies that used only neighborhood SES measures. We will also determine 
neighborhood SES (geocoded poverty and education levels) and use principal components approach to 
calculate a neighborhood SES index.99,123,124,125 Adjusted Clinical Groups scores94,95 will estimate comorbidity.  

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
14 
To account for SES variables’ data missingness among HDHP and traditional plan adult members, we will use 
3 methods to accommodate missingness: (a) multiple imputation, (b) modeling missingness as a member-level 
characteristic, and (c) excluding members with missing data.  If findings differ, we will present results using all 
3 methods in manuscripts or attached appendices.  We will use SAS procedures PROC MI and PROC 
MIANALYZE to impute the missing individual level covariates and conduct the statistical analyses under the 
assumption of missing at random. The SAS procedure PROC MI will impute the missing variables (including 
individual net worth, income level, poverty level and education level) using other available variables such as 
age, gender, census block level income, poverty, education and etc. Then we will use PROC MIANALYZE to 
analyze the imputed data. We will also explore certain Bayesian imputation techniques to handle missing data 
in case missing is not at random and compare the results.146,147 
 
C.9  Analysis Plan for Aims 1-3 [IR-3; HT-1 to HT-4]: Overview: These analyses will determine how specific 
outcomes change after employer-mandated switching from traditional plans to HDHPs.  To address 
methodological standard IR-3, we plan to use the most rigorous quasi-experimental retrospective longitudinal 
designs and analyses available for AIMS 1-3, namely interrupted time series with comparison series matched 
on multiple baseline covariates as well as the baseline trend of outcome measures.  For Aim 4 we will use 
appropriate standard techniques for qualitative analyses of interview data (detailed below).  We will first 
compare the characteristics of our matched cohorts (HSA-eligible and HSA-ineligible HDHP members, and 
traditional plan controls) using chi-square tests, t-tests and Poisson or quantile regression.126 We will contrast 
sociodemographic characteristics, employer characteristics, and baseline health care utilization. Subsequent 
analyses will use time-series plots and rigorous analytical methods appropriate for each outcome, described 
below. Data collected on the same individuals in successive years are correlated (“repeated observations”). 
The extended general linear models—generalized estimating equations (GEE) and generalized linear mixed 
models (GLMM) for binary, count, or continuous measured outcomes—are appropriate methods for estimating 
parameters to adjust for these correlations within subjects.82,83 The term of interest in our GEE or GLMM 
models will be the interaction between time (pre versus post) and cohort (HDHP versus traditional plan).  
Statistical analyses in Aims 1-3 will first compare those transitioning to HDHPs with patients remaining in 
traditional plans; then contrast effects between HDHP members with full drug cost-sharing (HSA-eligible plans) 
and those transitioning to HDHPs without full drug cost-sharing (HSA-ineligible HDHPs). Each HDHP group will 
have as controls only those traditional plan members paired to a given HDHP member during the propensity 
score match. To examine possible selection effects due to applying our enrollment criteria and propensity 
matching approaches (see C.10), we will also compare our analytic population to members with bipolar 
disorder who were excluded from the sample.[MD-4; CI-2] 
(Aim 1) Access to Appropriate Outpatient Care: We will assess changes before and after the index month in 
the utilization and adherence outcomes listed in C.7 for Aim 1 using patient-level interrupted time series with 
comparison series. The interrupted time series design includes multiple observations over time both before and 
after an intervention (i.e., the plan switch) and adjusts for most threats to internal validity (e.g., secular changes 
in prescribing, aging of the population) because it adjusts for baseline trends in study outcomes that are 
unrelated to the intervention.80 Furthermore, our strategy of matching on the baseline outcome trend should 
generate highly robust causal estimates. We will use patient-level segmented linear regression models58,84 to 
statistically estimate patient-level changes in both level and trend in outcomes after the switch to HDHPs, while 
controlling for autocorrelation of the data and individual-level covariates. The segmented regression models 
include a constant term; an integer variable indicating time in months (t) from the start of the observation period 
to estimate baseline trend; a binary variable, intervention, indicating the period after the index month, to 
estimate change in level; and an interaction between intervention and time, to estimate change in trend. We 
will test the statistical significance of level or trend changes, adjusting for first-order autocorrelation between 
sequential monthly measurements using the empirical sandwich estimator in GEE.58    
For selected quality measures (i.e., likelihood of a quarterly outpatient mental health visit or receiving 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
15 
guideline-recommended lab monitoring when using lithium, valproate, carbamazepine, and 2nd generation 
antipsychotics; C.7), our approach will be a pre-post with comparison group and difference-in-differences 
design. Logistic regression models will assess the independent effect of HDHPs on these outcomes, while 
controlling for patient-level covariates and secular utilization trends (by including index month).54,55,99 
(Aim 2) Adverse Patient Clinical Outcomes: To examine the impact of transitioning to HDHPs on changes in 
rates or risk of psychiatric/non-psychiatric ED visits and hospitalizations, we will use a pre-post with 
comparison group difference-in-differences design. We will control for patient-level covariates in the models. 
We will use GLMM to model the effect of HDHPs on annual hospitalization days. 
(Aim 3) Patient Out-of-Pocket Costs Burden: Analysis of changes in patient costs will rely primarily on a pre-
post with propensity-matched comparison group design and a difference-in-differences analytical framework. 
We will model costs to patients using two-part general linear models. We will select the conditional mean and 
variance functions based on the actual data, potentially using a log link with a Gamma error distribution.127,128 
To inform the final model specification, we will employ specification tests including the Pregibon Link test for 
non-linearity129 and a modified Park test to select the conditional variance function.127 
Stratified Analyses to Assess Effects on Vulnerable Subgroups: We will compare outcomes between 
HDHP and traditional plan members among strata defined by four covariates: comorbidity burden (e.g., higher 
versus low tertiles of Adjusted Clinical Groups score,81,82 SES (higher versus low tertiles); race/ethnicity (non-
Hispanic white versus black, Asian, Hispanic, and other), and rural versus urban residency. We will use GEE 
or GLMM to express relative changes in outcomes among HDHP member subgroups compared to changes 
among controls.130 If statistical power permits, we will add three-way interaction terms (e.g. 
study_group*study_period*SES) to assess whether the HDHP switch was associated with a change in 
baseline disparities in outcomes. 
C.10  Sample Size Estimates and Power Calculations [HT-1 to HT-4]:  Using 10 years of data (2003-2012), 
we found 42 million unique adolescents and adults aged 12-64, approximately 450,000 of whom (~1%) had at 
least one bipolar disorder diagnosis. This is consistent with our expectation of a lower disease prevalence 
among commercially-insured adults compared with the 3% prevalence estimated by epidemiological surveys,3 
because individuals with bipolar disorder less commonly have private insurance.131 A total of 109,012 of the 
450,000 with a bipolar diagnosis met our inclusion criteria of 2 years continuous enrollment, with the first year 
in a traditional plan, the second in a traditional or HDHP, and no choice from the employer at any point 
regarding plan type. Projecting to 2014, our sample will include over 10,800 HDHP members (1525 HSA and 
9275 non-HSA) and 43,200 matched traditional plan controls with bipolar illness (1:4 matching). Our smallest 
HDHP subgroups examined for primary outcomes will include 1525 HSA members (smallest medication 
adherence cohort) and 2160 non-white members (smallest emergency department and hospitalization 
outcome cohort).  
For interrupted time series analyses of medication adherence, based on assumptions including 12 months 
of follow-up, autocorrelation of less than 0.3, and 0.05 type I error,132 we will have greater than 90% power to 
detect a 6% relative decrease in adherence in our smallest medication outcome subgroup (HSA-HDHP 
members).  For example, we could detect an adherence rate change from 49% (as seen in previous 
literature133) to 46% with a standard deviation of 1.5%.  This is a smaller change in adherence than has been 
observed previously in HDHP-HSA studies.57  
We will use generalized estimating equations to model difference-in-differences analyses of our least 
frequent outcomes, including emergency department visits and hospitalizations. Based on the smallest 
subgroup of 2160 non-white HDHP members and their 8640 matched traditional plan controls, a baseline 
psychiatric hospitalization rate of 10% per year,89 (the lowest rate among our measures), 0.05 type I error, 80% 
power, and a conservative repeated observations correlation assumption of 1, we will be able to detect a 20% 
relative decrease (e.g., from 10% to 8%). This is smaller than the 27% hospitalization decline we detected in a 
previous HDHP study.54  We will have more power to detect changes for all other outcomes. The power 
estimates were calculated using PASS software.134 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
16 
C.11  Study Population and Methods for Aim 4 [RQ-3; PC-2]: Ours will be the first qualitative study to 
explore views of patients with bipolar illness regarding insurance-related barriers to care and experiences with 
commercial insurance benefits. Qualitative results will complement and inform our quantitative claims-based 
analyses in Aims 1-3. For Aim 4, DBSA will assist in recruiting individuals living with bipolar disorder or 
close family members to participate in semi-structured telephone interviews. The opportunity to volunteer for 
participation in interviews will be presented in local DBSA chapter in the form of a flyer for attendees.  These 
flyers will be distributed by DBSA to chapter leadership by means of US mail, email, and routine in-person 
outreach visits by DBSA officials to chapter meetings.  Interested individuals who see the flyer will telephone 
our co-investigator Phyllis Foxworth at DBSA in Chicago.  Ms. Foxworth will conduct an initial screening for 
eligibility, focused on insurance coverage type, English competence, and achieving a rough US geographic 
balance. Ms. Foxworth will transmit contact details of individual interested in participating to the DPM/HPHCI 
team. We anticipate identifying ~100 candidate respondents at DBSA, to allow for drop-out due to additional 
screening, scheduling conflicts, non-consent, etc. In order to achieve full saturation of themes, we anticipate 
needing 40 completed interviews, as described below.  We will schedule more interviews (~60 in total) in order 
to pilot test our instrument and to account for partial or non-informative interviews.  
 All interview respondents will be English-speaking adults. We plan to stratify the sample by patients with 
bipolar disorder and family members who may be able to provide perspectives about patients whose illness is 
more severe. In all cases, the patients themselves will be insured through an employer’s commercial plan. 
Patients with Medicare, Medicaid, military, or self-purchased coverage will be excluded. We will also stratify the 
sample along two other dimensions to facilitate 
contrasts of interest [RQ-4; RQ-5]: (1) low/no 
deductible insurance plan vs high deductible plan; 
(2) no major illness other than bipolar vs another 
major condition requiring continuous treatment. We 
initially plan to sample 5 individuals per subgroup, 
resulting in 20 individuals per stratum for contrast 
analyses. We expect that this sample will allow us 
to achieve saturation of response theme in each of the three sample strata. We will seek diverse geographic 
residence, age, sex, and race/ethnicity, but not compare across these characteristics.  
The interview domains [RQ-6; PC-3] will include accessing treatment for bipolar disorder and comorbidities, 
affordability of treatment, how patients learn about their insurance coverage, the types of care patients 
consider "high-value" (i.e. most important for their health and well-being), and making choices about care in the 
context of cost-sharing requirements.  
C.12  Analysis Approach for Aim 4 [IR-3]: The semi-structured interview instruments were developed with 
inputs from our Advisory Panel. The domains of inquiry have arisen from discussions during protocol 
development and from previous research on treatment for bipolar disorder. These domains represent 
questions not easily answered with claims data, such as patient learning and coping around insurance benefits, 
affordability of care, and making choices among services (see Appendix). Dr. Madden, senior co-Investigator 
Dr. Ross-Degnan, and Ms. Foxworth will lead and guide this activity.  
The interviews will be conducted by an experienced, well-trained research assistant interviewer based at 
the Department of Population Medicine at the Harvard Pilgrim Health Care Institute/Harvard Medical School.  
The interviewer will be equipped with a digital voice recorder (Olympus WS-852, with TP-8 telephone pick-up 
microphone).  Interviews will be scheduled at the convenience of both the interviewer and the interviewee, and 
the interviewer will conduct these from a quiet, private space at either the DPM office or another location, using 
either a DPM phone or a personal phone.  The voice recorder device will have been password-encrypted due 
to prior connection to the HPHCI computer network.  The brief written notes (e.g., noting date, time, general 
impressions of the interview, any methods concerns) from the interviewer and MP3 audio filenames will have 
only study IDs and no actual personal identifiers.  The interviewer will endeavor to avoid introduction of proper 
 
Patients 
Family members 
 
Low 
deductible 
insurance 
High 
deductible 
insurance 
Low 
deductible 
insurance 
High 
deductible 
insurance 
Bipolar illness 
only 
5 
5 
5 
5 
Other major 
comorbidities 
5 
5 
5 
5 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
17 
names of individuals or places into the interviews.  Written notes will be stored securely as soon as possible at 
HPHCI, where they will also be entered electronically and stored in secure data files on a restricted server 
directory.  MP3s will be uploaded from the recorder to a restricted server directory for storage.  MP3s will also 
be copied onto encrypted USB drives and sent by FedEx (or hand delivery) to a professional transcription 
service in downtown Boston that has been used in many prior department studies (Audio Transcription Center).  
The transcription service will transcribe the MP3 interviews verbatim, except that the service has agreed to 
replace all proper names with, e.g., “[NAME]”, thereby creating written transcripts in MSWord format, 
containing only study IDs as identifiers.  These MSWord files will be returned on the same encrypted USB 
drives to HPHCI via FedEx or hand delivery, and uploaded to the restricted server directory. 
We will code transcribed interview data by reviewing each transcript for responses in a range of conceptual 
categories, using the structure of our interview guide to define initial categories and domains. We will employ 
both deductive codes (themes identified a priori) and inductive codes (additional themes that emerge from the 
data). The study team will conduct a coding validation exercise on an initial transcript. The lead  investigator 
(Dr. Madden) and the research assistant interviewer will then independently review 2 transcripts and create 
additional codes as necessary for the emerging themes. All study investigators and the Advisory Panel will 
review and revise the initial coding scheme. We will develop a codebook with a detailed description of each 
code, inclusion and exclusion criteria, and examples. We will then code the remaining transcripts, iteratively 
refining the coding scheme as necessary until all transcripts are coded and no new or important themes 
emerge.  
We will apply thematic analysis to summarize the interview material using standard qualitative techniques 
and a grounded theory approach.135-137 Recurrence, similarities, and differences will be noted across 
transcripts. Using the coded data, we will describe each theme in detail, noting strength and predominance of 
opinions, patterns and linkages between themes, as well as similarities and differences across the subgroups. 
We will seek to detect divergent views among participants and contrast observations between respondent 
categories. Data coding and the analytic process will be conducted using QDA Miner Lite software.138 
C.11  Potential Generalizability and Limitations: This research has several possible limitations.  
Differential dropout: Because vulnerable populations such as those with low SES might choose to forego 
employer health insurance coverage when offered only an HDHP, we will analyze whether differential dropout 
by group occurred after employer-mandated coverage changes. We will minimize its potential impact on our 
analyses by performing individual-level propensity matching based on multiple employee-level factors including 
SES to develop tightly-matched control groups. We will restrict our follow-up duration to 12 months because 
differential dropout from HDHPs could become more pronounced after the first enrollment year. We expect that 
we will be able to detect even small changes in utilization soon after the HDHP switch among our very large 
sample (Section C.10), so that 12 months should be sufficient follow-up to generate policy-relevant insights.  
Financial incentives: We will also be unable to track amounts in Health Savings or Health Reimbursement 
Accounts because these data are only available from employers or financial institutions. This could lead to 
uncertainty regarding members’ financial incentives or disincentives to seek care. However, the presence of 
these accounts would bias results toward the null, so that any differences detected are likely to be robust. 
Identification of clinical cohort: We will use claims data to identify bipolar disorder. Structured clinical 
interviews are the gold standard for identifying bipolar illness with high accuracy and inter-rater reliability. 
Enrollees misdiagnosed as not having bipolar disorder (i.e., false negatives) will not be in our cohort and we 
will be unable to comment on the impact of HDHPs in this group. However, several studies have demonstrated 
that administrative data can be highly accurate in establishing bipolar disorder for quality assessment, finding 
that among those identified as having bipolar disorder, 94% are confirmed by chart review.66,92,93 
Generalizability: Our inclusion criteria require traditional plan membership for at least a year at baseline, 
making our results applicable largely to health plan members familiar with the general structure of traditional 
insurance benefits. This limits generalizability of our results to populations such as previously uninsured low 
income employees who enter HDHPs. However, given the substantial percentage of employers that are 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
18 
considering introducing HDHPs,22 our results will be highly relevant to a major segment of the private health 
insurance market. Our findings may also be generalizable to traditional plan members switching to HDHPs 
offered in the state health insurance exchanges mandated by the Affordable Care Act. 
D.  PROJECT MILESTONES AND TIMELINE 
This project will take 36 months to complete. We will obtain Institutional Review Board approval prior to 
conducting the studies. We arrange for re-licensing the insurance claims datasets, all years of which by the 
time of award will be available in-house. Below we list specific milestones by 6-month study period: 
Months 
1-6 
Refine study protocols with input from Advisory Panel and DBSA community feedback activities.  
(Review and refinement as needed of the protocols is considered necessary throughout the study.  
Any substantive revisions to the protocols must be submitted to the IRB for approval before action.) 
Aims 1-3.  Identify intervention and comparison cohorts. Identify insured members with diagnosed 
bipolar disorder who match inclusion criteria. Create employer-level and member-level covariates. 
Extract pharmacy, outpatient, and inpatient claims. Develop outcome measures of access to 
appropriate outpatient treatment, adverse patient health outcomes, and patient out of cost burden.  
Aim 4.  Develop consensus around initial interview guide. Preparation and training for recruitment of 
interview respondents and on interview procedures.  
Months 
7-12 
Additional refinement of protocols with Advisory Panel input and DBSA community feedback. 
Aims 1-3.  Data cleaning, validation, and error checking by characterizing 2004-2014 bipolar 
prevalence and outcome trends in members with bipolar disorder. Create study design by stratifying 
HDHP and control employers according to propensity to switch to an HDHP. Member-level 
propensity match of HDHP members and contemporaneous control members with bipolar disorder. 
Compare baseline characteristics of the 3 study groups and against broader population of insured 
members with bipolar illness. Preliminary analyses of changes in access to appropriate outpatient 
treatment (Aim 1) following switching to HDHPs. Stratify analyses contrasting HDHP members with 
and without full drug cost-sharing. Early dissemination of results through conference abstract 
submissions, additional stakeholder meetings, DBSA avenues of communication. 
Aim 4.  Pilot test and refine interview procedures and guide. Recruit 75% of interview respondents. 
Complete 50% of interviews. Ongoing transcription and create coding scheme for interview analysis. 
Months 
13-18 
Additional refinement of protocols with Advisory Panel input and DBSA community feedback. 
Aims 1-3.  Finalize Aim 1 main analyses and draft manuscripts 1 and 2 reporting changes in 
access to appropriate outpatient treatment and quality of care. Additional dissemination activities. 
Preliminary Aim 2 analyses of changes in adverse patient outcomes (psychiatric/non-psychiatric ED 
visits and hospitalizations) following switching to HDHPs. Contrast HDHP members with and without 
full drug cost-sharing. Early dissemination of Aim 2 results through conference abstract submissions, 
additional stakeholder meetings, and DBSA avenues of communication. 
Aim 4.  Refine coding for interview analysis. Completion of remaining interviews, attaining theme 
saturation. Ongoing transcription and analysis of interviews. Early dissemination of interview findings 
through conference submissions, stakeholder meetings, and DBSA avenues of communication. 
Months 
19-24 
Additional refinement of protocols with Advisory Panel input and DBSA community feedback. 
Aims 1-3.  Finalize Aim 2 main analyses and draft manuscript 3 reporting changes in adverse 
patient outcomes (psychiatric/non-psychiatric ED visits and hospitalizations) following switching to 
HDHPs. Additional dissemination activities from Aim 2. Preliminary Aim 3 analyses of changes in 
patient out-of-cost burden. Stratify analyses contrasting HDHP members with and without full drug 
cost-sharing. Early dissemination of Aim 3 results through conference abstract submissions, 
additional stakeholder meetings, DBSA avenues of communication. 
Aim 4.  Finalize analyses of interviews and draft manuscript 4 on patient perspectives on access to 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
19 
appropriate treatment of bipolar in the context of commercial plans. Dissemination of results.  
Months 
25-30 
Additional refinement of protocols with Advisory Panel input and DBSA community feedback. 
Aims 1-3.  Finalize Aim 3 main analyses and draft manuscript 5 reporting changes in patient out-of-
cost burden following switching to HDHPs. Additional Aim 3 dissemination activities. Conduct 
stratified analyses for Aims 1-3 by vulnerable subgroups (race/ethnicity, SES, rural residence, 
comorbidity burden). Early dissemination of results of stratified analyses through conference 
abstract submissions, additional stakeholder meetings, DBSA avenues of communication. 
Aim 1-4.  Develop framework for manuscript 6 on our study’s experiences in patient engagement. 
Months 
31-36 
Ongoing inputs from Advisory Panel and DBSA community feedback, including recommendations 
for future policy and research. Prepare and submit final report; final dissemination activities.  
Aims 1-3.  Finalize analyses of disparities in outcomes (Aims 1-3) following switching to HDHPs 
among particularly vulnerable subgroups (race/ethnicity, SES, rural residence, comorbidity burden). 
Draft manuscript 7 on disparities in outcomes; additional dissemination activities.  
Aim 1-4.  Draft manuscript 6 on our study’s experiences in patient engagement.  
E.  PATIENT POPULATION [RQ-3; PC-2] 
E.1  Populations Affected:  Aims 1-3 analyze retrospective de-identified data from 2004-2014 insurance 
claims. The characteristics and numbers of insurance plan members whose claims data will be analyzed in 
Aims 1-3 are described in Sections C.2, C.5, and C.10. We seek to determine the impacts of switching to 
HDHPs within the insured population that yielded these claims. This population is very minimally affected by 
the research itself because we are observing events that took place in the recent past, and the data have been 
de-identified.  However, people living with bipolar illness and their families have significant potential to benefit 
when our study findings are disseminated, in terms of better public understanding of the challenges these 
patients face, and improvements in health system policies. 
In Aim 4, interview respondents will voluntarily share their experiences on treatment of bipolar disorder in 
the context of commercial insurance coverage. Following DBSA recruitment efforts, we will contact ~100 
individuals to potentially participate in the study, and we plan to interview up to 60 patients or family members 
representing a range of geographic locations, both genders, patients aged 12-64, different race/ethnicities, and 
levels of socioeconomic status. Through the interview process, respondents may develop deeper awareness of 
the attendant issues of insurance coverage and treatment choices, or they may experience mild emotional 
distress discussing sensitive topics, but we expect no significant impacts on this population (see Protection of 
Human Subjects).  Our analyses will emphasize answers to the following types of questions: “Does your 
insurance help you pay for therapist visits?”, “How did you learn about these aspects of your coverage?”, and 
“When you have to make difficult choices, what do you prioritize?” 
Patient partners in our study will participate as equals on our team and, hopefully, derive net benefits in 
terms of collegiality, learning, and compensation. The broader population of the DBSA community (i.e., users 
of DBSA social media, etc.) and patients and stakeholders in the public at large will be able to learn about 
study results and provide feedback (see Section G), but these would not differ substantially from the effects of 
existing DBSA outreach activities and routine publication of research.  
F.  RESEARCH TEAM AND ENVIRONMENT  
Our proposed study will assess how patients with bipolar disorder fare when required to pay much higher out-
of-pocket costs under HDHPs. It will be conducted primarily at the Department of Population Medicine (DPM), 
which resides within the Harvard Pilgrim Health Care Institute (HPHCI), and is a department of Harvard 
Medical School.  The study capitalizes on DPM’s expertise using administrative data and strong quasi-
experimental study designs to evaluate the impacts of cost-containment and coverage policies on medical and 
psychiatric outcomes. We have conducted both early regional and ongoing large national HDHP studies 
analyzing health care utilization, costs, and outcomes in general health populations and populations defined by 
somatic conditions. Our experience with rigorous designs and innovative methods will produce reliable, 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
20 
generalizable results invaluable for policymakers in designing insurance benefits that improve the health of 
patients with bipolar disorder. 
F.1  Research Team Capability to Accomplish The Goals of The Proposed Research: The study 
personnel are ideally situated to address the study goals, and include nationally recognized leaders in the 
realms of mental health services research, HDHP health impacts, and quasi-experimental designs. J. Franklin 
Wharam, MB, BCh, BAO, MPH, Co-Principal Investigator is a general internist and Assistant Professor at 
DPM. Dr. Wharam is a leading national expert in the designs and outcomes of HDHPs, has published multiple 
studies reporting effects of HDHPs on appropriate health care, and leads four R01-level projects examining a 
national HDHP. He also leads four projects assessing the impact of health policies on patients living with 
mental illness or substance use disorders. Christine Y. Lu, MSc, PhD, Co-Principal Investigator is a 
pharmacoepidemiologist, clinical pharmacologist, health services researcher, and Instructor at DPM, with 
expertise in mental health research, health policy analysis and evaluation, disparities, quasi-experimental 
research designs, longitudinal data analysis, and qualitative research. She is a Co-Investigator of four related 
studies examining HDHP impacts. Jeanne Madden, PhD, Co-Investigator is an Instructor with extensive 
experience in the use of claims and survey data to address financial access to medications and other health 
services, in directing large projects, and in measurement and validation. Alisa Busch, MD, MS, Co-
Investigator/Subcontract, is a psychiatrist, Assistant Professor of Psychiatry and HealthCare Policy, and 
hospital-based clinical leader and administrator in psychiatry. She is a national expert in investigations of 
severe mental illness using insurance claims data. Stephen Soumerai, ScD, Co-Investigator is a Professor 
with decades of experience in mental health research and policy evaluation, well known globally for his work 
on coverage policies, quality of care, and patient outcomes. Dennis Ross-Degnan, ScD, Co-Investigator is 
an Associate Professor, health services researcher, and methodologist who is highly recognized for studies of 
the impacts of insurance benefit changes, pioneering longitudinal evaluation methods, and conducting surveys 
in many traditional and non-traditional settings. Fang Zhang, PhD, Biostatistician and Co-Investigator is an 
Assistant Professor and the nation’s expert in constructing observational research designs to study HDHP 
impacts. He was among the first to construct interrupted times series designs matched on the functional form 
of the baseline outcome, closely approximating randomized controlled trial results. Phyllis Foxworth, BS, Co-
Investigator/Subcontract, is an organizational leader and public speaker who works with patients and 
caregivers throughout the country, helping people meet the multitude of personal and system-related 
challenges of living with bipolar disorder and building local capacities for grassroots advocacy. 
F.2   Appropriateness of the Research Environment and Study Sites [PC-4]: Our study team has 
extensive connections to public and private policymakers (through, e.g., HPHC, HMORN, MHRN, 
Optum/United, Blue Cross, CMS, NIH, AHRQ, professional associations) and a history of major successes 
translating research findings into policy advances. The DPM research environment has a rich structural and 
intellectual capacity for research (see People and Places Template) and it has proven conducive not only to 
carrying out groundbreaking cost containment and HDHP studies, but also to informing policymakers and 
affecting policy change. Dr. Soumerai has frequently advised state and federal government entities. Prior DPM 
studies have helped shape Medicaid and Medicare Part D drug coverage policies regarding extra subsidies for 
vulnerable groups, closing the Part D coverage gap, and coverage of specific medication classes. Dr. 
Wharam's studies demonstrating that HDHPs reduce colonoscopy use were followed by an HPHC decision to 
exempt colonoscopies from deductibles. If we were to detect, for example, that more generous drug coverage 
within HDHPs protects patients from adverse HDHP outcomes, our research team is ideally situated to inform 
policymakers for appropriate reforms to optimize patient outcomes. 
The Depression and Bipolar Support Alliance (DBSA), our patient partner, is the leading grassroots 
organization advocating for people living with bipolar disorder, with rich prior research collaboration experience 
and a tremendous, multi-channel capacity to interact with the types of patients who are the focus of this study. 
Dr. Busch is a practicing psychiatrist and lead researcher at McLean Hospital, the largest psychiatric affiliate of 
Harvard Medical School and the premier US facility dedicated to improving the lives of persons with psychiatric 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
21 
illness. She will provide not only expertise in clinical psychiatry and health services research, but also access 
to high-level local and national leaders in mental illness policymaking.  
Our psychiatrist consultants are superb, long-time colleagues who are optimally positioned to bring 
additional vital perspectives to our work as it progresses and then turn our results into action. Dr. Simon is the 
director of the Mental Health Research Network and works closely with NIMH and 13 health system research 
centers on practical, feasible ways to improve mental health care in the US. Dr. Duckworth is the Medical 
Director for the National Alliance on Mental Illness, the other leading advocacy organization concerned with 
mental illness issues in the US. He is also an Associate Medical Director for Behavioral Health at a major 
health insurer and many years of leadership experience in public agencies and professional organizations.  
F.3  Our Research Populations Represent the Real World of Patients Living with Bipolar Disorder [PC-
3]: We selected the claims data resource for our HDHP studies because the major national insurer from which 
the claims derive was the first to promote HDHPs, and is now the largest seller of HDHPs and HSA-associated 
plans. Our data include about 25% of all HDHP members in the US, allowing us to examine rare outcomes, 
and facilitating the use of rigorous longitudinal designs. The demographics are nearly identical to those of 
privately insured respondents in the American Community Survey,139 on characteristics such as age and 
poverty level. The South and Midwest are somewhat over-represented, but zipcode information allows us to 
adjust for geographic influences. We have identified well over 100,000 individuals aged 12-64 and diagnosed 
with bipolar disorder who match our strict enrollment criteria in this large dataset. 
A major study advantage is that the experiment we will assess has already happened in the real world (and 
is continuing to occur). Over the last decade, many employers have required all employees to join HDHPs, 
generating a rigorous "natural experiment" with workers exogenously selected into different plan types. 
Moreover, we can contrast two built-in “interventions" because there is more generous drug coverage in one 
subset of HDHPs.  
Our qualitative study will gather perspectives from patients with bipolar disorder coping with varied 
insurance benefits, in the present day and across the US. Findings from our in-depth telephone interviews will 
round out the results from our claims analyses, providing stories of what choices individuals make, and why, 
when faced with cost-sharing requirements. Our patient representatives, including Ms. Foxworth and the two 
advisory panel members she identified, will bring these perspectives to our study on a deeper and longer-term 
basis; they will be fully familiar with our goals, methods, and progress, and provide their insights on specific 
study issues as they arise. Finally, we will be reaching out, through DBSA and our series of broader patient 
engagement activities, to thousands more patients and caregivers who are, unequivocably, the real world of 
people experiencing bipolar disorder and experiencing the modern US health care system.   
G.  ENGAGEMENT PLAN [RQ-3; PC-1; PC-2] 
G.1.  Planning the Study: The research team is fortunate to have had key stakeholders in bipolar illness 
involved throughout the development of the proposed research. We summarize our engagement to date and 
our post-award plans. Background. Senior team members (Drs. Soumerai, Ross-Degnan, and Simon) have 
long histories researching patient access and barriers to appropriate mental illness treatment. Drs. Soumerai 
and Ross-Degnan’s well-known studies of the effects of state Medicaid payment policies on medication and 
service use among vulnerable beneficiaries with mental illness36-38,41,44,140 were conducted with feedback from 
local patient advocacy groups in New Hampshire and experts in community psychiatry. The National Alliance 
on Mental Illness has used these studies in advocating for broader access to effective medications and in 
testimony before state and national legislatures. Drs. Lu, Madden, and Soumerai are affiliated with the NIMH-
funded Mental Health Research Network, led by Dr. Simon,66,141,142 which was established to advance mental 
health research in health plans.143,144 Drs. Wharam, Ross-Degnan, and Soumerai have spearheaded a series 
of groundbreaking studies on the impact of HDHPs.32,54,55,96,145 With this background, we began planning a 
national study of the effects of HDHPs on patients with bipolar disorder that would incorporate patient views in 
a more meaningful way than previous research.43-45 
Development.  At the suggestion of NAMI Medical Director Ken Duckworth (an in-kind consultant on this 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
22 
project), we engaged with DBSA in developing the proposal. During this collaboration, the proposed study has 
grown more ambitious, incorporating DBSA’s excellent ideas. In addition to facilitating access to patient 
respondents for in-depth interviews, co-investigator Phyllis Foxworth, the DBSA Director of Advocacy, and 
patient representatives Kimberly Allen and Kristin Olbertson have strengthened our recruiting strategy and 
suggested better ways to engage patients. They identified a need for Aim 4 interviews to focus on how patients 
adapt to benefits over time and on the complicating factor of provider networks. They suggested that we elicit 
input on study protocols and findings from the larger DBSA community. The patient partners have emphasized 
how much importance they attach to this study. As Ms. Allen stated, “Claims, finance, benefits, payment, and 
misinformation about all of it – this is the single most important item in mental health care!”  
Post-Award.  Although our protocols for Aims 1-3 are rooted in decades of experience in claims-based 
research and methodological choices are restricted by the types of data in claims, our selection of outcomes 
will continue to be informed by patient perspectives on their importance. Patients will also help us identify 
aspects of preliminary findings that merit deeper exploration. These inputs will come from our DBSA co-
Investigator and patient partners, but we will also elicit input from the broader DBSA community on the areas 
covered by the draft interview guide for Aim 4. Some questions may be “tested” for relevance in brief surveys 
done via DBSA’s chapter groups or Facebook page. Also, the iterative nature of in-depth interviews means that 
the direction of inquiry in Aim 4 will be shaped gradually over time by the respondents themselves. 
G.2. Conducting the Study: Our co-investigator Phyllis Foxworth and Patient/Stakeholder Advisory Panel 
members will participate in all stages of the proposed project. Ms. Foxworth will attend monthly and ad hoc 
meetings, reviewing all study protocols and result summaries. She will meet face-to-face in Boston in Year 1 
with Boston-based team members. She will have a central role in Aim 4 and in study-wide patient engagement 
activities (more below.) The Advisory Panel will meet with the full team four times each year by telephone (~90 
minutes) and also be available for modest, intermediate requests for study feedback (~monthly). 
For Aims 1-3, the Advisory Panel will review claims-based measures and help to identify high versus low-
value care. Specifically, our clinicians (Drs. Wharam, Busch, Simon, and Duckworth) will clarify which clinical 
services are supported by evidence and guideline recommendations. However, we will rely on our patient 
partners to help us understand which metrics represent high- or low-value care from a patient perspective, and 
the meaning of different outcomes in terms of impact on daily living.  
In Aim 4, Ms. Foxworth will lead interview respondent recruitment through DBSA. Working with DBSA’s 
Directors of Chapter Relations and Communications, she will train selected chapter leaders in appropriately 
presenting the study opportunity and directing interested individuals to Ms. Foxworth at DBSA in Chicago. Ms. 
Foxworth will act as primary contact, provide additional preliminary information and screening, and pass 
contacts along to Dr. Madden and the interviewer. Drs. Madden and Ross-Degnan, and Ms. Foxworth will 
review analysis plans, coding, themes, and results as the interviews proceed, refining ongoing recruitment 
procedures as needed to ensure sample balance and theme saturation. 
For all Aims, we will request the Panel’s advice on interpreting preliminary findings and framing the results 
in presentations and manuscripts to be meaningful to their audiences. All panel members will have 
opportunities to co-author study papers, if interested. DBSA, MHRN, and NAMI will be actively involved in 
planning and implementing the dissemination of our results. 
Throughout the study, we will engage patients through DBSA chapters, campaigns, and social media 
platforms. With guidance from the Advisory Panel, we will identify aspects of the study that can benefit from 
feedback from the broader DBSA community. In each instance, the team will identify the best vehicle for 
feedback; DBSA already uses multiple outreach tools, including quarterly web surveys, a Facebook page with 
over 100,000 followers, and the Care For Your Mind blog with 5,000 user sessions per month. Other potential 
avenues include the local chapters and a network of 10 regional DBSA-affiliated grassroots advocacy 
organizations. We will also consider whether these channels could be used to gather additional data that might 
supplement or inform our primary research findings. With additional IRB review and approval, we may ask 
community members about their experiences with insurance, access to care, and choices in healthcare.  

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
23 
In addition, we will take advantage of DBSA’s previously-established “Peer Council” to serve as a standing 
community feedback panel for our study.  On a quarterly basis, starting in the 4th quarter of the study, we will 
begin engaging this Peer Council by informing them about our study and requesting that they represent their 
DBSA peer community by commenting on selected study outputs and informational queries.  This standing 
panel, in the form of DBSA’s Peer Council (anticipated N~100), will thus combine longitudinal study 
engagement with broader representation than our more intensive Advisory Panel.  
We will monitor the dynamics of Advisory Panel meetings to ensure that the voices of all participants are 
heard. We will periodically check in with patient representatives and other Advisory Panel members by phone 
or e-mail to assess their experiences in the project and to obtain their suggestions for improving 
patient/stakeholder engagement. Advisory Panel inputs, both scientific and procedural, will be carefully 
documented and a report on patient engagement will be developed toward the end of the study. 
G.3. Disseminating the Study Results [PC-4]: Our Advisory Panel will advise the study team about which 
findings would be of greatest interest to patient communities, how to frame these findings effectively for 
patients, and what channels would be the most effective for reaching individuals living with bipolar illness and 
people involved in their care. DBSA’s wide range of outreach channels (such as the social media platforms, 
chapters, website, and monthly e-Update newsletter) offer promising opportunities for dissemination to patients. 
Due to DBSA’s dedication to the study, we anticipate being able to collect patient reactions to study findings 
and recommendations. We will assemble commentary from the community regarding the salience and 
acceptability of our work, and their ideas for health system change and future inquiries.  
G.4.  Principles for Engagement:  Reciprocal Relationships.  We will foster a mutually beneficial 
collaboration toward shared goals among the team of investigators, consultants, and patient representatives. 
Each patient and stakeholder partner will provide perspective as an individual and a representative of other 
similar stakeholders. Our DBSA co-investigator Ms. Foxworth has been instrumental in shaping our proposal, 
intensifying our respondent recruitment strategy, and encouraging the inclusion of a range of community 
feedback activities. Our psychiatrist co-Investigator (Alisa Busch) is a national leader in mental health services 
research who will oversee clinical inputs to the study. Consultant psychiatrists Greg Simon and Ken Duckworth 
are each national leaders with close affiliations to health systems, academic institutions, and major mental 
health patient advocacy organizations. Kimberly Allen, a patient representative, has professional expertise in 
employee assistance and chemical dependency programs, and holds insurance licensure. Kristin Olbertson, a 
patient family representative, holds doctoral degrees in law and US history. Three additional stakeholders were 
added to the advisory panel in the period between our initial application for funding and the announcement of 
our award; these additions were made in response to PCORI reviewers.  The additional panel members 
provide more voices for health insurers, employers, clinicians, and patients.  Ken Dolan-DelVecchio is Vice 
President for Health and Wellness at Prudential Financial, Inc.  He has 17 years of experience helping to 
ensure that Prudential's employees with mental illnesses receive essential health insurance benefits and 
health services.  Dr. Jim Sabin is Director of the Harvard Pilgrim Health Care Ethics Program, an 
internationally-renowned ethicist, and a psychiatrist.  He has spent many years convening employers and 
health insurance representatives to discuss challenging ethical issues in the health insurance industry, 
including issues around mental health benefits.  He therefore has a deep understanding of the challenges of 
optimizing mental health care from the perspective of both an employer and health insurer.  Francisca Azocar, 
PhD, is Vice President of Research and Evaluation for Behavioral Health Sciences at Optum Behavioral Health, 
the nation's largest behavioral health network that serves to address employee mental health issues.  She has 
extensive experience working with academic institutions to conduct research that advances care for patients 
with severe mental illness.  We will acknowledge the contributions of our Advisory Panel in all manuscripts and 
also offer the opportunity for co-authorship. 
Co-Learning.  Advisory Panel meetings will represent a forum for co-learning. Study investigators will learn 
patient and stakeholder perspectives, while patient representatives will learn about the research process. 
Individual check-in sessions afterward will offer opportunities for patient representatives to further educate the 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
24 
team about their perspective and to clarify questions. Selected DBSA chapter leaders will participate in Aim 4 
recruitment. Ms. Foxworth will directly participate in analyses of interviews, lending her insights as a patient 
representative and organizer. She will educate the Boston-based team on DBSA’s use of community outreach 
tools, how messages are selected and shaped, and pitfalls to avoid, and she will inform the team on how the 
issues raised in our research resonate with DBSA’s grassroots activities. Our Aim 4 interview respondents and 
community members (through feedback activities) will help to fill information gaps inherent in claims data and 
help us to infuse our findings with patient perspectives.  
Partnership.  In engaging the Advisory Panel as partners, the research team will balance the desire for 
more input with the need to be respectful of advisor’s jobs, families, and other commitments. Phyllis Foxworth 
will be a co-Investigator at 7% effort per year. Patient advisors will receive a $500 honorarium per year to 
compensate them for four 90-minute Advisory panel meetings, pre-meeting review of materials, after-meeting 
check-ins, and ad hoc communications. One psychiatrist consultant will receive up to $3000 per year for 3 
days of advice including his participation on our panel, while the other will be available for the panel meetings 
and additional ad hoc advice on an in-kind basis. Advisory Panel meetings will be conducted by telephone 
conference (due to our geographic diversity) and scheduled far in advance at members’ convenience. 
Trust, Transparency, Honesty.  We are privileged to partner with the diverse and highly experienced 
Advisory Panel members. Their involvement represents a commitment to a relationship based on mutual 
respect for the contributions of the entire study team. Open and honest communication will be crucial in 
facilitating effective collaboration. All members of our study team are accustomed to adhering to these 
principles within their own organizations and in past collaborations. Decisions about the study will be made by 
consensus with input from all partners; major study decisions will be discussed at Advisory Panel meetings and 
Advisory Panel members will be apprised of key decisions made outside of meetings. We will share study 
findings with DBSA, MHRN, NAMI, and other stakeholders, obtaining their inputs, while continuing the tradition 
of academic freedom whereby investigators can communicate unbiased findings and publish without censoring.

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
25 
 
 
REFERENCES CITED  
For detailed instructions, refer to the Application Guidelines for your PFA. Do not exceed 10 pages. 
 
Following scholarly citation practice, list the source material cited in this Research Plan.  
 
REFERENCES 
 
1. 
The Kaiser Family Foundation and Health Research & Educational Trust. Employer Health Benefits 
2014 Annual Survery: The Henry J. Kaiser Family Foundation;2014. 
2. 
World Health Organization. The Global Burden of Diseases: 2004 Update. Switzerland: Department of 
Health Statistics and Informatics;2008. 
3. 
Merikangas KR, Akiskal HS, Angst J, Greenberg PE, Hirschfeld RM, Petukhova M, Kessler RC. 
Lifetime and 12-month prevalence of bipolar spectrum disorder in the National Comorbidity Survey 
replication. Arch Gen Psychiatry. May 2007;64(5):543-552. 
4. 
Marangell LB. The importance of subsyndromal symptoms in bipolar disorder. J Clin Psychiatry. 
2004;65 Suppl 10:24-27. 
5. 
Hawton K, Sutton L, Haw C, Sinclair J, Harriss L. Suicide and attempted suicide in bipolar disorder: a 
systematic review of risk factors. J Clin Psychiatry. Jun 2005;66(6):693-704. 
6. 
Kupfer DJ. The increasing medical burden in bipolar disorder. JAMA. May 25 2005;293(20):2528-2530. 
7. 
Dean BB, Gerner D, Gerner RH. A systematic review evaluating health-related quality of life, work 
impairment, and healthcare costs and utilization in bipolar disorder. Curr Med Res Opin. 
2004;20(2):139-154. 
8. 
Colom F, Vieta E, Martinez-Aran A, Reinares M, Goikolea JM, Benabarre A, Torrent C, Comes M, 
Corbella B, Parramon G, Corominas J. A randomized trial on the efficacy of group psychoeducation in 
the prophylaxis of recurrences in bipolar patients whose disease is in remission. Arch Gen Psychiatry. 
Apr 2003;60(4):402-407. 
9. 
Lam DH, Hayward P, Watkins ER, Wright K, Sham P. Relapse prevention in patients with bipolar 
disorder: cognitive therapy outcome after 2 years. Am J Psychiatry. Feb 2005;162(2):324-329. 
10. 
Miklowitz DJ, George EL, Richards JA, Simoneau TL, Suddath RL. A randomized study of family-
focused psychoeducation and pharmacotherapy in the outpatient management of bipolar disorder. Arch 
Gen Psychiatry. Sep 2003;60(9):904-912. 
11. 
Perry A, Tarrier N, Morriss R, McCarthy E, Limb K. Randomised controlled trial of efficacy of teaching 
patients with bipolar disorder to identify early symptoms of relapse and obtain treatment. BMJ. Jan 16 
1999;318(7177):149-153. 
12. 
Rea MM, Tompson MC, Miklowitz DJ, Goldstein MJ, Hwang S, Mintz J. Family-focused treatment 
versus individual treatment for bipolar disorder: results of a randomized clinical trial. J Consult Clin 
Psychol. Jun 2003;71(3):482-492. 
13. 
Simon GE, Ludman EJ, Unutzer J, Bauer MS, Operskalski B, Rutter C. Randomized trial of a 
population-based care program for people with bipolar disorder. Psychol Med. Jan 2005;35(1):13-24. 
14. 
American Psychiatric Association. Practice guideline for the treatment of patients with bipolar disorder 
(revision). Am J Psychiatry. Apr 2002;159(4 Suppl):1-50. 
15. 
Glick ID, Suppes T, DeBattista C, Hu RJ, Marder S. Psychopharmacologic treatment strategies for 
depression, bipolar disorder, and schizophrenia. Ann Intern Med. Jan 2 2001;134(1):47-60. 
16. 
Perlis RH, Keck PE. The Texas implementation of medication algorithms update for treatment of bipolar 
I disorder. J Clin Psychiatry. Jul 2005;66(7):818-820. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
26 
17. 
Bauer MS, Callahan AM, Jampala C, Petty F, Sajatovic M, Schaefer V, Wittlin B, Powell BJ. Clinical 
practice guidelines for bipolar disorder from the Department of Veterans Affairs. J Clin Psychiatry. Jan 
1999;60(1):9-21. 
18. 
Judd LL, Akiskal HS, Schettler PJ, Endicott J, Maser J, Solomon DA, Leon AC, Rice JA, Keller MB. The 
long-term natural history of the weekly symptomatic status of bipolar I disorder. Arch Gen Psychiatry. 
Jun 2002;59(6):530-537. 
19. 
Judd LL, Akiskal HS, Schettler PJ, Coryell W, Endicott J, Maser JD, Solomon DA, Leon AC, Keller MB. 
A prospective investigation of the natural history of the long-term weekly symptomatic status of bipolar 
II disorder. Arch Gen Psychiatry. Mar 2003;60(3):261-269. 
20. 
Department of Health and Human Services. Effect of Health Care Cost Growth on the U.S. Economy. 
2008. http://aspe.hhs.gov/health/reports/08/healthcarecost/report.pdf. Accessed 26 October 2014. 
21. 
Yoo H. January 2010 Census Shows 10 Million People Covered by HSA/High-deductible Health Plans. 
2010; http://www.ahipresearch.org/pdfs/HSA2010.pdf. Accessed Sep 29, 2010. 
22. 
Kaiser Family Foundation. The Kaiser Family Foundation and Health Research and Educational Trust 
Employer health benefits 2010 annual survey. 2010; http://ehbs.kff.org/pdf/2010/8085.pdf. Accessed 
Sep 27, 2010. 
23. 
Internal Revenue Service. Health Savings Accounts and Other Tax-Favored Health Plans. Publication 
969.  http://www.irs.gov/pub/irs-pdf/p969.pdf. Accessed Apr 15, 2013. 
24. 
USDOT. All about HSAs. 2007; http://www.ustreas.gov/offices/public-affairs/hsa/pdf/all-about-
HSAs_051807.pdf. Accessed 24 March, 2008. 
25. 
Fendrick AM, Chernew ME. Value-based insurance design: a "clinically sensitive" approach to preserve 
quality of care and contain costs. American Journal of Managed Care. Jan 2006;12(1):18-20. 
26. 
Buntin MB, Damberg C, Haviland A, Kapur K, Lurie N, McDevitt R, Marquis MS. Consumer-directed 
health care: early evidence about effects on cost and quality. Health Aff (Millwood). Nov-Dec 
2006;25(6):w516-530. 
27. 
Waters TM, Chang CF, Cecil WT, Kasteridis P, Mirvis D. Impact of high-deductible health plans on 
health care utilization and costs. Health Serv Res. Feb 2011;46(1 Pt 1):155-172. 
28. 
Parente ST, Feldman R, Christianson JB. Evaluation of the effect of a consumer-driven health plan on 
medical care expenditures and utilization. Health Serv Res. Aug 2004;39(4 Pt 2):1189-1210. 
29. 
Dixon A, Greene J, Hibbard J. Do consumer-directed health plans drive change in enrollees' health 
care behavior? Health Aff (Millwood). Jul-Aug 2008;27(4):1120-1131. 
30. 
Kaiser Family Foundation. The Kaiser Family Foundation and Health Research & Educational Trust 
2013 Employer Health Benefits Annual Survey. 2013; http://ehbs.kff.org/pdf/2011/8225.pdf. Accessed 
Sep 27, 2013. 
31. 
Fronstin P. Employers May Move By 2018 To Avoid Tax. 2010; 
http://healthcare.nationaljournal.com/2010/05/a-future-for-consumerdirected.php. Accessed Aug 12, 
2010. 
32. 
Kozhimannil KB, Huskamp HA, Graves AJ, Soumerai SB, Ross-Degnan D, Wharam JF. High-
deductible health plans and costs and utilization of maternity care. Am J Manag Care. Jan 
2011;17(1):e17-25. 
33. 
Hodgkin D, Horgan CM, Garnick DW, Len Merrick E, Volpe-Vartanian J. Management of access to 
branded psychotropic medications in private health plans. Clin Ther. Feb 2007;29(2):371-380. 
34. 
Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. Aug 4 2005;353(5):487-497. 
35. 
Briesacher BA, Gurwitz JH, Soumerai SB. Patients at-risk for cost-related medication nonadherence: a 
review of the literature. J Gen Intern Med. Jun 2007;22(6):864-871. 
36. 
Soumerai SB, McLaughlin TJ, Ross-Degnan D, Casteris CS, Bollini P. Effects of Limiting Medicaid 
Drug-Reimbursement Benefits on the Use of Psychotropic Agents and Acute Mental Health Services by 
Patients with Schizophrenia. N Engl J Med. 1994;331(10):650-655. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
27 
37. 
Soumerai SB, Ross-Degnan D, Avorn J, McLaughlin T, Choodnovskiy I. Effects of Medicaid drug-
payment limits on admission to hospitals and nursing homes. N Engl J Med. Oct 10 1991;325(15):1072-
1077. 
38. 
Soumerai SB, Avorn J, Ross-Degnan D, Gortmaker S. Payment restrictions for prescription drugs 
under Medicaid. Effects on therapy, cost, and equity. N Engl J Med. Aug 27 1987;317(9):550-556. 
39. 
Zeber JE, Grazier KL, Valenstein M, Blow FC, Lantz PM. Effect of a medication copayment increase in 
veterans with schizophrenia. Am J Manag Care. Jun 2007;13(6 Pt 2):335-346. 
40. 
Gibson TB, Jing Y, Kim E, Bagalman E, Wang S, Whitehead R, Tran QV, Doshi JA. Cost-sharing 
effects on adherence and persistence for second-generation antipsychotics in commercially insured 
patients. Manag Care. Aug 2010;19(8):40-47. 
41. 
Soumerai SB, Zhang F, Ross-Degnan D, Ball DE, LeCates RF, Law MR, Hughes TE, Chapman D, 
Adams AS. Use of atypical antipsychotic drugs for schizophrenia in Maine Medicaid following a policy 
change. Health Aff. 2008;27(3):w185. 
42. 
Adams AS, Zhang F, LeCates RF, Graves AJ, Ross-Degnan D, Gilden D, McLaughlin TJ, Lu C, 
Trinacty CM, Soumerai SB. Prior authorization for antidepressants in Medicaid: effects among disabled 
dual enrollees. Arch Intern Med. Apr 27 2009;169(8):750-756. 
43. 
Zhang Y, Adams AS, Ross-Degnan D, Zhang F, Soumerai SB. Effects of prior authorization on 
medication discontinuation among Medicaid beneficiaries with bipolar disorder. Psychiatr Serv. Apr 
2009;60(4):520-527. 
44. 
Lu CY, Soumerai SB, Ross-Degnan D, Zhang F, Adams AS. Unintended impacts of a Medicaid prior 
authorization policy on access to medications for bipolar illness. Med Care. Jan 2010;48(1):4-9. 
45. 
Lu CY, Adams AS, Ross-Degnan D, Zhang F, Zhang Y, Salzman C, Soumerai SB. Association 
between prior authorization for medications and health service use by medicaid patients with bipolar 
disorder. Psychiatr Serv. Feb 2011;62(2):186-193. 
46. 
Druss BG, Bradford WD, Rosenheck RA, Radford MJ, Krumholz HM. Quality of medical care and 
excess mortality in older patients with mental disorders. Arch Gen Psychiatry. Jun 2001;58(6):565-572. 
47. 
Iacovides A, Siamouli M. Comorbid mental and somatic disorders: an epidemiological perspective. Curr 
Opin Psychiatry. Jul 2008;21(4):417-421. 
48. 
Busch AB, Lehman AF, Goldman H, Frank RG. Changes over time and disparities in schizophrenia 
treatment quality. Med Care. Feb 2009;47(2):199-207. 
49. 
Fleck DE, Hendricks WL, DelBello MP, Strakowski SM. Differential prescription of maintenance 
antipsychotics to African American and white patients with new-onset bipolar disorder. J Clin Psychiatry. 
Aug 2002;63(8):658-664. 
50. 
Depp C, Ojeda VD, Mastin W, Unutzer J, Gilmer TP. Trends in use of antipsychotics and mood 
stabilizers among Medicaid beneficiaries with bipolar disorder, 2001-2004. Psychiatr Serv. Oct 
2008;59(10):1169-1174. 
51. 
Chernew M, Gibson TB, Yu-Isenberg K, Sokol MC, Rosen AB, Fendrick AM. Effects of increased 
patient cost sharing on socioeconomic disparities in health care. J Gen Intern Med. Aug 
2008;23(8):1131-1136. 
52. 
Newhouse JP. Free for all?  Lessons from the RAND Health Insurance Experiment. Cambridge: 
Harvard University Press; 1996. 
53. 
Manning W, Wells KB, Valdez RB, Newhouse JP. Effects of mental health insurance: Evidence from 
the Health Insurance Experiment: The RAND Corporation;1989. 
54. 
Wharam JF, Landon BE, Galbraith AA, Kleinman KP, Soumerai SB, Ross-Degnan D. Emergency 
department use and subsequent hospitalizations among members of a high-deductible health plan. 
JAMA. 2007;297(10):1093-1102. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
28 
55. 
Wharam JF, Galbraith AA, Kleinman KP, Soumerai SB, Ross-Degnan D, Landon BE. Cancer screening 
before and after switching to a high-deductible health plan. Ann Intern Med. May 6 2008;148(9):647-
655. 
56. 
Nair KV, Park J, Wolfe P, Saseen JJ, Allen RR, Ganguly R. Consumer-driven health plans: impact on 
utilization and expenditures for chronic disease sufferers. J Occup Environ Med. May 2009;51(5):594-
603. 
57. 
Chen S, Levin RA, Gartner JA. Medication adherence and enrollment in a consumer-driven health plan. 
Am J Manag Care. Feb 2010;16(2):e43-50. 
58. 
Reiss SK, Ross-Degnan D, Zhang F, Soumerai SB, Zaslavsky AM, Wharam JF. Effect of Switching to a 
High-Deductible Health Plan on Use of Chronic Medications. Health Serv Res. Mar 17 2011. 
59. 
Reed M, Benedetti N, Brand R, Newhouse JP, Hsu J. Perspectives from deductible plan enrollees: plan 
knowledge and anticipated care-seeking changes. BMC Health Serv Res. 2009;9:244. 
60. 
Kullgren JT, Galbraith AA, Hinrichsen VL, Miroshnik I, Penfold RB, Rosenthal MB, Landon BE, Lieu TA. 
Health care use and decision making among lower-income families in high-deductible health plans. 
Arch Intern Med. Nov 22 2010;170(21):1918-1925. 
61. 
Galbraith AA, Ross-Degnan D, Soumerai SB, Rosenthal MB, Gay C, Lieu TA. Nearly half of families in 
high-deductible health plans whose members have chronic conditions face substantial financial burden. 
Health Aff (Millwood). Feb 2011;30(2):322-331. 
62. 
Hall JP, Carroll SL, Moore JM. Health care behaviors and decision-making processes among enrollees 
in a state high-risk insurance pool: focus group findings. Am J Health Promot. May-Jun 2010;24(5):304-
310. 
63. 
Sajatovic M, Jenkins JH, Cassidy KA, Muzina DJ. Medication treatment perceptions, concerns and 
expectations among depressed individuals with Type I Bipolar Disorder. J Affect Disord. Jun 
2009;115(3):360-366. 
64. 
Sajatovic M, Levin J, Fuentes-Casiano E, Cassidy KA, Tatsuoka C, Jenkins JH. Illness experience and 
reasons for nonadherence among individuals with bipolar disorder who are poorly adherent with 
medication. Compr Psychiatry. May-Jun 2011;52(3):280-287. 
65. 
DBSA Consumer and Family Survey Center. 2010 Medications & Perceptions Survey Summary: 
Depression and Bipolar Support Alliance;2011. 
66. 
Simon GE, Unutzer J. Health care utilization and costs among patients treated for bipolar disorder in an 
insured population. Psychiatr Serv. Oct 1999;50(10):1303-1308. 
67. 
Peele PB, Xu Y, Kupfer DJ. Insurance expenditures on bipolar disorder: clinical and parity implications. 
Am J Psychiatry. Jul 2003;160(7):1286-1290. 
68. 
Guo JJ, Keck PE, Jr., Li H, Jang R, Kelton CM. Treatment costs and health care utilization for patients 
with bipolar disorder in a large managed care population. Value Health. May-Jun 2008;11(3):416-423. 
69. 
Greene J, Hibbard J, Murray JF, Teutsch SM, Berger ML. The impact of consumer-directed health 
plans on prescription drug use. Health Aff (Millwood). Jul-Aug 2008;27(4):1111-1119. 
70. 
Wilson AR, Rodin H, Garrett NA, Bargman EP, Harris LA, Pederson MK, Plocher DW. Comparing 
quality of care between a consumer-directed health plan and a traditional plan: an analysis of HEDIS 
measures related to management of chronic diseases. Popul Health Manag. Apr 2009;12(2):61-67. 
71. 
Hardie NA, Lo Sasso AT, Shah M, Levin RA. Behavioral healthcare services use in health savings 
accounts versus traditional health plans. J Ment Health Policy Econ. Dec 2010;13(4):159-165. 
72. 
Crowley MJ, McCrory DC, Chatterjee R, Gierisch JM, Myers ER, Schmit KM, Coeytaux RR, Correll CU, 
Kendrick AS, Sanders GD. Prioritization of research addressing antipsychotics for adolescents and 
young adults with bipolar disorder. Ann Intern Med. Apr 1 2014;160(7):492-498. 
73. 
Patient-Centered Outcomes Research Institute. National Priorities for Research and Research Agenda: 
Patient-Centered Outcomes Research Institute;2012. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
29 
74. 
Committee on Comparative Effectiveness Research Priorization: Board on Health Care Services. Initial 
National Priorities for Comparative Effectiveness Research Washington, D.C.: Institute of Medicine of 
the National Academies;2009. 
75. 
Practice guideline for the treatment of patients with bipolar disorder (revision). Am J Psychiatry. Apr 
2002;159(4 Suppl):1-50. 
76. 
National Alliance on Mental Illness. NAMI 2013 Annual Report: National Alliance on Mental Illness;2013. 
77. 
St. Clair T, Cook TD, Hallberg K. Examining the internal validity and statistical precision of the 
comparative interrupted times series design by comparison with a randomized experiment. American 
Journal of Evaluation. Forthcoming. 
78. 
Briesacher BA, Soumerai SB, Field TS, Fouayzi H, Gurwitz JH. Medicare part D's exclusion of 
benzodiazepines and fracture risk in nursing homes. Arch Intern Med. Apr 26 2010;170(8):693-698. 
79. 
AARP Press Center. AARP Statement on President Obama's Signing of Health Reform Package. 2010; 
http://www-alpha.aarp.org/aarp/presscenter/pressrelease/articles/statement_obama_signing_hcr.html. 
Accessed Apri 07, 2011. 
80. 
Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized 
causal inference. Boston: Houghton Mifflin; 2002. 
81. 
Aaserud M, Austvoll-Dahlgren A, Sturm H, et al. Pharmaceutical policies: effects on rational drug use, 
an overview of 13 reviews. No.: CD004397. Cochrane Database of Systematic Reviews. 2006(2). 
82. 
Liang KY, Zeger SL. Longitudinal Data Analysis Using Generalized Linear Models. Biometrika. 
1986;73(1):13-22. 
83. 
Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 
1986;42(1):121-130. 
84. 
Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted 
time series studies in medication use research. J Clin Pharm Ther. Aug 2002;27(4):299-309. 
85. 
Schreyogg J, Stargardt T, Tiemann O. Costs and quality of hospitals in different health care systems: a 
multi-level approach with propensity score matching. Health economics. Jan 2011;20(1):85-100. 
86. 
Wharam JF, Zhang F, Xu X, Landon BE, Ross-Degnan D. National Trends and Disparities in Cervical 
Cancer Screening among Commercially Insured Women, 2001-2010. Cancer Epidemiology, 
Biomarkers and Prevention. Aug 15 2014. 
87. 
Akazawa M, Halpern R, Riedel AA, Stanford RH, Dalal A, Blanchette CM. Economic burden prior to 
COPD diagnosis: a matched case-control study in the United States. Respir Med. Dec 
2008;102(12):1744-1752. 
88. 
Nordstrom BL, Norman HS, Dube TJ, Wilcox MA, Walker AM. Identification of abacavir hypersensitivity 
reaction in health care claims data. Pharmacoepidemiol Drug Saf. Mar 2007;16(3):289-296. 
89. 
Busch AB, Ling D, Frank RG, Greenfield SF. Changes in the quality of care for bipolar I disorder during 
the 1990s. Psychiatr Serv. Jan 2007;58(1):27-33. 
90. 
Busch AB, Huskamp HA, Landrum MB. Quality of care in a Medicaid population with bipolar I disorder. 
Psychiatr Serv. Jun 2007;58(6):848-854. 
91. 
Busch AB, Frank RG, Sachs G. Bipolar-I depression outpatient treatment quality and costs in usual 
care practice. Psychopharmacol Bull. 2008;41(2):24-39. 
92. 
Unutzer J, Simon G, Pabiniak C, Bond K, Katon W. The treated prevalence of bipolar disorder in a 
large staff-model HMO. Psychiatr Serv. Aug 1998;49(8):1072-1078. 
93. 
Unutzer J, Simon G, Pabiniak C, Bond K, Katon W. The use of administrative data to assess quality of 
care for bipolar disorder in a large staff model HMO. Gen Hosp Psychiatry. Jan-Feb 2000;22(1):1-10. 
94. 
The Johns Hopkins ACG Case-Mix System Reference Manual, Version 7.0. Baltimore, MD: The Johns 
Hopkins University; 2005. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
30 
95. 
Reid RJ, Roos NP, MacWilliam L, Frohlich N, Black C. Assessing population health care need using a 
claims-based ACG morbidity measure: a validation analysis in the Province of Manitoba. Health Serv 
Res. Oct 2002;37(5):1345-1364. 
96. 
Wharam JF, Landon BE, Zhang F, Soumerai SB, Ross-Degnan D. High-deductible insurance: two-year 
emergency department and hospital use. Am J Manag Care. Oct 2011;17(10):e410-418. 
97. 
Wharam JF, Graves AJ, Landon BE, Zhang F, Soumerai SB, Ross-Degnan D. Two-year trends in 
colorectal cancer screening after switch to a high-deductible health plan. Med Care. Sep 
2011;49(9):865-871. 
98. 
Wharam JF, Graves AJ, Zhang F, Soumerai SB, Ross-Degnan D, Landon BE. Two-year trends in 
cancer screening among low socioeconomic status women in an HMO-based high-deductible health 
plan. J Gen Intern Med. Sep 2012;27(9):1112-1119. 
99. 
Wharam JF, Graves AJ, Zhang F, Landon BE, Soumerai SB, Ross-Degnan D. Two-year Trends in 
Colorectal Cancer Screening after Switch to a High-deductible Health Plan. Med Care. 2011;In press. 
100. 
Galbraith AA, Ross-Degnan D, Soumerai SB, Miroshnik I, Wharam JF, Kleinman K, Lieu TA. High-
deductible health plans: are vulnerable families enrolled? Pediatrics. Apr 2009;123(4):e589-594. 
101. 
Cook EF, Goldman L. Performance of tests of significance based on stratification by a multivariate 
confounder score or by a propensity score. J Clin Epidemiol. 1989;42(4):317-324. 
102. 
D'Agostino RB, Jr. Propensity score methods for bias reduction in the comparison of a treatment to a 
non-randomized control group. Stat Med. Oct 15 1998;17(19):2265-2281. 
103. 
Rubin DB. The design versus the analysis of observational studies for causal effects: parallels with the 
design of randomized trials. Stat Med. Jan 15 2007;26(1):20-36. 
104. 
Coca-Perraillon M. Local and Global Optimal Propensity Score Matching. 2007; 
http://www2.sas.com/proceedings/forum2007/185-2007.pdf. Accessed Dec 19, 2009. 
105. 
Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means 
and differences in proportions in observational studies. Pharm Stat. Apr 27 2010. 
106. 
Farley JF. Medicaid prescription cost containment and schizophrenia: a retrospective examination. Med 
Care. May 2010;48(5):440-447. 
107. 
Karve S, Cleves MA, Helm M, Hudson TJ, West DS, Martin BC. Prospective validation of eight different 
adherence measures for use with administrative claims data among patients with schizophrenia. Value 
Health. Sep 2009;12(6):989-995. 
108. 
Kilbourne AM, Post EP, Bauer MS, Zeber JE, Copeland LA, Good CB, Pincus HA. Therapeutic drug 
and cardiovascular disease risk monitoring in patients with bipolar disorder. J Affect Disord. Sep 
2007;102(1-3):145-151. 
109. 
Busch AB, Frank RG, Lehman AF, Greenfield SF. Schizophrenia, co-occurring substance use disorders 
and quality of care: the differential effect of a managed behavioral health care carve-out. Adm Policy 
Ment Health. May 2006;33(3):388-397. 
110. 
Burns ME, Busch AB, Madden JM, Le Cates RF, Zhang F, Adams AS, Ross-Degnan D, Soumerai SB, 
Huskamp HA. Effects of Medicare Part D on guideline-concordant pharmacotherapy for bipolar I 
disorder among dual beneficiaries. Psychiatr Serv. Mar 1 2014;65(3):323-329. 
111. 
Marcus SC, Olfson M, Pincus HA, Zarin DA, Kupfer DJ. Therapeutic drug monitoring of mood 
stabilizers in Medicaid patients with bipolar disorder. Am J Psychiatry. Jul 1999;156(7):1014-1018. 
112. 
Allison DB, Mentore JL, Heo M, Chandler LP, Cappelleri JC, Infante MC, Weiden PJ. Antipsychotic-
induced weight gain: a comprehensive research synthesis. Am J Psychiatry. Nov 1999;156(11):1686-
1696. 
113. 
Newcomer JW. Second-generation (atypical) antipsychotics and metabolic effects: a comprehensive 
literature review. CNS Drugs. 2005;19 Suppl 1:1-93. 
114. 
Consensus development conference on antipsychotic drugs and obesity and diabetes. Diabetes Care. 
Feb 2004;27(2):596-601. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
31 
115. 
U.S.Bureau of Labor Statistics. Consumer Price Index. 2011; http://www.bls.gov/cpi/ Accessed May 25, 
2011. 
116. 
U.S. Bureau of the Census.  Geographical Areas Reference Manual, Washington, D.C., U.S. Bureau of 
the Census. 1994. 
117. 
Fiscella K, Fremont AM. Use of geocoding and surname analysis to estimate race and ethnicity. Health 
Serv Res. Aug 2006;41(4 Pt 1):1482-1500. 
118. 
Stewart SL, Swallen KC, Glaser SL, Horn-Ross PL, West DW. Comparison of methods for classifying 
Hispanic ethnicity in a population-based cancer registry. Am J Epidemiol. Jun 1 1999;149(11):1063-
1071. 
119. 
Swallen KC, Glaser SL, Stewart SL, West DW, Jenkins CN, McPhee SJ. Accuracy of racial 
classification of Vietnamese patients in a population-based cancer registry. Ethn Dis. 1998;8(2):218-
227. 
120. 
Swallen KC, West DW, Stewart SL, Glaser SL, Horn-Ross PL. Predictors of misclassification of 
Hispanic ethnicity in a population-based cancer registry. Ann Epidemiol. Apr 1997;7(3):200-206. 
121. 
Jacobs EA, Lauderdale DS. Receipt of cancer screening procedures among Hispanic and non-Hispanic 
health maintenance organization members. Cancer. Jan 1 2001;91(1 Suppl):257-261. 
122. 
Coronado GD, Koepsell TD, Thompson B, Schwartz SM, Wharton RS, Grossman JE. Assessing 
cervical cancer risk in Hispanics. Cancer Epidemiol Biomarkers Prev. Oct 2002;11(10 Pt 1):979-984. 
123. 
Krieger N, Chen JT, Waterman PD, Rehkopf DH, Subramanian SV. Race/ethnicity, gender, and 
monitoring socioeconomic gradients in health: a comparison of area-based socioeconomic measures--
the public health disparities geocoding project. American Journal of Public Health. 2003;93(10):1655-
1671. 
124. 
Vyas S, Kumaranayake L. Constructing socio-economic status indices: how to use principal 
components analysis. Health Policy Plan. Nov 2006;21(6):459-468. 
125. 
Bonito AJ, Bann C, Eicheldinger C, Carpenter L. Creation of New Race-Ethnicity Codes and 
Socioeconomic Status (SES) Indicators for Medicare Beneficiaries. 2008; 
http://www.ahrq.gov/qual/medicareindicators/medicareindicators.pdf. Accessed 8 January, 2011. 
126. 
Koenker R, Bassett GW. Regression Quantiles. Econometrica. 1978;46:33-50. 
127. 
Manning WG, Mullahy J. Estimating log models: to transform or not to transform? J Health Econ. Jul 
2001;20(4):461-494. 
128. 
Buntin MB, Zaslavsky AM. Too much ado about two-part models and transformation? Comparing 
methods of modeling Medicare expenditures. J Health Econ. May 2004;23(3):525-542. 
129. 
Pregibon D. Goodness of link tests for generalized linear models. Applied Statistics. 1980;29:15-24. 
130. 
Harper S, Lynch J, Meersman SC, Breen N, Davis WW, Reichman ME. An Overview of Methods for 
Monitoring Social Disparities in Cancer with an Example Using Trends in Lung Cancer Incidence by 
Area-Socioeconomic Position and Race-Ethnicity, 1992-2004. American Journal of Epidemiology. Mar 
15 2008. 
131. 
Suppes T, Leverich GS, Keck PE, et al. The Stanley Foundation Bipolar Treatment Outcome Network. 
II. Demographics and illness characteristics of the first 261 patients. J Affect Disord. Dec 2001;67(1-
3):45-59. 
132. 
Zhang F, Wagner A, Ross-Degnan D. Simulation-based power calculation for designing interrupted 
time series analyses of health policy interventions. J Clin Epidemiol. 2011;In press. 
133. 
Keck PE, Jr., McElroy SL, Strakowski SM, Bourne ML, West SA. Compliance with maintenance 
treatment in bipolar disorder. Psychopharmacol Bull. 1997;33(1):87-91. 
134. 
NCSS Statistical Software. Power Analysis and Sample Size (PASS) Software. 2005; 
http://www.ncss.com/pass.html. Accessed Apr 15, 2011. 
135. 
Goodenough W. Describing a Culture. Description and Comparison in Cultural Anthropology. 
Cambridge: Cambridge University Press; 1970. 

 
 
 
 
Wharam, James Franklin 
 
 
 
PCORI Research Plan Template                                                                                                                                               
 
 
32 
136. 
Charmaz K, Belgrave LL. Qualitative Interviewing and Grounded Theory Analysis. The Sage Handbook 
of Interview Research: The Complexity of the Craft. 2nd ed: Thousand Oaks : SAGE; 2012. 
137. 
Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction to qualitative 
methods in health and health services research. BMJ. Jul 1 1995;311(6996):42-45. 
138. 
NVivo Qualitative Analysis Program [computer program]. Version 2.0.2003. Melbourne2003. 
139. 
United States Census Bureau. American Community Survey. 2014; http://www.census.gov/acs/www/. 
140. 
Wagner AK, Ross-Degnan D, Gurwitz JH, Zhang F, Gilden DB, Cosler L, Soumerai SB. Effect of New 
York State regulatory action on benzodiazepine prescribing and hip fracture rates. Ann Intern Med. Jan 
16 2007;146(2):96-103. 
141. 
Simon GE, Bauer MS, Ludman EJ, Operskalski BH, Unutzer J. Mood symptoms, functional impairment, 
and disability in people with bipolar disorder: specific effects of mania and depression. J Clin Psychiatry. 
Aug 2007;68(8):1237-1245. 
142. 
Simon GE, Hunkeler E, Fireman B, Lee JY, Savarino J. Risk of suicide attempt and suicide death in 
patients treated for bipolar disorder. Bipolar Disord. Aug 2007;9(5):526-530. 
143. 
Lu CY, Zhang F, Lakoma MD, et al. Changes in antidepressant use by young people and suicidal 
behavior after FDA warnings and media coverage: quasi-experimental study. BMJ. 2014;348:g3596. 
144. 
Simon GE, Stewart C, Beck A, Ahmedani BK, Coleman KJ, Whitebird RR, Lynch F, Owen-Smith AA, 
Waitzfelder BE, Soumerai SB, Hunkeler EM. National Prevalence of Receipt of Antidepressant 
Prescriptions by Persons Without a Psychiatric Diagnosis. Psychiatr Serv. July 2014 2014;65(7):944-
946. 
145. 
Wharam JF, Zhang F, Landon BE, Soumerai SB, Ross-Degnan D. Low-socioeconomic-status enrollees 
in high-deductible plans reduced high-severity emergency care. Health Affairs. Aug 2013;32(8):1398-
1406.  
 
146.  Diggle, P. and Kenward, M. G. (1994). Informative Drop-out in Longitudinal Data Analysis (with  
discussion).Journal of the Royal Statistical Society, Series C (Applied Statistics),43, (1), 49–93. 
147.  Mason, A., Richardson, S., Plewis, I., and Best, N. Strategy for modelling non-random missing data 
mechanisms in observational studies using Bayesian methods. Journal of Official Statistics. 28(2): 279–
302 (2012). 
148.    University of Washington - Rural Health Research Center. Rural-Urban Commuting Area Codes. 2010;   
http://depts.washington.edu/uwruca/. Accessed May 11, 2012. 
149.     Francis ML, Scaife SL, Zahnd WE, Cook EF, Schneeweiss S. Joint replacement surgeries among 
medicare beneficiaries in rural compared with urban areas. Arthritis Rheum. Dec 2009;60(12):3554-3562. 
150.     Morden NE, Berke EM, Welsh DE, McCarthy JF, Mackenzie TA, Kilbourne AM. Quality of care for 
cardiometabolic disease: associations with mental disorder and rurality. Med Care. Jan 2010;48(1):72-78. 
151.     Williams EC, McFarland LV, Nelson KM. Alcohol consumption among urban, suburban, and rural 
Veterans Affairs outpatients. J Rural Health. Spring 2012;28(2):202-210. 
152. Wharam JF, Graves AJ, Zhang F, Soumerai SB, Ross-Degnan D, Landon BE. Two-year trends in cancer 
screening among low socioeconomic status women in an HMO-based high-deductible health plan. J Gen 
Intern Med. 2012 Sep;27(9):1112-9.  
153. Wharam JF, Landon BE, Xu X, Zhang F, Ross-Degnan D. National Trends and Disparities in 
Mammography Among Commercially Insured Women, 2001-2010. J Public Health Manag Pract. 2015 Sep-
Oct;21(5):426-32.