A method for estimating cost savings for population health management programs.

Rising health care costs are a major concern facing employers, managed care organizations, and other payers of health care services across the nation. Those who use the highest amount of health care resources are often those with multiple chronic conditions (Hoffman, Rice, and Sung 1996; Wolff, Starfield, and Anderson 2002), and a large portion of their costs are thought to be preventable through behavior modification and symptom management (Wolff, Starfield, and Anderson 2002). In an effort to contain costs for multimorbid populations, Population Health Management (PHM) programs have flourished in the past decade (Wheatley 2002; Juster et al. 2009). PHM has been described as “a proactive, organized, and cost-effective approach to prevention that utilizes newer technologies to help reduce morbidity while improving health status, health service use, and personal productivity of individuals in defined populations” (Chapman and Pelletier 2004). For managing those with chronic illness, interventions typically consist of nurse case management, telemonitoring, health coaching, motivational interviewing, and condition-specific health education to inform individuals about their condition and to assist them in developing self-management skills. The goals of PHM include improved health and productivity, reduction in modifiable risk factors, promotion of appropriate health care utilization, and reduction of preventable hospitalizations, which should ultimately result in lower health care costs (Johnson 2003; Kindig and Stoddart 2003; Villagra and Ahmed 2004; Seto 2008; Gary et al. 2009; Hibbard, Greene, and Tusler 2009). Despite the belief in the success of PHM programs, there is little evidence to suggest that they significantly reduce health care costs. The cost-effectiveness results in the literature tend to be inconclusive or mixed, with some studies demonstrating savings (Sidorov et al. 2002; Cousins and Liu 2003; Villagra and Ahmed 2004; Sylvia et al. 2008; Cloutier et al. 2009; Dall et al. 2010), whereas others do not (Galbreath et al. 2004; Turner et al. 2008; Nyman, Barleen, and Dowd 2009; Peikes et al. 2009). Results from these prior studies are often suspect, however, due to the frequent use of weak research designs. The validity issue is particularly concerning given that studies lacking in methodological rigor tend to report higher savings and return on investments (ROIs) than studies with more rigorous designs (Linden 2006; Maciejewski, Chen, and Au 2009). For example, studies that have demonstrated significant cost reductions are often those using weaker research methodologies such as pre-post designs without a comparison group, small sample sizes, inadequate observation lengths, and oversimplified statistical methods (Wagner 1998; Selby et al. 2003; Congressional Budget Office 2004; Fitzner et al. 2004). On the other hand, studies using randomized techniques often failed to detect significant cost savings related to PHM (Galbreath et al. 2004; Turner et al. 2008; Peikes et al. 2009). Even in experimental settings, however, the use of an intention-to-treat approach that does not consider intensity or duration of participation in the delivered intervention may dilute the impact of PHM programs and hinder the ability to detect significant cost savings. To confidently evaluate the fiscal impact of PHM programs, an improved method is needed. In health care settings, an experimental design is not often feasible due to ethical or contractual issues. The National Committee for Quality Assurance (NCQA) has recognized that randomized control trials are impractical within health plans (Fetterolf, Wennberg, and Devries 2004). Consequently, most researchers are limited to observational studies using some form of pre-post design (Wilson et al. 2004). The difficulty is that these types of designs are plagued by multiple confounding factors that need to be addressed to preserve the internal validity of the findings. One group of pre-post designs, although the most prominent design used for program evaluation (Johnson 2003; Linden, Adams, and Roberts 2003), can be limited by a number of confounders, including history, maturation, and regression to the mean. History effects may call into question the results if an event occurred between the pre and post measurements that impacted the outcome but was unrelated to the intervention or not of specific interest (Cook and Campbell 1979). Maturation involves the natural changes within participants through the passage of time, such as aging, tiredness, and hunger. This threat is of particular concern for PHM program evaluation given that a participant may improve naturally over time, regardless of whether they participate in the program (Cook and Campbell 1979). Regression to the mean suggests that extreme observations (in either direction) will eventually move toward the mean (Galton 1886), particularly if the observations are extreme due to measurement error (Cook and Campbell 1979). When PHM participants are selected based on extreme characteristics, such as a catastrophic event that leads to an expensive hospitalization, it is likely that participant costs will decrease over time regardless of the intervention (Cook and Campbell 1979; Fetterolf, Wennberg, and Devries 2004; Galton 1886). Using a one-group pre-post design, it is difficult to determine whether a change in participant outcomes is due to the intervention or to these confounding effects. The introduction of a comparison group can limit each of these threats. By measuring the changes in an equivalent comparison group from pre to post, which is also impacted by history, maturation, and regression to the mean, the remaining differences between groups can be more confidently attributed to PHM program involvement. Other common pre-post design elements that may limit study validity, even in a randomized setting, are the use of inadequate observation periods and few repeated measures to estimate the impact of PHM on cost. PHM evaluations frequently compare costs in the year prior to implementation to costs in the following year. Although this is an accepted method of evaluation, it may not adequately control for bias or other confounders. Employing a longitudinal design, with longer study periods and more frequent repeated cost measures, has been recommended as a better method for measuring changes over time in a natural environment (Linden, Adams, and Roberts 2003). Longitudinal data analysis can distinguish temporal changes within individuals from cross-sectional baseline group differences while accounting for the correlation between repeated observations. When the longitudinal relationship between PHM participation and cost is smaller than the cross-sectional relationship, the efficiency of the cost savings estimate can be improved by taking additional measurements on each person. Furthermore, increasing the within-subject variation in time with longer observation periods will increase the power, and thus the precision, of the savings estimate (Diggle et al. 2002). Longer follow-up periods can also help reduce any lingering effects of regression to the mean by continuing to measure group differences after the natural regression has occurred (Linden, Adams, and Roberts 2003). One method of analysis that combines the benefits of longitudinal data with a quasi-experimental design is mixed model smoothing. This semiparametric longitudinal modeling approach uses both fixed and random effects to estimate risk-adjusted cost trends over time while accounting for autocorrelation. The flexibility of mixed model smoothing allows for the detection of group-specific cost trend fluctuations over time, while the smoothing component reduces the influence of outliers (Anderson and Jones 1995; Wang 1998; Wand 2003; Ngo and Wand 2004; Maringwa et al. 2008; Jacqmin-Gadda, Proust-Lima, and Amieva 2010). In addition, inclusion of regular (e.g., quarterly) spline terms can aid in the measurement of natural changes due to seasonality and other history effects that impact both groups. Once these natural variations are removed, the remaining differences can be used to estimate the impact of PHM on costs. In a quasi-experimental setting, identification of an adequate comparison group is also key to ensuring the internal validity of savings estimates. Without randomization, it is important to reduce and/or statistically control for preexisting group differences as much as possible to assess causality. Selection bias is a common threat for quasi-experimental designs. When participants self-select their study group based on whether they choose to participate in PHM, selection bias is introduced. This is particularly concerning when those who choose to participate are inherently different from those who do not. Some of the methods for controlling selection bias include the use of a historical control group (Linden, Adams, and Roberts 2003; Dall et al. 2010), the selection of participations from other groups or sites that do not offer PHM services (Cousins and Liu 2003), and difference-in-difference models that use repeated observations to compare participants against themselves at earlier time points and then compare pre-post differences between groups (Nyman, Barleen, and Dowd 2009; Song et al. 2011). The primary goal of the current research is to develop a consistent model for producing sound PHM cost savings estimates in a quasi-experimental setting. The framework should generate savings estimates that can account for natural temporal change, selection bias, and subject-specific regression to the mean effects and should be flexible enough to accommodate regular updates for the continued monitoring of PHM program effectiveness. A secondary goal is to assess the influence of sampling variability on the precision of the savings estimate to perform basic tests of statistical inference.