Selection Bias in Economic Evaluation With Observational Data

Economic evaluation of medical interventions aims to aid decision makers to achieve the goal of efficient use of health care resources at a community level by quantifying the tradeoffs between resources for medical care and the resulting health outcomes. Ideally, these evaluations would lead to the adoption of treatment options that provide value for money and the elimination of those that do not. Unless these evaluations express the causal relationship between the economic endpoint and the treatment, interventions that do not provide sufficient value may be adopted and treatments that do may be eliminated. Hence, it is essential that estimates of differences between treatments reflect the causal effect of treatment on outcomes. When observational data rather than data from randomized clinical trials are used for the estimates of economic endpoints, there is always a possibility that selection bias may limit an investigator's ability to generate an unbiased estimate of the causal relationship between treatment and the economic outcome. The treatment option delivered in a clinical setting is typically the result of decisions made by patients and physicians. Selection bias arises when factors that can influence the treatment choice such as patient health and provider skills also influence outcomes. Adequately accounting for these factors is necessary if observational data are to be used in economic evaluations. The primary objective of this talk will be to describe how observational data can be used to estimate the causal relationship between treatment and outcomes. We will define the parameters of interest for economic evaluations, describe observational study designs, explain selection bias and the mechanism that generates selection of treatment, and then review and compare the various methods introduced in the literature that attempt to address selection bias in observational studies. We will compare instrumental variable (IV) analysis and propensity scores to correct for observational data bias in a cost-effectiveness analysis of two treatments for localized breast cancer (breast conserving surgery with radiation therapy (BCSRT) versus mastectomy (MST)). The data source was the Medicare claims for a national random sample of 2,907 women (age 67 or older) with localized breast cancer who were treated between 1992 and 1994. We constructed instrumental variables for treatment received from a linear probability model of the effects of economic factors and patient characteristics on actual treatment. We then estimated a linear probability model of three-year survival with both observational data (actual treatment) and the instrumental variables for treatment. We used 5 propensity score stratum, these strata were validated and then treatment effects were estimated within each strata. Contrary to the results of randomized clinical trials that found no difference in survival, analysis with the observational data found highly significant differences in survival among the three treatment alternatives: 79.2% survival for BCSO, 85.3% for MST, and 93.0% for BCSRT. The IV results and the propensity score results, in contrast, were consistent with the clinical trial results in that survival rates were not significantly different from each other. Observational data on health outcomes of alternative treatments for localized breast cancer should not be used for cost-effectiveness studies without appropriate adjustment. Both the instrumental variable method and propensity score methods produce results that are consistent with randomized clinical trials. The appropriate method depends on whether there is more potential for bias for non-linearity or unobserved variables.