The use of linear instrumental variables methods in health services research and health economics: a cautionary note.

When analyzing data with the goal of informing health policy, the ability to draw true causal inference from the estimation results is of paramount importance. The typical health policy analysis focuses on identifying the effect that a variable, over which there exists some degree of policy control (xp—henceforth the policy variable), has on an outcome of some policy interest (y—the outcome variable). Empirical analyses that offer estimates of mere associations between xp and y are of little value to policy makers. Estimating the desired causal effect of xp on y is not straightforward and is particularly difficult in the context of nonexperimental (survey) data. In observational surveys respondent behavior, as manifested in the value of y, can be influenced by a myriad of stimuli aside from the policy variable xp. Such alternative influences on y will obfuscate causal inference if they are also correlated with xp. If not properly taken into account, this will lead to bias in conventional estimation of causal effects.1 For example, Mullahy (1997) posits that an individual's habit stock, accumulated over previous periods of smoking, will have an effect on current cigarette demand.2 We might observe, for instance, that individuals with higher habit stocks have greater demands for cigarettes. Interpreting such an observation, as indicative of the causal effect of habit stock on current cigarette smoking will, however, be upward biased if “health-minded” individuals tend to smoke less both currently and in the past. Some confounding influences (henceforth confounders) are observable and can be controlled through the use of regression analysis or matching methods. Others, such as an individual's overall health-mindedness in the above example, are unobservable and their influences cannot, therefore, be controlled by standard estimation methods. In such cases, the policy variable of interest is said to be endogenous.3 A commonly implemented method that is designed to deal with endogeneity is the instrumental variables (IV) method.4 The conventional IV method is based on the assumption that the regression of the outcome of interest (y) on the policy variable (xp) and the observable confounders is linear. The IV method has, nonetheless, been implemented by many applied researchers in health services research and health economics contexts that are inherently nonlinear; e.g., binary response models, count data models, and limited dependent variable models. In the present paper we show, via simulation, that such applications of the conventional IV method, in which nonlinearity in the specification of the regression of y on xp and the confounders (both observable and unobservable) is ignored, can lead to bias in the estimation of the causal effect of xp on y. We conduct the simulation analyses in two common nonlinear regression contexts—binary response models (e.g., probit analysis and logistic regression) and models with nonnegative-dependent variables (e.g., count and duration models).5 We find the potential bias to be substantial. This point is further illustrated in the context of two applied examples using actual data. We first revisit Mullahy's (1997) model of the effect of habit stock (a potentially endogenous continuous variable) on cigarette demand (a count variable). We find, using data obtained from the author, that there is substantial divergence between the linear IV estimate of the habit stock effect and that which we obtained under a flexible specification that accounts for both endogeneity and the inherent nonlinearity of the count regression model. As a second example, we estimate the effect of substance abuse (a potentially endogenous binary variable) on employment status (a binary variable—employed versus not employed). The model is a binary analog to the one used by Mullahy and Sindelar (1996) and Terza (2002) for the estimation of the effect of alcohol abuse on a three-category employment outcome (out of the labor force, unemployed, and employed). We find a substantial difference between the IV and bivariate probit (BVP) estimates of the substance abuse effect. Moreover, we find that, the former is statistically insignificant while the latter is highly significant. The remainder of the paper is organized as follows: in the next section, we review the health services research and health economics literatures, focusing on applications of the linear IV method in inherently nonlinear modeling contexts. We then describe the sampling design for, and results from, our simulation study in which substantial evidence of potential IV bias is demonstrated. Our analyses of the effect of habit stock on cigarette demand and the impact of substance abuse on employment are discussed in the following section. Comparisons of the IV and nonlinear regression estimates of the causal effects in both cases comport with the results of our simulation study. The final section summarizes the results and recommends caution when using the conventional IV estimator in inherently nonlinear settings.