Discussion of "Connections Between Survey Calibration Estimators and Semiparametric Models for Incomplete Data" by T. Lumley, P.A. Shaw & J.Y. Dai.

We congratulate the authors (henceforth LSD) on a long overdue, detailed review of the connection between the AIPW estimators derived in the incomplete data context via semiparametric theory by Robins, Rotnitzky, and colleagues and the survey calibration estimators used widely in survey sampling. Although this connection has been noted previously (e.g. Robins & Rotnitzky, 1998; Rotnitzky, 2009), the present article appears to be the first in the statistical literature to offer a more comprehensive account. We had only a passing familiarity with this connection, and we are grateful to the editors for the opportunity to offer this discussion, whose preparation required us to acquire a deeper understanding. In what follows, we hope to complement the presentation of LSD by highlighting some further relationships and differences between the two perspectives. We adopt the notation used by the authors and consider estimation of the population total T. We focus on the regression estimator T^reg, which, as in Section 2.1 of LSD may be written equivalently as T^(β^), where T^(β)=T^−∑i=1N(Ri−πiπi)xiβ, (1) a representation that may be more familiar to statisticians well-versed in the AIPW literature. We reiterate and expand upon some important differences between the missing data and survey sampling contexts noted by LSD. In survey sampling, the realizations (x1, y1), . , (xN, yN) that comprise the population are regarded as fixed, and inference on T=∑i=1Nyi, or, equivalently, the population mean N−1T, is the goal. This is based on data (xi, Ri, Riyi), i = 1, . , N, drawn from the population according to a fixed, known design, where n=∑i=1NRi, and Pr(Ri = 1) = πi and Pr(Ri = 1, Rj = 1) = πij for πi known and πij known or unknown, i, j = 1, . , N. The (xi, yi) may be viewed as realizations of random variables (Xi, Yi), i = 1, . , N, representing an independent and identically distributed (iid) sample from some superpopulation; however interest focuses on the fixed quantity T (Sarndal et al., 2003). In contrast, in the incomplete data context, interest is in estimation of μ = E(Y), a parameter associated with the superpopulation. Instead of observing a realization of iid (Xi, Yi), i = 1, . , N, we observe a realization of iid (Xi, Ri, RiYi), i = 1, . , N, where Ri is an indicator of whether or not the value of Yi is observed or missing. Ordinarily, the probabilities of observing Yi for each i are not fixed by design; rather, missingness arises according to some unknown mechanism about which some assumption is made. A common assumption is that Ri is conditionally independent of Yi given Xi, the so-called “missing at random” (MAR) assumption, under which π(Xi) = Pr(Ri = 1|Xi, Yi) = Pr(Ri = 1|Xi). MAR cannot be verified from the observed data, so the validity of inference on μ depends on its unknown relevance. Even if MAR is plausible, which we assume henceforth, the function π(X) is not known and thus must be estimated based on the observed data, usually via maximum likelihood for a posited parametric model, yielding predicted values π^i. From the survey sampling perspective, because the πi are known, T^(β) in (1) is a consistent estimator for T for any fixed β, and the choice β^ given in Section 2 of LSD leading to T^reg is meant to yield an estimator that is more precise than T^. In the incomplete data setting, N−1T^(β) with the π^i substituted for the πi, need not be consistent for μ unless the model for π(Xi) is correctly specified. If this model is correct, then it follows from semiparametric theory that all regular, asymptotically linear estimators for μ may be written in the form (Robins et al., 1994) N−1∑i=1N{RiYiπ^i−(Ri−π^iπ^i)ϕ(Xi)}, (2) where ϕ(X) is an arbitrary function of X. The choice of ϕ(X) leading to the most precise estimator within class (2) is ϕ(X) = E(Y|X). Accordingly, a (usually parametric) model for E(Y|X) may be posited and fitted, and the predicted values substituted for ϕ(Xi) in (2). Scharfstein et al. (1999) made the critical observation that such an estimator is “doubly robust” (DR), i.e. is consistent for μ as long as at least one of the posited models for π(X) or E(Y|X) is correct. Because of the protection afforded by this property, DR estimators have been advocated for routine use. In practice, posited models for E(Y|X) might be linear, generalized linear, or arbitrarily nonlinear in a parameter β, depending on the nature of Y . Usually, the posited model is fitted using ordinary or iteratively reweighted least squares based on the pairs (xi, yi) for which Ri = 1, which, under MAR, would yield a consistent estimator for in a correctly specified model m(X, β), say, for E(Y|X). Kang and Schafer (2007) evaluated the performance of the usual DR estimator for μ in a missing data context under specific simulation scenarios with continuous Y, linear m(x, β) = xβ, and β estimated by ordinary least squares. The estimator exhibited poor performance under scenarios where the models for π(X) and E(Y|X) were only slightly misspecified and/or when the relative magnitudes of the π^i were extremely disparate, with π^i relatively very small for some i, leading Kang and Schafer to issue a strong warning against its routine use. Because of the observational nature of the data, where missingness is by happenstance rather than design, such π^i may be encountered in practice. These results led us (Cao et al., 2009; see also Tan, 2006, 2007 and Tsiatis & Davidian, 2007) to speculate that this poor performance may be partly a consequence of the method used to estimate β in the posited model m(x, β). We proposed considering the class of DR estimators in (2), where ϕ(Xi) is replaced by m(Xi, β), and, among such estimators indexed by β, found the value of β that minimizes the variance of estimators within the class when π(X) is correctly specified regardless of whether or not m(Xi, β) is correct, and a means of estimating this optimal β. The estimator for the optimal β is not ordinary or usual weighted least squares but, rather, involves, in the ideal case where the πi were known, a weighted regression with weights (1 − πi)/πi2, with modification when πi are estimated by π^i as above. Cao et al. (2009) reported simulations showing that the DR estimator incorporating this estimator for the optimal β demonstrated vastly improved performance in the Kang and Schafer and other scenarios. Tsiatis et al. (2011) extended this idea to DR estimators in the more complex setting of longitudinal studies with monotone dropout. We were interested to learn that the same tactic of finding the optimal β minimizing the variance of estimators for T of the form T^(β) in (1) and an estimator for this optimal β using a weighting scheme similar in spirit to that in Cao et al. (2009) was proposed by Montanari (1987); see also Berger et al. (2003). Given that in the survey sampling context, with the πi determined by design, the issue of disparate πi would not be as pronounced, we suspect that the gains in performance realized by such an approach may not be as dramatic. We again compliment the authors on an insightful and useful article.