1 

Comparing Weighting Methods When Adjusting 

 for Logistic Unit Nonresponse 

 

 

 

 

 

Phillip S. Kott and Dan Liao 

RTI International 

pkott@rti.org; dliao@rti.org

One way to adjust for unit nonresponse in a sample survey is by fitting a logistic response function using some 

variant of maximum likelihood and then treating the inverses of the fitted response probabilities as second-phase 

sampling weights. An slightly different approach fits the logistic response model using a calibration equation. 

Theory suggests that the calibration approach should be more efficient than its maximum-likelihood-based 

competition when estimating the population total (or mean) of a survey variable that is roughly a linear function of 

the covariates in the response model. Suppose, however, as may be the case in some establishment surveys, the 

survey variable is indeed roughly a linear function of a single size measure, but unit response is a logistic function of 

the log of that measure, that is, a one percent increase in the size measure results in a q percent change in the odds of 

responding. What estimation strategy should then be used? We will investigate this question empirically using data 

from public-use files of the Drug Awareness Warning Network (DAWN) survey of drug-related emergency-room 

visits. Prediction (or outcome) models will be employed in interpreting the sometimes surprising results. 

Key Words: Bias; Calibration Weighting; Quasi-random response model; Prediction model; WTADJUST.

1. Introduction

Several methods of adjusting for unit nonresponse begin by fitting a logistic response function to the original sample 

using either maximum-likelihood (ML) or weighted-maximum likelihood (WML). In the next step, either the fitted 

probabilities of response are inverted and treated like second-phase probability-sampling weights or the sample is 

sorted by the fitted probabilities of response into (say) five reweighting classes. 

A similar approach fits the logistic response model indirectly using a calibration equation. When the survey variable 

of interest is roughly a linear function of the covariates in the logistic-response model, Kim and Riddles (2012) 

argue that this method can be more efficient than a weighting scheme derived using maximum-likelihood methods. 

They do not suppy an intutive reason for this, but one explanation is that if the survey variable really obeyed a 

prediction (or outcome) model in which its expectation were a linear funtion of the covariates, then only calibration 

weighting would always produce an unbiased estimator under the model. 

We will compare various methods of the unit nonresponse adjustments based on a simple, one-covariate logistic 

response model using data from the 2008 public use file of the Drug Awareness Warning Network or DAWN 

(Substance Abuse and Mental Health Services Administration, 2011). The DAWN is an annual survey of drugrelated visits to hospital emergency rooms based on a stratified simple random sample of hospitals. 

As is in many establishment surveys, there is an auxiliary size variable attached to each hospital on the DAWN 

frame – all emergency-room visits in a previous year. Using this auxiliary variable in estimation can improve the 

statistical efficiency of estimated totals for DAWN survey variables having a rough linear relationship with it. We 

will investigate some of the estimators, like the simple ratio, that exploit this relationship. 

In one set of simulations, we will generate unit response as a logistic function of the log of the auxiliary variable, so 

that a one percent increase is the size of the auxiliary variable results in a q percent increase (or decreases) in the 

odds of response. Some preliminary exploration of DAWN data reveal this better mimics the behavior of survey 

response than a model in which unit response is a logistic function of the auxiliary variable. Unfortunately, it may 

also remove the theoretical advantage of calibration weighting, which exploits the near linear relationship of the 

survey variable and the auxiliary. 
2 

In a second set of simulations, we will generate unit response as a logistic function of the square root of the 

auxiliary variable, which also fits the actual response behavior of DAWN survey data reasonably well, but construct 

the estimators wrongly assuming response to be a function of the log of the auxiliary variable. This will let us assess 

how robust the estimators are when this failure of the response-model assumption occurs. 

 

 

We will interpret the results drawing on both quasi-probability (often called “quasi-design-based”) and predictionmodel (often called “model-based”) sampling theory. In the first, response is treated as a second phase of 

probability sampling. In the second, the survey value of interest is treated like a random variable with an 

expectation that is a function of the auxiliary variable. 

Our goals here are very modest. We are not attempting to undercover the best model for for adjusting for the unit 

nonresponse in establishments surveys in general or the DAWN in particular. We are simply exploring how well 

various adjustment methods based on fitting a logistic model work under relatively clean circumstances with a 

single cause of nonresponse. Nevertheless, some of our results provide useful insights. In particular, we see that in 

out setup, near unbiasedness under the prediction model is a particularly valuable property. 

After reviewing some of the quasi-probability sampling theory underpinning response modeling in Section 2, we 

provide a description of the data to be used and estimators to be computed in Section 3. We display and interpret 

our empirical results in Section 4 and offer some concluding remarks in Section 5. 

All the calibration weighting in this paper is done using the WTADJUST procedure in SUDAAN 10® (RTI 

International, 2008), which produces only positive weights for respondents. 

2. Some Theory 

Suppose we have a randomly drawn sample S of size n and no nonresponse. Using probability-sampling principle 

(also called “design-based inference”), we can estimate a population total, = , y k k i U U T y y    where U denotes 

the population, with the expansion estimator ,

E

y k k k k k S U t y y I       where Ik = 1 when k  S and 0 

otherwise. We can also write ,

E

y k k k k U S t d y d y     where dk = k k I  is the design weight of element k. 

Treating the Ik as random variables, it is easy to see that E

y

t is an unbiased estimator for Ty

. 

One popular way of handling unit nonresponse is to assume a model where whether (or not) a unit responds is 

treated as an additional phase of probability sampling. In the simplest version of this quasi-probability-sampling 

approach, each element k  U is assumed to have a probability of response ρk if it is sampled. The probability 

elements k and j jointly respond if sampled is ρkρ j, and the magnitude of ρk is independent of whether k is chosen for 

the original sample. The value of ρk is itself unknown, but it is assumed to be expressible as a function ρk = f(xk 

T

γ), 

where x is a vector of characteristics for k that is known when k  S. Although the form of f (.) is assumed to be 

known, its governing parameter vector γ is not; it needs to be estimated from the sample. 

When a consistent estimator g is found for γ, it is not hard to show that under the assumptions of the response model 

and the original probability-sampling mechanism, the double expansion estimator E*

y

t  ( ) U k k k k k  y I Q p  

Rw yk k is a nearly (i.e., asymptotically) unbiased estimator for , Ty

 where Qk = 1 when k responds if sampled and 

0 otherwise, pk = f(xk 

T

g), R is the subset of S that responds, and wk= 1/(k pk) = dk /pk is the sampling weight for 

unit k, the unit’s design weight adjusted for nonresponse. The interested reader is referred to, for example, Chang 

and Kott (2008) for a rigorous treatment of the underlying theory. 

A commonly used response model assumes that the log odds of unit k responding is linear function of xk. This 

means that f(.) is the logistic function f(xk 

T

γ) = 1/[1 + exp( xk 

T

γ)]. One can then estimate γ through maximum 

likelihood (ML), which comes done to using Newton’s method (i.e., successive linearizations) to find a g that 

satisfies the estimating equation: 
3 

 

 

 

  . (1) k

T Q f k k

k S

 

      

 x g x 0

The ML method is not only consistent under mild conditions on the population and original sample design, but also 

produces the most efficient estimator for . That is why it is arguably superior to its “design-based” alternative that 

finds a solution to the weighted estimating equation: 

  . (2) k k

T d Q f k k

k S

 

      

 x g x 0 

The weighted-maximum-likelihood (WML) method in equation (2), which also produces a consistent estimator for 

, may be more commonly used in practice. See, for example, Diaz-Tina et al. (2002). From here on, we drop the 

cumbersome phrase “under mild conditions on the population and original sample design” for convenience. The 

reader should be aware not only that the phrase is missing but also that we always assume those mild conditions to 

be met. 

Kim and Riddles (2012) assert that if the goal is estimating Ty

, and yk is roughly linear in xk, then a consistent 

estimator for g that is likely to be more efficient than either ML or WML when creating a nearly unbiased estimator 

for Ty

 is the solution of the equation: 

  1 . (3) k

k k T

k

Q

d

k S f

 

     

    

 x 0

x g

Equation (3) is equivalent to the second equality in the calibration equation: 

  

k

k k k k k k T

k

Q

w d d

k R k S k S f



  

   x x x 

x g

 (4) 

It is easy to see from equation (4) that if yk were exactly equal to a linear function of xk, say xk

T

, then adjusting for 

nonresponse by solving for g in equation (3) and then computing E*

y

t  Rw yk k would produce the same 

estimator for Ty

 as if there were no nonresponse (since T Rwk k x β  ). T

S k k  d x β The same cannot be said, in 

general, if g were estimated using ML or WML. That is the reasoning behind the assertion in Kim and Riddles. 

Another advantage of using the calibration approach in adjusting for unit nonresponse is that it can be used when xk

is not known for every sampled unit as long as the vector of population totals Tx  U k

x is known (or there is an 

unbiased estimator for Tx

). This can be done by finding a g such that 

  . (5) k

k k k k k T

k

Q

w d

k R k S k U f

 T

  

   x x x   x

x g

Calibration weighting can also be used in the absence of nonresponse  or after the sampling weights have been 

adjusted for nonresponse  to increase the efficiency the estimator when yk is roughly linear in xk. In fact, Deville 

and Särndal (1992) developed calibration weighting for that purpose. 
4 

The inverse of the logistic function can be written generically as 1/f() = 1 + exp(), and is always greater than 1. 

When calibrating in the absense of nonresponse to increase efficiency, 1/f() is replaced by an h() such that h(0) = 

h'(0) = 1. One then finds a vector g “estimating” 0 such that 

 

 

  . (6) T w d h k k k k k k

k S k S k U

 T

  

   x x g x x   x

Although g has a known target (0), a more efficient estimator for Tycomes from solving for g in the calibration 

equation (6). 

All choices for h() are asymptotically equivalent; that is, combining them with equation (6) lead to estimators for 

Ty

that have asymptotically identical mean squared errors. Two common ones are 1+ and exp(). When the 

former is used, it is possible to solve for g using matrix algebra. This convenience is offset by the possibility that a 

calibration weights, wk, will fall below zero. When exp() is used, that can never happen. It is possible, however, 

that no g satisfying the calibration equation exists. 

Deville and Särndal noted that when the components of xk are binary, and h() = exp(), g can be solved using 

iterative proportional fitting or raking. Folsom and Singh (2000) allowed the h() to vary across the units. 

Observe that, if we replace ( ) T

k

h x g in equation (6) by ( ), T

k

h x g where 

 hk() = 1 when dk = 1 

 1/ dk + [1  (1/ dk)] exp([1  (1/ dk)]-1) otherwise, 

then wk will never fall below 1, a property many find desirable. 

We do not presently know how useful calibration weighting is when there is unit nonresponse but the survey 

variable is not roughly a linear function of the covariates of the response model. Moreover, we do not know 

whether it is better to make a single calibration adjustment in this situation or two (one to adjust for nonresponse and 

one to increase statistical efficiency). Those issues will be addressed in the following sections. 

3. Data and Estimators 

We generate a synthetic population, U, of hospitals from the 2008 DAWN public-use file in the following manner. 

The file is at the case level. We take the weighted sums across all cases in a hospital to create hospital-level 

variables attached to a hospital record k. We then treat that record as ak independent records in U, where ak is the 

minimum of the case weights in the hospital randomly rounded to an integer. 

After creating U, we independently draw 2,500 stratified simple random samples of size 400 from U using the strata 

definitions on the public-use file. These definitions incorporate information on location and hospital ownership 

(public or private) not directly provided on the file. 

We set the stratum sample sizes roughly proportional to a size measure, but never less than 4. For the size measure, 

z, we use total drug-related emergency-room visits. The actual DAWN frame size variable, total emergency-room 

visits in a previous year according to the American Hospital Association, is not on the file. Design weights in our 

simulations vary between 4.375 and 48. 

For each simulated sample, we generate a respondent sample R based on Bernoulli draw from the logistic function:

 k = (zk) = (1 + exp(3.735  .4log(zk))-1, (7) 

We also create alternative respondent samples using 

 k = (zk) = (1 + exp(.597  .005zk

1/2)-1

.  (8) 
5 

 

 

 

Both response models produce unweighted overall response rates of around 54%, which is similar to actual DAWN 

experience, where response is also a mildly increasing function of size. 

Finally, we treat drug-related emergency-room visits with an adverse pharmaceutical reaction (y1k) and deaths (y2k) 

as two survey variables of interest and compute Ty = U k  y for each (suppressing the indices 1 or 2 for 

convenience). We also create a third synthetic variable for investigative purposes: y3k = zk

1.3

. 

Many of the estimators we analyze have the form E*

y

t  . Rw yk k For two, wk= dk /pk, where dk is the inverse of 

the hospital’s selection probability into the sample of 400, and pk is either the solution to equation (1) or (2) with

f() =[1 + exp()]-1, and xk = (1 log(zk))'. We label these estimators ML

y

t and ,

WML

y

t respectively. The 

corresponding calibration estimator derived from solving equation (4) with xk = (1 log(zk))' is labeled 1

.

Cal

y

t It is 

computed using the nonresponse option in SUDAAN’s WTADJUST, setting the lower bound to 1 and the center to 

2. 

All three of these estimators should be nearly unbiased when equation (7) is used to generate response. A fourth 

estimator, 

1_

,

Cal pop

y

t makes use of both the response model and the presumption that the survey variable is roughly 

linear in z. Its weights come from solving the calibration equation in (5) with xk = (1 log(zk) zk)'. The the post

option in WTADJUST is used to compute those weighting with the lowerbound set to 1 and the center to 2. 

A more conventional way to exploit the presumed rough linear relationship between the survey variable and z is 

with a ratio estimator of the form: 

*

,

E

y r t  .

k k

k R

z

k k

k R

w y

t

w z









We label the ratio version of the ML estimator 

,

ML

y r t and define 

,

WML

y r t and 1

,

Cal

y r t analogously. 

One can also apply a second calibration-weighting adjustment to the weights in Cal1

y

t to exploit the presumed rough 

linear relationship between the survey variable and z. The estimator Cal2

y

t begins by setting the dk in equation (6) to 

the weights from 1

,

Cal

y

t replaces h() with hk() = 1/dk + [1  (1/ dk)] exp([1  (1/ dk)]-1), and sets xk = zk (note 

that because there is a previous nonresponse adjustment, no dk = 1). This is done with the post option in 

WTADJUST by setting the lower bound to 1/dk (the center takes on its default setting of 1). Alternative two-step 

calibration estimators, labeled Cal2 _1

y

t and Cal z z 2 _ log

y

t , set xk = (1 zk)' and xk = (zk zklog(zk))', respectively. 

Little (1986) suggested that instead of using fitted pk-values directly from either (1) or (2), reweighting groups 

should be created based on the sorted pk-values. In our context, this is the same as grouping by the sorted z-values. 

We sort a sample S into five nearly equal groups and compute these grouped estimators: 

5

1

,

c

c

c

k

Gr k S

y k k

c k R k

k R

d

t d y

d



 



   

  

 

 



  
6 

 

 

 

 

 

5

1

,

5

1

, and

c

c

c

c

c

c

k

k S

k k

c k R k

Gr k R

y r k

k U k

k S

k k

c k R k

k R

d

d y

d

t z

d

d z

d



 







 



   

 

     

        

 

   



  





  

5

1

.

c

c

c

k k

SGr k R

y k

c k U k k

k R

d y

t z

d z



 



 

      



  

where the subscript c denotes the group. 

These estimators effectively estimates k with uk'p+, where uk is a five-element vector of group-membership 

indicators (i.e., ukc = 1 when k  Uc, and 0 otherwise). When k  Uc, k is implicitly estimated by k c p p   

/

c c R S j j   d d in both the grouped ( Gr

y

t ) and grouped-ratio (

,

Gr

y r t ) estimators. Little and Varitvarian (2003) 

argue for using the unweighted response rate in group c (p+c= rc/nc) in this context, but Kott (2011) presents reasons 

for preferring our formulation. 

The separate-grouped-ratio estimator ( SGr

y

t ) has the word “ratio” in its name for an obvious reason: there is a ratio 

in every group. Nevertheless, it can be expressed in double-expansion form as E*

y k k R

t w y    ( / ) R k k k  d p y

if we view the estimate of the group-c response rate as c

p  / .

c c R U j j j   d z z

Although Gr

y

t , 

,

Gr

y r t , and SGr

y

t are not nearly unbiased under the logistic response model when the z-values vary 

within each group, their use should remove a good deal of the potential for response bias when response is generated 

by either equation (7) or (8) because the z-values do not vary within a group as much as they do across the entire 

population. 

For comparison purposes we also compute the following two estimators that naively estimate pk by the overall 

unweighted response rate r/n: 

0 0

,

, and .

k k

k R

y k k y r k

k R k U k k

k R

d y

n

t d y t z

r d z



 



 

     



  

4. The Results and their Interpretation 

Table 1 contains the formulae for all the estimators. Table 2 displays the relative empirical biases and root mean 

squared errors of the competing estimators when response is generated using equation (7). Not surprisingly, none 

of the ML, WML and calibrated estimators designed to remove response bias have an empirical bias that is more 

than 12% of the corresponding empirical root mean squared error, which means that the contribution of bias to mean 

squared error is always less than 1.5% (since 0.122 < 0.015), usually much less. 

Among the three one-step estimators ,

ML

y

t ,

WML

y

t and 1,

Cal

y

t the calibration estimator is always the least efficient 

(i.e., has the largest mean squared error). This efficiency disadvantage usually goes away when the estimators are 
7 

in ratio form ( 

,

ML

y r t , 

,

WML

y r t , 

1

,

Cal

y r t ), except for the contrived third survey variable. This is likely because that 

variable is not as nearly linear in the size measure z as the other two. 

Applying a second calibration adjustment to Cal1

y

t in place of a ratio, as Cal2

y

t does, improves efficiency especially 

for the third variable (it is not shown, but adding the restriction on the weights for Cal2

y

t that none fall below unity 

has virtually no effect on its empirical bias or root mean squared error). Using a single calibration to adjust for 

nonresponse and exploit the rough linear relationship between survey variable and size measure simultaneously , as 

Cal pop 1_

y

t does, appears less efficient than the ratio estimators for the first two survey variables (adverse reaction to 

pharmaceuticals and deaths). 

Adding a constant to the x-vector ( 2 _1)

Cal

y

t usually has a negative impact on efficiency in Table 2. Adding 

zklog(zk) in its place ( 2 _ log )

Cal z z

y

t has the opposite effect. This is likely because each of the survey variables is 

better fit as a linear model of z and zlog(z) than as a function of z and an intercept or of z alone. This is confirmed 

by running regressions in U (not shown). 

The naïve expansion estimator 0

y

t performs poorly both in terms of bias and mean squared error as expected. It 

ratio version, however, 0

,

, y r t appears more efficient than any of the ML, WML, or calibration estimators for deaths. 

Let us explain why we think that happens. 

 

The expectation of 0

y r,

t under quasi-probability theory is roughly: 

0

, , ,

( )

( ) .

( )

k k k k k k

k U k U

I Q y r k I Q k

k U k U k k k k k k

k U k U

d I Q y z y

E t z E z

d I Q z z z

 

 

 

       

               

 

   





 (9) 

Now suppose yk itself is a random variable such that Epr(yk|zk) =zk. Under this prediction model, the expression on 

the far right of equation (9) has the same expectation as Ty; that is Tz. Since Ty is being treated as a random 

variable, strictly speaking, 0

y r,

t is an predictor of Ty, which is origin of the term “prediction model.” 

In point of fact, the prediction-model asumption Epr(yk|zk) =zk does not appear to hold for any of our three survey 

variables, but it does not fail too badly for deaths. As a consequence, 0

y r,

t is not noticeably biased, and a relatively 

low empirical mean squared error results. 

The near equality between the estimator and its target under the prediction-model assumption Epr(yk|zk) =zk can be 

shown to apply to all our ratio and two-step calibration estimators. Moreover, this near equality, although 

dependent on the prediction-model assumption, does not require the response response function (zk) to be a logistic 

in log(zk); it can be any function of zk. In other words, the actually shape of the response function is ignorable as 

long as it is a function of zk. Notice that we have not made the same assumption about the probability-sampling 

mechanism, which incorporates information about location and ownership not reflected in the assumed prediction 

model. 

The prediction-model for Cal2 _1

y

t can be expanded to Epr(yk|zk) = 0 + 1zk . Similarly, the prediction-model can 

be expanded to Epr(yk|zk) = 1zk + 2zk log(zk) for Cal z z 2 _ log

y

t and to Epr(yk|zk) = 1zk + 2log(zk) for 1_.

Cal pop

y

t

Although none of these expanded prediction-model assumptions may hold for any of our three survey variables, they 
8 

are often closer to being true than the simple ratio-model assumption. This can explain the results in Table 3. When 

we simulate responses using equation (8) rather than (7), these three estimators tend to have smaller empirical 

biases and often smaller empirical mean squared errors than their non-grouped competitors. 

 

The poor relative showing of these three estimators for deaths is likely the result of the expectation of deaths being 

nearly linear in the size measure, drug-related hospital-room visits. One should also note that although 

Cal z z 2 _ log

y

t does an excellent job estimating Ty for the third variable, its empirical bias to root mean squared error 

ratio in Table 3 is over 20%. 

What stands out most in both Tables 2 and 3 is that the separate-grouped-ratio estimator 

,

SGr

y r t has the smallest mean 

squared error among the competitors for adverse pharmaceutical reactions and deaths. To see why that may be, 

observe that the triple expectation of 

,

SGr

y r t and its target (Ty) would be equal under a prediction model where 

Epr(yk|zk) = zkuk' (recall that uk itself depends on zk). This may be viewed as another way of expanding the 

assumption of a simple ratio prediction model since zkuk' = zk + zkvk', where vk = ( uk1 ... uk4)', and c = c  . 

Even though the model lacks plausibility when   0, it does feature four parameters in addition to simple-ratio 

slope , so it is not too surprising that 

,

SGr

y r t is often superior to its competitors. 

Notice that in both tables, the empirical bias for the separate-grouped-ratio estimator is always less than that for the 

grouped estimator Gr

y

t even though both estimators rely on the same approximate response model: a constant 

response rate within each group. This suggests that the prediction model supporting the latter provides a closer fit 

to the population data than the prediction model supporting the former: Epr(yk|zk) = uk'. 

The grouped-ratio estimator 

,

Gr

y r t is supported by the simple ratio-model assumption, which, as we noted before, is 

not too bad for deaths. This may be why it has less empirical bias for deaths than the separate-grouped-ratio in both 

tables, and its empirical mean squared error tends to be among the lowest of the competiting estimators. 

In DAWN as in many establishment surveys, the impact on mean squared error of the increased variability of the 

weights due to calibration adjustments is not as clear as in surveys where ideally every unit has the same weight. 

Still, too much unproductive weight adjustment from a prediction-model viewpoint does seem to have an adverse 

effect on empirical mean squared error. We have already noted that 0

y r,

t is more efficient than many of its 

competitors for deaths. In addition, the nonresponse-adjusted-only calibration estimator Cal1

y

t is less efficient than 

its WML counterpart in all cases, which in turn is usually less efficient than its ML analogue. 

5. Some Concluding Remarks 

One of the take-away messages from the the last section is that calibration weighting appears most useful for 

nonresponse adjustment when it is also employed to reduce prediction-model bias under an assumed model. In 

addition, a single-step calibration-weighting routine that includes variables to remove both response-model bias and 

prediction-model bias (as used in Cal pop 1_

y

t ) does not appear to be as effective as employing separate calibration 

steps for each purpose. 

Another take-away message is that a calibration-weighting step that reduces the prediction-model bias under a 

roughly-holding model can decrease overall mean squared error whether or not the bias due to unit nonresponse has 

been (asymptotically) removed by the nonresponse-adjustment step. The same can be said about a ratio adjustment 

when the simple-ratio model roughly holds. 

We should not be too impressed by the strong performance of the separate-ratio estimator in the simulations of the 

previous two sections because both the true response model and the true prediction model were monotonic functions 

of a single variable (although we do not know for certain that this monotonicity holds for deaths and adverse 
9 

pharmaceutical reactions, it seems likely). That will rarely be the case in practice. In the real DAWN, for example, 

both response and survey variables are functions of hospital size, ownership, geographic location, and level of 

urbanization. Separate modeling steps seem prudent especially since survey variables tend to be a function of size, 

other variables, and interactions between some of those other variables and size , while unit response in better 

modeled as a function of the log of size, other variables (but not necessarily the same list of other variables as in the 

prediction model), and interaction between some of those variables and log of size. 

We end with a caveat about drawing strong conclusion from limited simulations. We also need to point out that the 

issue of mean-squared-error estimation has not been addressed. One reason to focus on the relative size of the 

empirical bias compared to empirical mean squared error is that when bias plays too big a role in mean squared 

error, large-sample variance estimators based on the assumption that bias is asymptotically ignorable, like those for 

one-step calibration in Kott (2006), will fail. 
10 

Table 1. Summary of Estimators Being Analyzed 

Estimators in double expansion form: E*

y

t  

 

 

 

 

 

 

 

 

 

 

 solves 

( / ) R k k k  d p y



0

y

t pk = rc

/ nc for k  Rc 

ML

y

t     1 2

1 1 0

1 exp log( ) solves 

log( ) 0

k k k k S

k

p g g z Q p

z

    

                  

WML

y

t     1 2

1 1 0

1 exp log( ) solves 

log( ) 0

k k k k k S

k

p g g z d Q p

z

    

                  

Cal1

y

t   1 2

1 1 0

1 exp log( ) solves 

log( ) 0

k k

k k k S

k k

Q p p g g z d

p z

       

                   

Cal pop 1_

y

t   1 2 3

1

1 exp log( ) k k k p g g z g z



        

1 1

solves log( ) log( ) k

S U k k k

k

k k

Q

d z z

p

z z

           

    

 

Gr

y

t k

p  /

c c R S j j   d d for k  Rc 

SGr

y

t k

p  /

c c R U j j j   d z z for k  Rc 

Estimators in ratio form: *

,

E

y r t

( / )

( / )

R k k k

U k

R k k k

d p y

z

d p z











0

y r,

t pk = rc

/ nc and k  Rc 

,

ML

y r t     1 2

1 1 0

1 exp log( ) solves 

log( ) 0

k k k k S

k

p g g z Q p

z

    

                  

,

WML

y r t     1 2

1 1 0

1 exp log( ) solves 

log( ) 0

k k k k k S

k

p g g z d Q p

z

    

                  

1

,

Cal

y r t   1 2

1 1 0

1 exp log( ) solves 

log( ) 0

k k

k k k S

k k

Q p p g g z d

p z

       

                   

,

Gr

y r t k

p  /

c c R S j j   d d for k  Rc

Ratio forms for Cal pop 1_

y

t and SGr

y

t are not displayed because each is equal to its own ratio form. 

Two-step Calibration Estimators: 

E*

y

t  ( / ) , R k k k k  d p h y where pk comes from Cal1

y

t



Cal 2

y

t hk = 1/dk + [1  (1/dk)] exp{[1  (1/dk)]-1

 zkg} solves ( ) R U k k k k k   d p h z z 

Cal2 _1

y

t hk = 1/dk + [1  (1/dk)] exp{ [1  (1/dk)]-1 [a + zkg] } 

1 1

solves ( ) U

R k k k

k U k

d p h

z z

   

        







Cal z z 2 _ log

y

t hk = 1/dk + [1  (1/dk)] exp{ [1  (1/dk)]-1 [zkg1 + zk log(zkg2] } 

( )

log( ) log( )

k U k

R k k k

k k U k k

z z

d p h

z z z z

   

        






11 

Table 2. Summary of Simulation Results When Equation (7) is Used to Generate Response

Estimator Relative Empirical Bias (%) 

  2500 ( )

1

1

2500

a

y y a

y

t T

T

  

Relative Empirical Root 

Mean Squared Error (%) 

 2 2500 ( )

1

1

2500

a

y y a

y

t T

T

  

Adverse 

reactions

Deaths Synthentic 

variable1 

Adverse 

reactions 

Deaths Synthentic 

variable1

0

y

t

19.53 19.24 26.28 20.93 23.98 28.32 

ML

y

t

-.0.03 -0.05 -0.01 4.51 11.80 5.59 

WML

y

t

0.10 0.03 0.21 4.60 11.61 6.38 

Cal1

y

t

-0.05 -0.18 0.02 5.27 11.95 7.32 

0

y r,

t

-.068 -0.89 4.76 2.77 10.88 5.40 

,

ML

y r t

0.01 -0.01 -0.03 2.41 11.18 2.08 

,

WML

y r t

-0.01 -0.06 0.00 2.41 11.15 2.28 

1

,

Cal

y r t

-0.01 -0.13 -0.07 2.41 11.15 2.47 

Cal pop 1_

y

t

-0.20 -0.27 0.11 2.47 11.28 0.98 

Cal2

y

t

0.01 -0.11 -0.01 2.40 11.14 1.46 

2 _1

,

Cal

y r t

0.00 -0.13 0.03 2.43 11.19 1.21 

Cal z z 2 _ log

y

t

0.03 -0.16 -0.01 2.35 10.98 0.28 

Gr

y

t

1.43 1.49 1.90 4.43 11.42 5.88 

,

Gr

y r t

-0.24 -0.16 0.16 2.40 10.76 1.98 

Sgr

y

t

-0.27 -0.62 0.45 2.29 10.00 1.02 

 

1

 (Drug-related hospital visits)1.3 
12 

Table 3. Summary of Simulation Results When Equation (8) is Used to Generate Response

Estimator Relative Empirical Bias (%) 

  2500 ( )

1

1

2500

a

y y a

y

t T

T

  

Relative Empirical Root Mean 

Squared Error (% ) 

 2 2500 ( )

1

1

2500

a

y y a

y

t T

T

  

Adverse 

reactions

Deaths Synthentic 

variable1

Adverse 

reactions

Deaths Synthentic 

variable1

0

y

t

18.60 16.41 26.87 20.10 21.67 28.91 

ML

y

t

 

0.09 -1.42 1.62 4.59 11.52 5.93 

WML

y

t

2.06 0.55 4.20 5.10 11.48 7.84 

Cal1

y

t

2.79 1.27 5.19 5.88 11.83 9.00 

0

y r,

t

-1.09 -2.90 5.64 2.94 10.93 6.17 

,

ML

y r t

-0.35 -1.84 1.10 2.49 11.06 2.34 

,

WML

y r t

-0.43 -1.87 1.55 2.53 11.04 2.76 

1

,

Cal

y r t

-0.46 -1.89 1.75 2.54 11.03 2.98 

Cal pop 1_

y

t

-0.05 0.40 -0.05 2.38 11.44 0.87 

2

,

Cal

y r t

-0.16 -1.12 0.93 2.42 11.01 1.69 

Cal2 _1

y

t

-0.03 -0.72 0.48 2.41 11.08 1.27 

Cal z z 2 _ log

y

t

0.35 0.26 -0.06 2.31 11.06 0.26 

Gr

y

t

1.16 1.45 2.32 4.37 11.55 6.02 

,

Gr

y r t

-0.49 -0.19 0.57 2.44 10.94 1.97 

Sgr

y

t

-0.48 -0.63 0.58 2.31 10.23 1.04 

1

 (Drug-related hospital visits)1.3 
13 

References 

Chang, T. and Kott, P.S. (2008). Using Calibration Weighting to adjust for Nonresponse Under a Plausible Model, 

Biometrika, 95, 557-571.

Deville, J-C. Särndal, C-E. (1992). Calibration Estimators in Survey Sampling, Journal of the American Statistical 

Association, 87, 376-382. 

Diaz-Tena, N., Potter, F., Sinclair, M. and Williams, S. (2002). Logistic Propenisty Models to Adjust for 

Nonresponse, ASA Proceedings of the Survey Research Methods Section, 776-781.

Folsom, R. E. and Singh, A. C. (2000). The Generalized Exponential Model for Sampling Weight Calibration for 

Extreme Values, Nonresponse, and Poststratification, Proceedings of the American Statistical Association, Survey 

Research Methods Section, 598-603.

Kim, J.K. and Riddles, M. (2012). Some Theory for Propensity Scoring Adjustment Estimator, under review by 

Survey Methodology. 

Kott, P.S. (2011). Why One Should Incorporate the Design Weights When Adjusting for Unit Nonresponse Using 

Response Homogeneity Groups, Survey Methodology, forthcoming. 

Kott, P.S. (2006). Using Calibration Weighting to Adjust for Nonresponse and Coverage Errors, Survey 

Methodology, 32, 133-142.

Little, R.J. (1986). Survey Nonresponse Adjustments, International Statistical Review, 54, 139-157.

Little, R., and Vartivarian, S. (2003). On Weighting the Rates in Non-response Weights. Statistics in Medicine, 22, 

1589-1599. 

RTI International (2008). SUDAAN Language Manual, Release 10.0. Research Triangle Park, NC: RTI 

International. 

United States Department of Health and Human Services (2011). Drug Abuse Warning Network (DAWN), 2008 

(Computer file of survey conducted by the Substance Abuse and Mental Health Services Administration, Center for 

Behavioral Health Statistics and Quality). Ann Arbor, MI: Inter-university Consortium for Political and Social 

Research.