ESTIMATING THE MODEL VARIANCE OF A RANDOMIZATION-CONSISTENT

REGRESSION ESTIMATOR

Phillip S. Kott and K.R.W. Brewer

National Agricultural Statistics Service, Fairfax, VA 20230, USA and

Australian National University, ACT 0200, Australia

KEY WORDS: Asymptotic; Calibration equation;

Weighted-residual-mean-squared-error estimator

1. INTRODUCTION

Suppose we want to estimate a population

total, T = 3U yk, based an a sample, S, of n elements.

Randomization-based theory tells us we can do that

with a regression estimator of the form:

t = 3 (yk /Bk) 

 k0S (1)

+ [ 3 xk ! 3 

(xk /Bk

)]( 3 xk

!xkdk /Bk

)

-1 3 xk

!ykdk /Bk , 

 k0U k0S 

 

 k0S k0S

where Bk

 is the sample selection probability of element

k, xk = (xk1 , ..., xkP) is a row vector of values associated

with element k, 3U xk is known, and the dk are arbitrary

non-negative constants. Särndal, Swensson, and

Wretman (1989) call t a “general regression estimator”

or GREG. From a mod el-based point of view,

however, t is not very general. That is why we do not

use that name here. 

The estimator t can be written as t = 3S akyk,

where ak = (1 /Bk) + [ 3U xi ! 3S (xi /Bk)](3S xi!xidi /Bi)

-1

xk!dk/Bk. Often the dk in equation (1) are chosen so that

these ak

 have desirable properties (e.g., being positive;

see Brewer, 199 9). The ak

 have been constructed in

such a way that the calibration equation (Deville and

Särndal 1992), 3S akxk = 3U xk, is satisfied. 

Under mild conditions, t is randomization

consistent (see Isaki and Fuller, 1982, who use the

synonymous term “design consistent;” Brewer, 1979,

introduces a similar property). We will not be deeply

interested in randomization-based properties here. Our

focus, instead will be on the properties of t as an

estimator for T under the linear model: 

yk = xk$ + ,k, 

where $ is an unspecified P-member column vector,

E(,k*zk) = 0 for all k 0 U, and zk = (xk, Bk, dk). It is

easy to see that t is a unbiased estimator for T under the

model in the sense that 

))))))))))))))))))))))))))))))))))))))

Prepared for the Joint Statistical Meetings, August

2001, Atlanta, Georgia.

E(t ! T) = E(3S akyk ! 3U yk) = 3S akxk$ ! 3U xk$ = 0 .

We concentrate on the model variance of t as

an estimator for T (also called the “prediction variance

of t”) because evidence suggests that such a focus can

produce variance estimators with better coverage

properties (see Kott 1990). This phenomenon results

from the model-based approach!s attention to the

realized sample and the repercussion of using that

sample for inference. Randomization-based inference,

by contrast, averages over all possible samples. 

We will further assume that E(,k,i*zk ,zi,) =

*ikFk

2

, where Fk

2

 may be a function of zk. The variance

of t as an estimator for T under the model we have

specified is 

 E[(t ! T)2

] = E[( 3 akyk ! 3  yk)

2

]

 k0S k0U

 = E[( 3 ak,k ! 3 ,k)

2

]

 k0S k0U

 = 3 ak

2

Fk

2 ! 23 

akFk

2

 + 3 Fk

2

. (2)

 k0S 

 

 k0S k0U

The weighted-residual-mean-squared-error

estimator (Särndal Swensson, and Wretman 1989; p.

432, eq. (4.6)) for t under Poisson sampling is 

vR

 = 3S ak

2

(1 ! Bk

)rk

2

, 

where 

rk = yk ! xkb, and 

b = (3S xk

!xkdk /Bk

)

-1 3S xk

!ykdk /Bk

is an unbiased estimator for $. We will be concerned

here with adapting vR

 to estimate the model variance

expressed in equation (2). By starting with a

varian ce/mean-squared-error estimator from

randomization-based theory, we protect ourselves

somewhat from model failure. Many of our results are

given in a different context by Royall and Cumberland

(1978). Sections 2 through 7 discuss alternativet

asymptotic setups. Section 8 provides a summary and

a discussion. 
2. WHEN THE SAMPLE IS LARGE AND THE

POPULATION IS LARGER

The simplest situation to discuss is when the

sample size is large and the population is larger. By the

former, we mean that terms of the same relative

asymptotic order as 1/n can be ignored. By the later,

we mean that terms of the same relative asymptotic

order as n/N, where N is the size of U, can be ignored.

All the Bk

 are assumed here to be O(n/N), so

that vR

 can be approximated by

 v0 = 3S ak

2

rk

2

when relative O(n/N) terms are ignored. We will

assume, not unreasonably, that both 3SakFk

2

/ 3S ak

2

Fk

2

and 3U Fk

2 / 3S ak

2

Fk

2

 are O(n/N). Thus, when relative

O(n/N) terms are ignorable, the model variance of t and

an estimator of T from (2) is (approximately) 

V0 = 3S ak

2

Fk

2

.

Observe that

E(rk

2

) = E[( yk ! xkb)

2

]

 = E[( ,k ! xk ( 3S xi!xidi /Bi)

-1 3S xi

!,idi /Bi)

2

]

 

 = Fk

2 ! 2 xk

 ( 3S xi!xidi /Bi)

-1xk

!(dk /Bk)Fk

2

 +

 

 xk ( 3S xi!xidi/Bi)

-1[ 3S xi

!xi(di/Bi)

2

Fi

2

](3S xi!xidi /Bi)

-1xk

!

 

 

 

We can reasonably assume that 

xk (3S xi!xidi /Bi)

-1xk

!(dk /Bk)Fk

2

 and 

xk (3S xi!xidi /Bi)

-1[3S xi

!xi(di /Bi)

2

Fi

2

] (3S xi!xidi /Bi)

-1xk

!

are O(1/n) when P is fixed as n grows arbitrarily large.

Consequently, v0

 is an (approximately) unbiased

estimator for V0

 and, thus, the model variance of t as an

estimator for T when both relative O(n/N) and O(1/n)

terms are ignorable. 

3. WHEN THE POPULATION IS LARGE AND Fk

2

IS KNOWN UP TO A CONSTANT

In this section O(n/N) is again ignorably small,

but O(1/n) may not be. If Fk

2

 = kvk

 for known vk

, then

from equation (3):

E(rk

2

) = Fk

2

{1 ! 2 xk ( 3S xi!xidi /Bi)

-1xk

!(dk /Bk) +

xk ( 3S xi!xidi /Bi)

-1[ 3S xi

!xi(di/Bi)

2

vi](3S xi!xidi/Bi)

-1xk

!/vk}.

 

Thus, an (approximately) unbiased estimator for the

model variance of t as an estimator for T is 

 v(2) = 3S ak

2

rk

(2)

, 

 

where 

rk

(2) = rk2

/{1 ! 2 xk (3S xi!xidi /Bi)

-1xk

!(dk /Bk) +

xk(3S xi!xidi /Bi)

-1[3S xi

!xi(di /Bi)

2

vi](3S xi!xidi /Bi)

-1xk

!/vk}.

Note that v(2) remains approximately unbiased

when O(1/n) is ignorably small even when Fk

2

 = kvk

fails providing that 

xk

 (3S xi

!xidi /Bi

)

-1xk

!(dk /Bk

) and 

xk (3S xi!xidi /Bi)

-1 [3S xi

!xi(di /Bi)

2

vi] (3S xi!xidi /Bi)

-1xk

are O(1/n). 

4. WHEN THE POPULATION IS LARGE AND Fk

2

IS NOT KNOWN UP TO A CONSTANT 

Let us rewrite rk = yk ! xkb as 

rk

 = ,k ! xkC, = ,k ! gk,, 

where (3S xi

!xidi /Bi

)

-1xk

!(dk /Bk

) is the kth column of 

C = {cpk}Pxn, , = (,1, ..., ,n)!, and 

gk = xkC = (gk1, ..., gkn)!. 

Now E(ri 

2

) = (1 ! 2gii)Fi 

2

 + 3k gik

2

Fk 

2

 or

E[ri 

2

/ (1 ! 2gii)] = Fi 

2

 + (1 ! 2gii)

-1

 3 gik

2

Fk 

2

 k0S

 

 P

 = Fi 

2

 + (1 ! 2gii)

-1 3 

[ 3 xip cpk]

2

Fk 

2 

 k0S p=1 (4)

 = Fi 

2

 + (1 ! 2gii)

-1 3 

[ 3 (2 ! *pp!)xipcpkxip!cp!k] Fk

2

.

 p$p!

Equation (4) can also be expressed in matrix

form as

E(r*) = F

2

 + QHF

2

, 

where r* = (r1

2

/ [1 ! 2g11], ... , rn

2

/ [1 ! 2gnn])!, 

F

2

 = (F1

2 

, ...,Fn

2

), Q = {qif}nxF, F = P(P+1)/2, 

qif = (1 ! 2gii)

-1(2 ! *pp!

)xipxip! (f corresponds to pp!),

H = {hfk}Pxn , and hfk = cpkcp!k. 

Observe that if every *cfk* # O(1/n), then each

*hfk* # O(1/n2).
Letting I(n) denote the nxn identity matrix, an

unbiased estimator for F

2

 is

 s2 = (I(n) + QH)

-1r*

when (I(n) + QH) is invertible (see Chew 1970). If

F < n, a more computationally convenient form is 

 s2 = {I(n) ! QH + (QH)

2 ! (QH)3

 + ... } r*

 = {I(n) ! Q[I(F) ! HQ + (HQ)

2

 + ... ]H}r* 

 = {I(n) ! Q[I(F) + HQ]

-1H}r*, 

which requires (I(F) + HQ) to be invertible. 

Thus, 

 vs = 3S ak

2

sk

2

, 

 

where s

2

 = (s1

2 , ..., sn2

), is an unbiased estimator for the

model variance of t as an estimator for T when relative

O(n/N) terms can be ignored. 

5. A SIMPLE EXAM PLE COMPARING rk

(2) AND

sk

2

Consider the following simple example.

Suppose P=1, so the vector xk reduces to the scalar xk.

In addition, let dk = 1/xk, a popular formulation. The

estimator t becomes the ratio estimator, 

t = 3U xk ( 3S yk /Bk) / ( 3S yk /Bk). 

The element-k residual has the form: 

rk

 = yk ! xk

( 3S yi /Bi

) / ( 3S yi /Bi

).

When we can assume Fk

2

 = kvk

, rk

(2) becomes

 rk

(2) = rk2 /{1 ! [2(xk /Bk

)/( 3S xi /Bi)] + 

 (xk

2 /vk

)( 3S vi /Bi

2

) / ( 3S xi /Bi)

2

}.

 

Observe that when Bk

 = xk

 = 3⁄4vk

, rk

(2) = nrk /(n !1).

Deriving sk

2

 is greatly simplified because

P = F = 1. The matrix Q becomes the column vector,

Q = (x1 

2

/(1 - 2 T1), ..., xn

2

/(1!2 Tn))!, where Tk = gkk =

(xk/Bk) / 3S (xi /Bi). Similarly, H becomes the row

vector, H = ((T1 /x1

)

2

, ..., (Tn /xn

)

2

). After some

manipulation we get

sk

2

 = [rk

2 /(1!2Tk

)] !

[xk

2

/(1!2Tk)][3S(Ti

ri /xi )

2

/(1!2Ti)]/[1+3S Ti 

2

/(1!2Ti)].

 

This is similar to what we would get from replacing the

vi

 in the above expression for rk

(2) by the corresponding

ri

2

 (the difference is O(1/n2)). That should not come as

a surprise. 

6. WHEN THE POPULATION IS NOT LARGE

AND Fk

2

 = xk(

Suppose we cannot assumed that O(n/N) terms

are ignorably small. If the element variances have the

form Fk

2

 = xk( for some not-necessarily-specified (,

then equation (2) can be rewritten as

 E[(t ! T)2] = 3 ak

2

Fk

2 ! 3 akFk2

 k0S k0S

 = 3 (ak

2

 ! ak)Fk

2

 , (5)

 k0S

because 3S akFk

2

 = 3S akxk( = 3U xk( = 3U Fk

2

. 

An (approximately) unbiased estimator for the

variance of t as an estimator would simply replace the

Fk

2

 in (4) with rk

2

, rk

(2), or sk2

 depending on what other

assumptions are being made. 

Observe that even when n is large, and we

choose vm1 = 3S (ak

2

 ! ak

)rk

2

 as the variance estimator,

it differs from the randomization estimator, vR, when

ak  1/Bk. The model-based and randomization

variance estimators are asymptotically equivalent under

mild conditions, however, because akBk

 = 1 + OP

(13⁄4n),

where P here denotes the probability space generated by

the random sampling. 

7. OTHER POSSIBILITIES

Suppose we can not assume that Fk

2

 = xk( for

some (. We can, however, assume that n is large.

Under mild conditions, 3S

akFk

2 / 3U Fk

2

 = 1 + OP

(3⁄4n).

Although we are still interested exclusively in model

expectations, we can nonetheless use this

randomization-based equality to establish the relative

size of terms when n is large. 

This equality provides alternative justification

for the variance estimators discussed in the last section.

It may appear that the replacement of Fk

2

 in equation (5)

by rk

2

 is the only sensible policy when n is large because

it is computationally the easiest. Suppose, however,

that O(n/N) = O(1/n), and we are willing to ignore

relative bias term of probability order n-3/2, but not of

order 1/n. It then becomes more reasonable to use rk

(2)

(when vk is assumed known) or sk

2

 (otherwise).

Finally, suppose neither the sample nor the

population is large, and we can not assume that Fk

2

 =

xk( for some (. This leave us no alternative better than

assuming some model structure for all the Fk

2

, fitting

that model with the in the sample, and then applying the

results to estimate Fk

2

 for those elements not in the

sample. 
8. SUMMARY AND DISCUSSION

We have essentially proposed three estimators

for the model variance of t and an estimator for T in

most situations: 

vm1 = 3S (ak

2

 ! ak)rk

2

,

 

 vm2 = 3S (ak

2

 ! ak)rk

(2), and

 

vm3 = 3S (ak

2

 ! ak)sk

2

.

The first is the simplest to compute, while the last is

nearly unbiased under the broadest range of

circumstances. 

All three of these variance estimators have the

same large-sample-size randomization-based properties

as vR

 when the sample is drawn using Poisson sampling.

 The randomization-mean-squared-error estimator, vR

,

itself is nearly (i.e., large-n asymptotically)

randomization unbiased under Poisson sampling.

Given a more general design, the weighted-residualmean-squared-error estimator of Särndal, Swensson,

and Wretman (1989) is 

vR

! = vR

 + 3k,i 0 S; ki (1 - BkBi /Bik)ak rk ai ri

.

 

For simple random sampling, this reduces to 

vR! = [n/(n !1)]{vR ! (1 ! n/N)(3S ak rk)

2 /n}, 

which is large-n asymptotically identical to vR

 when

3S ak rk

 < OP

(N). A similar argument can me made for

broader range of designs satisfying 

BkBi /Bik . n/(n ! 1) + O(1/N) 

when O(n/N) = O(1/n). 

Whatever general theoretical advantage vR!

offers over vR in terms of a potentially reduced

randomization-bias can be lost to increased

randomization variance resulting from the as many as

n(n ! 1)/2 distinct terms in vR! but not in vR. In quite a

few practical situations the model will come close to

holding, and the three model-based variance estimators

proposed above will not only estimate the model

variance of t as an estimator for T better than vR!, they

will estimate the randomization mean squared error of

t better as well.

From a purely model-based point of view, any

model-unbiased estimator for T of the form t = 3S ai yi

will satisfy the calibration equation. The three

estimators for the model variance of t as an estimator

for T retain the same properties as those discussed in

the text except that akBk

 = 1 + OP

(13⁄4n) can not be

assumed. In order to compute these model variance

estimators, however, we need define the sample

residuals. In Section 1, they are defined by 

rk

 = yk ! xk

(3S xi

!xidi /Bi

)

-1 3S xi

!yidi /Bi

, 

where the choice for the dk imply the ak. From a modelbased point of view, the two need not be related.

Indeed, given values for the ak

 that satisfy the

calibration equation, any choice for dk will do to define

the rk

. 

REFERENCES

BREWER, K.R.W. (1979). A class of robust sampling

designs for largescale surveys. Journal of the

American Statistical Association, 74, 911-915. 

BREWER, K.R.W. (1999). Cosmetic calibration with

unequal probability sampling. Survey Methodology, 25,

205-212.

CHEW, V. (1970). Covariance matrix estimation in

linear models. Journal of the American Statistical

Association, 65, 173-181.

DEVILLE, J-C. and SÄRNDAL, C-E. (1992).

Calibration estimators in survey sampling. Journal of

the American Statistical Association, 87, 376-382. 

ISAKI, and FULLER, W.A. (1982). Survey design

under the regression superpopulation model. Journal

of the American Statistical Association, 77, 89-96.

KOTT, P.S. (1990). Estimating the conditional

variance of a design consistent regression estimator.

Journal of Statistical Planning and Inference, 24, 287-

296. 

ROYALL, R.M. and CUMBERLAND, W.G. (1978).

Variance estimation in finite population sampling,

Journal of the American Statistical Association, 72, 

351-358.

SÄRNDAL, C-E, SWENSSON, B., and WRETMAN,

J. (1989). The weighted residual technique for

estimating the variance of the general regression

estimator of a finite population total. Biometrika, 76,

527-537, 

SÄRNDAL, C-E, SWENSSON, B., and WRETMAN,

J. (1992). Model Assisted Survey Sampling. New

York: Springer.