SSC Annual Meeting, June 1997

Proceedings of the Survey Research Section

A (PARTIALLY) MODEL-BASED LOOK AT JACKKNIFE

VARIANCE ESTIMATION WITH TWO-PHASE SAMPLES

Phillip S. Kott1

ABSTRACT

This paper focuses on a design-consistent regression estimator in which the “auxiliaries” are

estimated from a stratified cluster sample and the regression coefficients from an arbitrary

subsample of the original sample. The reweighted expansion estimator described in Stukel

and Kott (1997) is an example of such an estimator. Assuming that the target variable is a

linear function of the auxiliaries plus an error term, asymptotic properties for both this

estimator and the jackknife estimator of its mean squared error are developed. These

theoretical results are used to explain some of Stukel and Kott’s empirical findings, which

in turn shed light on the asymptotic underpinnings of the theoretical results.

KEY WORDS: Asymptotic; Bias; Double expansion estimator; Primary sampling unit;

Reweighted expansion estimator. 

 1. INTRODUCTION

This paper will focus on a two-phase designconsistent regression estimator for the mean of a

single target variable computed in the following

manner: first, a vector of covariate means is

estimated from a stratified cluster sample; then, a

vector of regression coefficients relating the target

variable to the covariates at the element level is

estimated from an arbitrary subsample of the firstphase sample. The estimator has the same form as

the conventional design-consistent regre ssion

estimator for a population mean of a target variable

except that a first-phase estimated mean for the

vector of covariates is used in place of the population

mean for a vector of auxiliary variables. 

We will assume a model in which the target

variable is a linear function of the covariates plus an

error term. The two-phase regression estimator and

an expression for its variance under a combined

randomization (for the first phase) and model (for the

second phase) framework is discussed in Section 2.

 A jackknife estimator for this variance is analyzed

in Section 3. Section 4 concentrates on the special

case of a two-phase regression estimator in projection

form. Some empirical results for the weighted

expansion estimator partially published in Stukel and

Kott (1997) are reviewed in light of this analysis.

Other results from the same empirical study are

discussed in Section 5, which extends the theoretical

treatment in earlier sections to ratios of two-phase

regression estimators. Section 6 provides a more

general discussion. 

2. NOTATION FOR THE TWO-PHASE

REGRESSION ESTIMATOR

W e need quite a bit of notation. Let i denote an

element in the population of interest, j a first-phase

primary sampling unit (PSU), and h ( = 1, ..., H) a

first-phase stratum. There are nh sampled PSU’s in

stratum h and n sampled PSU’s overall. The

population size (in number of elements) is M, while

the second-phase sample size is m. Let S(hj) denote

the set of elements in PSU j of stratum h. Let S

denote the entire first-phase sample of elements (an

element is in the first-phase sample if it is contained

within a PSU in the first-phase sample). 

Let w1i denote the first-phase expansion factor

for i; that is the inverse of the first-phase selection

probability of the PSU containing i. Let w2i denote

the second-phase expansion factor for i: when

element i is in the second-phase sample, w2i is the

inverse of the conditional probability of selecting i

for that phase (conditioned on the first-phase

sample); when i in not in the second-phase sample,

w2i = 0.

The two-phase regression estimator for the
population mean y0

 = 3

M yi /M we will be addressing

here has the form:

t = 3S w1iw2iyi /M +

 (3S w1ixi /M ! 3S w1iw2ixi /M)b, (1)

where xi denotes a row vector of K covariates, 

b = (3S ciw1iw2ixi'xi /M)

-1 3S ciw1iw2ixi

'yi /M, (2)

and the ci

 are arbitrary constant. 

In many applications, M is unknown.

Consequently, t in equation (1) is not a practical

estimator for a population mean. Observe, however,

that Mt is a practical estimator for a population total.

The two estimators have parallel properties. They

have identical relative biases and identical relative

mean squared errors. We focus here on t to simplify

the asymptotics.

If the ci

 in equation (2) have the form ci =

1/(xi8), where 8 is a column vector, then t can be put

in projection form:

tPROJ = ( 3S w1ixi /M)b

since

 (3S w1iw2ixi /M)b = 

 (3S w1iw2ixi /M)(3S ciw1iw2ixi'xi /M)

-1

 

 3S ciw1iw2ixi'yi /M 

 = (3S [ci8'xi

']w1iw2ixi /M)(3S ciw1iw2ixi

'xi /M)

-1

 3S ciw1iw2ixi

'yi /M

 = 8'(3S ciw1iw2ixi

'yi /M = 3S w1iw2iyi /M.

 

The reweighted expansion estimator in Kott ans

Stukel (1997) is in projec tion form as is the more

general two-phase regression estimator discussed

later in that paper. 

We assume the yi and xi are related by the

following model:

 yi

 = xi$ + ei

, (3)

where E(ei *{xk}), and E(eieg *{xk}) = 0 for i and g

from different PSU’s, while E(eieg *{xk}) is bounded

other-wise. This structure allows the elemental

errors ) the ei ) within the same PSU to be correlated

in an arbitrary manner. It should be noted, however

that if the second-phase of sampling uses a clustered

selection process, and second-phase clusters cut

across first-phase PSU’s, elements from the same

second-phase cluster but different PSU’s are assumed

not to have correlated errors.

For our purposes, the target of estimation, y0

, is

virtually identical to 3

M xi$/M = x0$. As a result, we

will treat x0$ as the target of estimation from now

on. 

The difference between tPROJ and x0$ is 

tPROJ ! x0$ = (x1 ! x0)$ + x1(b ! $), 

where x1 = 3S w1ixi /M. The model mean squared

error of tPROJ is then

Ee[(tPROJ ! x0$)]2 = [(x1 ! x0)$]

2

 + x1Vare(b)x1'.

The first term of this expression is the square of the

model bias of tPROJ . The expectation of this term

with respect to randomization (i.e., its design

expectation) is the randomization variance of x1$.

For our purposes, then the randomization-model

variance of tPROJ is 

VarRM(tPROJ) = Var1(x1$) + x1Vare(b)x1',

where the subscript 1 denote randomization inference

with respect to the first phase of sampling. The right

hand side of this expression differs from the model

expectation of the randomization mean squared of t

(or, equivalently, the randomization expectation of

the model mean squared error of t) in that the model

variance component, x1Vare(b)x1', is conditioned on

the realized first-phase sample. 

When t in equation (1) cannot be put in

projection form, the situation is a bit messier. The

model variance of t is Vare

{e2

 + (x1 ! x2

)b}, where

e2 = 3S w1iw2iei /M, and x2 is defined analogously.

The randomization component of the randomizationmodel variance of t is the same as that of tPROJ (i.e.,

Var1(x1$)). 

Let x1hj = 3S(hj) w1ixi /M; x2hj = 3S(hj) w1iw2ixi /M;

y2hj = 3S(hj) w1iw2iyi /M; qhj = 3S(hj) ciw1iw2ixi'yi /M; 

e2hj = 3S(hj) w1iw2iei /M; uhj = 3S(hj) ciw1iw2ixi

'ei /M; and

Zhj = 3S(hj) ciw1iw2ixi

'xi

/M. Just as x2

 = 3

H 3 j x2hj, let

q = 3

H 3 j qhj (since the subscript 2 is not needed for

clarification, it has been suppressed). Define u and

Z analogously. 

Now b = Z

-1q and Vare

(b) = Vare

(Z

-1u) =

Z

-1E(uu')Z-1

 = 3

H 3 j Z

-1E(uhjuhj')Z

-1

 = 

3 3 Var( Z

-1uhj). The model variance of t is thus 

Vare(t) = 3 3 Vare (e2hj + [x1 ! x2]Z

-1uhj)

 

 

 = 3 3 Vare (ahj),

where ahj = e2hj + [x1 ! x2]Z

-1uhj. For tPROJ, this

collapses to 3 3 Vare

 (x1Z

-1uhj).

The randomization component of the

randomization-model variance of t is 
 Var1

( x1$) = 3 3 Var(x1hj$).

We will assume either, 1, the first-phase sample was

drawn with replacement but that the population,

strata definitions, and design are such that an

element can almost never be selected more than

once, or, 2, the first-phase sample was drawn

without replacement but that the population, strata

definitions, and sample design are such that using

the with-replacement variance estimator has an

ignorably small bias. 

If $ were known, the standard with-replacement

variance estimator for Var1(x1$) is 

 var1(x1$) = 3

H

 (nn /[nh ! 1]) [ 3 j (x1hj$)

2 !

 ( 3 j x1hj$)

2

/nh]. 

 

We will call 

vI(t) = var1(x1$) + 3

H 3j

 Vare(ahj) (4)

the ideal estimator for the randomization-model

variance of t; that is

VarRM (t) = Var1(x1$) + 3

H 3j

 Vare(ahj). 

 

3. THE JACKKNIFE VARIANCE 

ESTIMATOR FOR t

Let f(hj) = f ! (nn/[nh ! 1])( fhj ! 3g fhg /nh), where f has

a linear form such as y2, x1, x2, e, q, u, or Z. The

expression f(hj) is called the hj’th jackknife replicate

of f.

Now t can be rendered as y2 + (x1 ! x2)Z

-1q. 

The hj’th jackknife replicate of t is 

t

(hj) = y2(hj) + (x1(hj) ! x2(hj))Z

-1

(hj)q(hj). 

The jackknife variance estimator for t is

vJ (t) = 3

H [(nh ! 1)/nh

] 3j (t(hj) ! t)2.

To evaluate this variance estimator, we need

first evaluate the differences t(hj) ! t: 

t

(hj) ! t = x2(hj)$ + e2(hj) + (x1(hj) ! x2(hj))Z(hj)

-1q(hj)

 ! x2$ ! e2 ! (x1 ! x2

)Z

-1q

 = (e2(hj) ! e2) + (x1(hj) ! x1)$ (5) 

 + (x1(hj) ! x2(hj))Z(hj)

-1u(hj) ! (x1 ! x2

)Z

-1u

We need some asymptotics to handle the Z-inverse

terms. We will assume that all the linear

expressions like fhj are Op(1/n), while f itself is no

more than OP(1). In fact, e and u are OP(1/on), since

each are the sum of n independent OP(1/n) terms.

Let f(hj) = f ! f(hj) = (nn /[nh ! 1])( fhj ! 3g fhg /nh),

so that 

Z(hj)

-1 = [Z ! Z(hj)]-1= [Z (I ! Z-1Z(hj))]-1

 = (I + Z

-1Z(hj) + Z-1Z(hj)Z-1Z(hj))Z-1 + Op

(1/n3).

Plugging Z(hj)

-1 and the definition of f(hj) into the left

hand side of equation (5) yields:

t

(hj) ! t = ! x1

(hj)$ ! e2

(hj) ! (x1 ! x2

)Z

-1u(hj)

 ! (x1

(hj) ! x2(hj))Z-1(u ! u(hj))

 + (x1 ! x2)Z

-1Z(hj)Z-1[u ! u(hj)] + Op

(n-5/2) 

 = ! x1

(hj)$ ! e2(hj) ! (x1 ! x2

)Z

-1u(hj)

 (6)

 ! {(x1

(hj) ! x2

(hj)) ! (x1 ! x2

)Z

-1Z

(hj)}Z

-1[u ! u

(hj)]

 + Op(n-5/2).

Dropping terms of order 0P

(n-3/2), we have

t

(hj) ! t . ! x1

(hj)$ ! e2(hj) ! (x1 ! x2

)Z

-1u(hj)

 = ! x1

(hj)$ ! a(hj)

 = ! {(nn /[nh ! 1])( x1hj$ ! 3g x1hg$/nh) 

 + (nn /[nh ! 1])( ahj ! 3g ahg /nh)}. (7)

So that 

 Ee

[(t(hj) ! t)2

] .

 [nh /(nh ! 1)]2

{(x1hj$ ! 3g x1hg$/nh)

2

 + 

 [(1 ! 2/nh

)Vare

(ahj) + 3g

 Vare

(ahg)/nh

]},

and

Ee[vJ(t)] = 3

H [(nh ! 1)/nh

] 3j Ee[ (t(hj) ! t)2

]

 . var1(x1$) + 3

H 3j

 Vare(ahj).

This last near equality tells us that the jackknife

is a good estimator for the randomization-model

variance of t discussed in the last section. In fact,

incorporating some of the higher order terms dropped

from equation (6), we can conclude that 

 Ee[vJ(t)] = var1(x1$) + 3

H 3 j Vare

(ahj) + O(1/n2)

 = vI(t) + O(1/n2), 

where vI(t) is the ideal variance estimator defined in

equation (4). If we make the additional mild

assumptions that the sampling design and population

are such that vI(t) and VarRM(t) are O(1/n), then the

relative bias of the jackknife relative to the “gold

standard” of the ideal variance estimator is O(1/n). 
4. THE JACKKNIFE FOR THE TWO-PHASE

REGRESSION ESTIMATOR

IN PROJECTION FORM

When the two-phase regression estimator can be put

in projection form; that is, when ci = 1/(xi8) for some

vector 8, equation (6) collapses to

t

(hj)PROJ ! tPROJ = ! x1

(hj)$ ! x1Z

-1u

(hj)

 ! (x1

(hj) ! x1Z

-1Z(hj))Z-1[u ! u(hj) ]

 + Op(n-5/2). (6')

This is because 8'cixi' = 1, so that x2Z

-1

 = 8', 

8'uhj = e2hj, and e2

(hj)

 = x2Z

-1u(hj) . In addition, 

8'Zhj = x2hj, so that x2

(hj)

 = x2Z

-1Z(hj)

. 

Denote E(uhjuhj') by Vhj. Recall that the uhj are

independent and u

(hj) =[nh /(nh ! 1)](uhj ! 3g uhg /nh

] ,

so that E(u

(hj)u) = [nh /(nh ! 1)](Vhj ! 3g Vhg /nh

) and

E(u

(hj)u

(hj)’) = [nh /(nh ! 1)]2

 (Vhj[1 ! 2/nh] +

3g Vhg /nh

2

). From (6'), we have 

 

Ee

[(t(hj)PROJ ! tPROJ)

2

] = (x1

(hj)$)

2

 + Vare

(x1Z

-1u

(hj)) 

 + 2(x1

(hj) ! x1Z

-1Z

(hj))Z

- 1

 Ee([u ! u

(hj)]u(hj)’)Z-1x1

'

 + (x1

(hj) ! x1Z

-1Z(hj))Z-1

 Ee

(uu’)Z

-1(x1

(hj) ! x1Z

-1Z(hj) )' 

 + O(n-7/2)

 

 . (x1

(hj)$)

2

 + Vare(x1Z

-1u

(hj)) 

 + 2(x1

(hj) ! x1Z

-1Z

(hj))Z

-1[nh /(nh ! 1)2

]

 (Vhj ! 3g Vhg /nh)Z

-1x1

' 

 + (x1

(hj) ! x1Z

-1Z

(hj))Z

-1(3 3 Vfg)Z

-1

 (x1

(hj) ! x1Z

-1Z

(hj))'. 

 

 Letting V denote 3 3 Vhj and V

(hj ) denote

[nh/(nh ! 1)](Vhj ! 3g Vhg /nh), we can express the

expected value of the jackknife variance estimator for

tPROJ as

 Ee(vJ[PROJ]) .

 var1(x1$) + Vare(x1Z

-1u) (8)

 + 23 (1/nh

) 3 (x1

(hj) ! x1Z

-1Z(hj))Z-1V(hj)Z-1x1

'

 + 3 3 ([nh ! 1]/nh

)(x1

(hj) ! x1Z

-1Z(hj))Z-1VZ-1

 (x1

(hj) ! x1Z-1Z(hj))'.

 

The asymptotic bias of the jackknife is captured by

the last two lines on the left hand side of equation (8).

For many applications, V will be roughly equal

to a multiple of Z. Observe that both x1

(hj) and x1Z

-

1Z(hj) have randomization expectations o f

(asymptotically) zero, but that the former is often

likely to be a good less variable because it is based

on the entire first-phase sample. Consequently, in

many applications the contribution to the asymptotic

bias of the jackknife from first-phase stratum h )

roughly proportional to 

([nh ! 3]/nh) x1Z

-1Z

(hj)Z

-1Z

(hj)Z

-1x1 ! will be negative

(positive) when nh

 is less (greater) than 3. 

One popular example of the two-phase

regression estimator in projection form is the

reweighted expansion estimator for a population total

explored in Stukel and Kott (1997). For this

estimator xi is a vector of group membership

indicators, where the groups are mutually exclusive,

and all the ci

 are equal to 1. Consequently, the

components of Mx1

 and Mx2

 are estimators of the

group population totals based on the first- and

second-phase samples respectively, Z is a diagonal

matrix with the same values of x2, and b is a vector of

estimates of the group y-means based on the secondphase sample. 

The reweighted expansion estimator in Stukel

and Kott had the form Mt = Mx1b. They also

inves-tigated the double expansion estimator, which

had the form Mx2b. Diane Stukel performed a

simulation in which, first, a with-replacement,

stratified, simple random sample of area clusters

(PSU’s) was drawn, then all the individuals from the

sampled clustered were restratified into 5 age groups,

and a without-replacement stratified, simple random

second-phase sample of individuals was drawn. Two

PSU’s were sampled from each of the 18 first-phase

strata, while second-phase stratum sample sizes ran

from 5 to 50 individuals. Four thousand (4, 000)

simulations were conducted for each sample size.

For comparison purposes, completely-enumerated

first-phase sample were also simulated. More details

on the data set and simulations are provided in Stukel

and Kott (1997).

The results of this empirical analysis for the

variable “total employment” are presented in Table 1.

For the jackknife of the double expansion estimator,

the replicate t

(hj) was set equal to y2(hj). A less

successful alternative formulation discussed in

Stukel and Kott is not presented here. Note that

when the entire first-phase sample is enumerated the

reweighted and double expansion estimators are the

same.

The reweighted expansion estimator appears to

have a trivial relative bias which increases in absolute

value as the second-phase sample size decreases.

The double expansion estimator is unbiased. The

tiny relative biases for this estimator in the Table are

due to the finite nature of the simulations. 

The reweighted expansion estimator is modestly

more efficient (has less mean squared error) that the

corresponding double expansion estimator. The

efficiency gain appears to increase as the secondphase sample size decreases. 
The jackknife variance estimates for the

reweighted expansion estimators have small negative

biases, which were anticipated by the theory

developed

 in this section. These biases tend to increase in

absolute value as the second-phase sample size

decreases. In theory, they should be an asymptotic

function of the first-stage sample size only. In

practice, the increasing relative variability of the Zhj

within first-phase strata ) and thus x1Z

-1Z(hj)Z-1Z(hj)

Z

-1x1

' (see equation (8) and the discussion following

it) ) may be the determining factor. Note that the

components of the diagonal matrix MnhZhj are

estimators of the number of individuals within firstphase stratum h from a particular age group (secondphase stratum) based on the second-phase sample in

PSU j of stratum h. The smaller the second-phase

sample size, the more variable the Zhj within strata

given fixed nh. 

The jackknife variance estimates for the double

expansion estimators have a strong upward bias. Am

explanation for this can be found in Kott (1990),

where using the standard two-stage, withreplacement variance estimator ) equivalent to the

jackknife in this case ) is shown to be biased

upward. 

Table 2 presents the results of another set of

simulations conducted by Stukel using the same data,

first-phase strata, and second-phase sample design as

in Stukel and Kott, but with 70 first-phase sample

PSU’s (out of 220). In these new simulations, 8 of

the strata have 4 or more sampled PSU’s. The other

10 again have 2.

The absolute relative biases of the reweighted

expansion estimator is a bit larger in the new

simulations but are still small. Surprisingly, the

efficiencies of the estimators go down (except , of

course, when the whole first-phase sample is

enumerated). 

The efficiency gains from using the reweighted

over the double expansion estimator are more

pronounced in the new simulations. a reasonable

explanation for this is that the precision of the

estimator x1 is greatly improved by adding PSU’s,

while the precision of x2 ) based a second-phase

sample that has not increased in size ) is not. 

The relative biases of the jackknife for the

reweighted expansion estimator remain small but now

are not all negative. This may be due to those strata

with more than 3 sampled PSU’s. 

))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

Table 1. Estimating Total Employment With a Two-PSU-Per-Stratum Design

Second-Phase Reweighted Expansion Estimator Double Expansion Estimator

 Stratum 

Sample Size % RelBias of Scaled RelBias of % RelBias of Scaled RelBias of 

 Estimate MSE* Jackknife Estimate MSE* Jackknife 

 All 0.04 100 0.94 0.04 100 0.94

 50 0.14 183 -0.99 0.16 186 46.4 

 20 -0.30 323 -2.51 -0.01 360 68.2

 10 -0.29 549 -5.81 0.03 632 78.2

 5 -0.56 1002 -5.13 0.12 1171 86.2

))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

Table 2. Estimating Total Employment With a Variable-PSU-Per-Stratum Design

Second-Phase Reweighted Expansion Estimator Double Expansion Estimator

 Stratum 

Sample Size % RelBias of Scaled RelBias of % RelBias of Scaled RelBias of 

 Estimate MSE* Jackknife Estimate MSE* Jackknife 

 All 0.11 59 -1.94 0.11 59 -1.94

 50 -0.15 186 -4.49 0.09 287 31.9 

 20 -1.12 366 -6.56 -0.34 620 41.1

 10 -1.51 622 -6.03 -0.06 1197 41.6

 5 -2.35 1088 3.57 0.14 2319 44.7

))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

 All results based on 4,000 simulations.

* Scaled so that original estimator based on the full first-phase sample has Variance (MSE) equal to 100. 
There does not appear to be reductions in

relative bias for a given second-phase sample size

from increasing number of PSU’s. One reason for

this may be the reduced first-phase variance, which

) all other things being equal ) increases the

potential contribution to relative bias from secondphase variance estimation. Another possibility is

that the Zhj have become more variable as the same

second phase sample size is distributed over move

PSU’s.

 Finally, the jackknifes for the double

expansion estimator have reduced, but still

unacceptably high, relative biases.

5. THE RATIO OF TWO-PHASE

REGRESION ESTIMATORS

Let us adopt the framework of Section 2 for two

variables of interest, yi

[1] and yi

[2]

. In particular for

k = 1 or 2, let y0

[k]

 = 3

M yi

[k]/M, 

t

[k]

 = 3S w1iw2iyi

[k]/M + 

 ( 3S w1ixi /M ! 3S w1iw2ixi /M)b

[k]

, 

and

 b[k] = (3S ciw1iw2ixi

’xi /M)-1 3S ciw1iw2ixi

’yi

[k]/M.

We assume that each yi

[k] is related to xi

 through the

model:

 yi

[k]

 = xi$

[k] + ei

[k]

, 

where the ei

[k] have mean zero and are uncorrelated

across PSU’s and bounded within PSU’s. It is not

necessary for ei

[1] and ei

[2] to be uncorrelated.

We are interested in the properties of

r = t[1]/t[2] as an estimator for y0

[1]/y0[2], which for

our purposes is indistinguishable from R =

x0$

[1]/x0$[2]

. Define yi as yi

[1] ! Ryi[2]

, and define ei

analogously. Defining other terms from Section 2

as before based on yi

, ei

, yi

[2], and ei

[2] above, we

have

r ! R = (t[2])

-1[x1

($

[1] ! R$

[2]) + 3 3 ahj]

 

 

 = (t[2])

-1[(x1 ! x0

)($

[1] ! R$[2]) + 3 3 ahj].

Under the asymptotic assumptions

analogous to those in Section 3, we have

 t[2]/x1$

[2] = 1 ! (x1$

[2])-1 3 3 ahj

[2] + 0p

(1/n). 

So that

Ee

[(r - R)2] = (x1$

[2])-2{[(x1 ! x0

)($

[1] ! R$[2])]2 

 + 3 3 Vare ahj}[1 + 0(1/n)]. (9)

If the population and first-phase sampling design

are such that 

Var1{(x1$

[2])

-1[(x1 ! x0

)($

[1] ! R$

[2])]} =

 Var1{(x0$

[2])-1[(x1 ! x0

)($

[1] ! R$[2])]}

 [ 1 + 0(1/n)], (10)

then a reasonable expression for randomizationmodel mean squared error of r is

MSERM(r) = {(x0$

[2])-2Var1

[x1($

[1] ! R$[2])] 

 + (x1$

[2])-23 3 Vare ahj}[1 + 0(1/n)].

Using similar arguments, we can show

that the jackknife variance estimator based on the

jackknife replicates r(hj) = t(hj)

[1]/t(hj)[2]

 has a model

expectation (given a realized sam ple )

asymptotically equal to

[x1$

[2]]-2var1

[x1

($

[1] ! R$[2])] + (x1$

[2])-2 3 3 Vare ahj,

which is itself asymptotically close to

[x0$

[2]]

-2var1

[x1($

[1] ! R$

[2])] + (x1$

[2])

-2 3 3 Vare ahj.

Thus, the jackknife provide a reasonable estimator

for the randomization-model variance of r. 

Table 3 is based on the same set of

simulations as Table 1 from the last section. Here,

the employment rate, the ratio of total employment

to the total number of individuals in the workforce,

is the target of estimation. 

Observe that the ratio of two double

expansion estimators is usually no less efficient

than the ratio of two reweighted expansion

estimators. Moreover, the jackknife provides

reasonable mean squared error estimates for both

the double and reweighted expansion estimators.

These two results have a simple explanation: 

($

[1] ! R$

[2]) in equations (9) and (10) must be

close to 0; that is, the employment rate must be

close to equal across second-phase strata. This

(near) equa lity (virtually) removes the

randomization component, which revolves around

the estimation of x0 with either x1 or x2, from the

mean squared errors of the two expansion

estimators. As a result, the jackknife need only

estimate the model variance of the two estimators,

which it does reasonably well in both cases. Its

negative bias for the double expansion estimator

can be shown to have an explanation analogous to

the reweighted expansion estimator for the
employment total. 

The absolute relative biases for the

jackknife in Table 3 seem larger than those in

Table 1. This may be due to the (near)

disappearance of the randomization component of

mean squared error increasing the potential for

contribution to relative bias in variance estimation

from the second phase of sampling.

Table 4 returns to the data set, sampling

design, and target as Table 1. Now, however, two

new estimators are considered. Both divide the

first-phase strata into four post-strata. Two of

these post-strata contain 2 strata, a third contains 4,

and the last 10.

Within each post-stratum, the ratio of total

employment to total number of individuals is estimated using the ratio of reweighted and double

expansion estimators, respectively (yj

[k] is set to 0

when i is out of the post-stratum). Census counts

for the four post-strata are used to weight the

estimated ratios together. This produces

estimators that are usually more efficient than

analogousreweighted expansion estimators (except

when the second-phase sample size is 5 elements

per stratum). 

The post-stratified estimators have more

noticeable, although still small, relative biases.

The post-stratified double expansion estimator is

less efficient than the post-stratified reweighted

expansion estimator and its jackknife variances

have pronounced upward bias. It seems that

within post-strata ($

[1] ! R$[2]) is not close to 0;

that is, the ratio of total employment to total

number of individuals varies across second-phase

strata.

The absolute relative biases for the

jackknife of the post-stratified reweighted

expansion estimator are the largest we have seen

for reweighted estimators. This is likely due to the

small number of PSU’s within some post-stratum

ratios, which severely challenges a theoretical

result based on asymptotics. 

Stukel conducted analogous simulations

to those summarized in Tables 3 and 4 based on 70

first-phase sample PSU’s. They offer little

additional insight, however, and will not been

reviewed here.

))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

Table 3. Estimating the Employment Rate With a Two-PSU-Per-Stratum Design

Second-Phase Reweighted Expansion Estimator Double Expansion Estimator

 Stratum 

Sample Size % RelBias of Scaled RelBias of % RelBias of Scaled RelBias of 

 Estimate MSE* Jackknife Estimate MSE* Jackknife 

 All -0.09 100 2.08 -0.09 100 2.08

 50 -0.09 314 -3.53 -0.08 314 -2.46 

 20 -0.31 663 -3.45 -0.27 662 -1.53

 10 -0.19 1261 -7.09 -0.12 1251 -5.21

 5 -0.26 2525 -6.55 -0.13 2516 -7.41

))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

Table 4. Estimating Total Employment With a Two-PSU Per Design and a Post-stratified Estimator 

Second-Phase Reweighted Expansion Estimator Double Expansion Estimator

 Stratum 

Sample Size % RelBias of Scaled RelBias of % RelBias of Scaled RelBias of 

 Estimate MSE* Jackknife Estimate MSE* Jackknife 

 All 0.06 33 3.30 0.06 33 3.30

 50 -0.08 117 4.88 -0.05 122 28.5 

 20 -0.93 273 6.42 -0.71 284 32.0

 10 -1.96 522 12.03 -1.67 541 35.3

 5 -4.44 1101 9.20 -3.98 1141 22.4

))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

 All results based on 4,000 simulations.

* Scaled so that original estimator based on the full first-phase sample has Variance (MSE) equal to 100. 
6. DISCUSSION

Kott and Stukel (1997) shows that the jackknife

provides an asymptotically unbiased estimator for

the randomization mean squared error of the twophase design-consistent regression estimator, at

least in projection form, when, 1, the second phase

of sampling is stratified, simple random sampling

at the element level, and 2, the covariates in the

regression estimator include indicator variables for

all the second-phase strata. 

By invoking the usual linear model, we

saw here that even without those two restrictive

cond itions, the jackknife pro vides an

asymptotically unbiased estimator of what we

called the randomization-model variance; that is,

the model variance of the two-phase designconsistent regressio n estima tor plus the

randomization expectation of its squared model

bias. The lone non-asymptotic restriction needed

is that the element errors be uncorrelated across

PSU’s. 

The randomization-model variance of a

two-phase design-consistent regression estimator is

related to more conventional measures of accuracy

in the following manner. The randomization

expectation of the randomization-model variance

is the randomization expectation of the model

mean squared error or, equivalently, the model

expectation of the randomization mean squared

error. 

The subject of the variance of the

jackknife has not yet been addressed. Employing

equation (7), the jackknife variance estimator for

the two-phase regression estimator can be

expressed as

 vJ

(t) = 3

H 3j

 (nn /[nh ! 1]){( x1hj$ ! 3g x1hg$/nh

) +

 ( ahj ! 3g ahg /nh)}2

 = 3

H 3j

 (nn /[nh ! 1])( x1hj$ ! 3g x1hg$/nh)

2

 + 3

H 3j

 (nn /[nh ! 1])( ahj ! 3gahg /nh)

2

 

 (12)

 + 2 3

H 3j

 (nn /[nh ! 1])( x1hj$ ! 3g x1hg$/nh)

 ( ahj ! 3g ahg /nh).

Let up call the expressions in the last three lines of

equation (12), a, B, and 2C, respectively . Even

allowing the simplifying assumptions that E(AC) =

E(BC) = Cov(a, B) = 0 and Var(C) = E(AB), we

have Var[vJ(t)] = Var(a) + Var(B) + 6Var(C). This

expression is difficult to analyze without specifying

a model for xi, which is beyond the scope of this

analysis. 

Recall that with the Stukel and Kott data,

the randomization-model variance of employment

rate estimator virtually had no randomization

component. As a result, the variance of the

jackknife is, for all intents and purposes, the

variance of B. Since the relative variance of B

tends to increase as the second-phase sample size

decreases, it is not surprising that Stukel and Kott

found that the coefficient of variation (CV) of the

jackknife of the ratio of two reweighted expansion

estimators increased as the second-phase sample

size per stratum decreased. With 50 individuals

per stratum the CV was 59.2%, with 20 individuals

65.7%, with 10 individuals 74.2%, and with 5

individuals per stratum 103.1%. The number are

quite similar for the ratio of two double expansion

estimators and for the set of simulations with 72

PSU’s. 

One surprising result in Stukel and Kott

was the 78.4% CV for the jackknife when the firstphase sample was completely enumerated. This

may be due to the impact on variance estimation of

with-replacement first-phase sampling. If the same

PSU was selected twice in a stratum, that stratum

only contributed to the variance estimate when the

first-phase sample was not completely enumerated.

This is likely the cause of the relative instability of

the jackknife for the fully-enumerated first-phase

sample. When Stukel increased the number of

PSU’s to 72, the CV shrunk to 30.4% for the fullyenumerated first-phase sample, while staying in the

same general neighorhood for the other sub-sample

sizes. 

This begs the question: if Stuckel’s

simulations contained double-hits of some PSU’s,

why do her results have any relevance to the

analysis in the text, which assumed away such a

possiblity? The reason they are relevant is because

the likelihood of double-hits at the element level

for the subsampling simulations is very small.

Hence, the model-based portion of the analysis in

the text applies as long as we assume independence

(or near independence) of the element errors, the ei,

across elements in the same PSU. 

It is a simple matter to extend the analysis

in the text to an estimator similar to t in equation

(1) but with a more complex covariate mean

estimator than x1 = 3S w1ixi /M. For example, the

estimator x1 may itself be a multi-phase designconsistent estimator. It may also incorporate

auxiliary variables whose population means are

known and do not have to be estimated. For

example, a better use of the post-stratum

population sizes in the Stukel and Kott data would
have been in the estimation of Mxi

, the vector of

age group population totals (i.e., second-phase

strata) based on the first-phase sample.

The randomization variance of x1$, a key

component of the randomization-model variance of

t, may itself be replaced by a randomization-model

variance or even by a model variance; for example,

by assuming the Mxi are have a common mean

within first-phase strata and are uncorrelated across

PSU’s. 

REFERENCES

Isaki, C.T. and Fuller, W.A. (1982). Survey

Design Under the Regression Superpopulation

Model. Journal of the American Statistical

Association, 77, 89-96.

Kott, P.S. (1990). Variance Estimation when a

First Phase Area Sample is Restratified. Survey

Methodology, 99-104.

Kott, P.S. and Stukel (1997). Can the Jackknife Be

Used With a Two-Phase Sample? Submitted to

Survey Methodology. 

Rao, J.N.K, and Shao, J. (1992). Jackknife

Variance Estimation With Survey Data Under Hot

Deck Imputation. Biometrika, 811-822.

Rust, Keith (1985). Variance Estimation for

Complex Estimators in Sample Surveys. Journal

of Official Statistics, 1, 381-397.

Sarndal, C.E., Swensson, B. and Wretman, J.H.

(1992). Model Assisted Survey Sampling. New

York: Springer-Verlag.

Singh, M.P., Drew, J.D., Gambino, J.G. and

Mayda, F. (1990). Methodology of the Canadian

Labour Force Survey: 1984-1990. Statistics

Canada publication, Catalogue 71-526. 

 

Stukel, D.M. and Boyer, R. (1992). Calibration

Estimation: An Application to the Canadian

Labour Force Survey. Statistics Canada Branch

Working Paper, SSMD # 009E.

Stukel, D.M. and Kott, P.S. (1997). Jackknife

Variance Estimation Under Two-Phase Sampling:

An Empirical Investigation. Statistics Canada

Branch Working Paper, HSMD - 97- # 009E.