i

The views expressed herein are not necessarily those of NASS or USDA. This report was

prepared for limited distribution to the research community outside the U.S. Department of

Agriculture.

USING THE DELETE-A-GROUP JACKKNIFE VARIANCE ESTIMATOR IN NASS

SURVEYS, by Phillip S. Kott, Research Division, National Agricultural Statistics Service, U.S.

Department of Agriculture, Washington, DC, March 1998, NASS Research Report, RD-98-01

(Revised July 2001).

 

ABSTRACT 

The National Agricultural Statistics Service (NASS) plans to use estimation strategies of increasing

complexity in the future and will need to estimate the variances resulting from those strategies. This

report describes a relatively simple method of variance/mean squared error estimation, the delete-agroup jackknife, that can be used meaningfully in a remarkably broad range of settings employing

complex estimation strategies. The text describes a number of applications of the method in abstract

terms. It goes on to shows how the delete-a-group jackknife has been applied to some recent NASS

surveys.

KEY WORDS

Calibrated weight; Multi-phase sampling; Poisson sampling; Ratio-adjusted weight; Replicate

weight; Restricted regression; Systematic probability sampling.

ACKNOWLEDGEMENTS

The author would like to thank Susan Hicks, John Amrhein, Matt Fetter, Carl Scott, and Don

McKinnie for their help in making the use of the delete-a-group jackknife at NASS a reality. Special

thanks go to Bill Iwig, Tapan Nayak, and Susan Hicks (again) for constructive comments on earlier

drafts of this report. 
ii

TABLE OF CONTENTS

ABSTRACT ................................................................ i

SUMMARY ............................................................... iii

INTRODUCTION ........................................................... 1

WHY USE THE DELETE-A-GROUP JACKKNIFE? ............................. 2

COMPUTING A DELETE-A-GROUP JACKKNIFE: AN OVERVIEW .............. 3

PARTICULAR CASES (IGNORING FINITE POPULATION CORRECTION) ...... 4

Stratified Simple Random Sampling ......................................... 4

Stratified Probability Sampling ............................................. 4

Restricted Regression (A Form of Calibration) ................................. 5

Ratio-Adjusted Weights (Another Form of Calibration) . ......................... 6

NASS Applications of Ratio-Adjusted Weighting ................................ 6

Composite Estimators .................................................... 7

SINGLE-PHASE POISSON SAMPLING AND FINITE POPULATION

CORRECTION ............................................................. 8

A Calibrated Estimator for a Total .......................................... 8

An Estimator for a Ratio .................................................. 9

Some Explanations and Extensions ......................................... 9

SUMMARY OF NASS USES (SO FAR) ....................................... 10

A DIGRESSION ON MODEL-BASED INFERENCE ............................ 11

CONCLUDING REMARKS ................................................. 13

JULY 2001 UPDATE ....................................................... 13

REFERENCES ............................................................ 14

APPENDIX A: Justifying the Delete-a-Group Jackknife Under a Single-Phase, 

 Stratified, Sampling Design ...................................... 15

APPENDIX B: Justifying the Delete-a-Group Jackknife for a Restricted-Regression

 Estimator Under a Two-Phase Sample .............................. 17

APPENDIX C: Justifying the Delete-a-Group Jackknife for Certain Composite 

 Estimators in the ARMS .......................................... 19

APPENDIX D: Justifying the Delete-a-Group Jackknife with Finite Population 

 Correction for a Single-Phase Poisson Sample ........................ 21
iii

SUMMARY

Historically, NASS has employed mostly expansion estimators and ratios of expansion estimators

based on stratified, simple random samples when computing indications of agricultural activity.

This is changing due in no small part to increasing demands on the agency to make more efficient

use of the information it collects. Fortunately, parallel increases in computing power are allowing

NASS to use more sophisticated estimation strategies involving multi-phase sampling designs and

calibration estimators. For example, the 1996 Agricultural Resource Management Study (ARMS)

used a multi-phase sampling design (a first-phase sample is randomly drawn, then a second-phase

subsample is randomly drawn from the first-phase sample, and so forth). Ratio adjustments of the

initial inverse-probability sampling weights capture relevantinformation from the ARMS screening

phase about farms not selected for a particular survey module.

This report shows how different variations of a delete-a-group jackknife can be used to estimate the

variances (or, more precisely, the mean squared errors) of a variety of estimation strategies. Many

of these are strategies are currently in use by NASS. 

The delete-a-group jackknife is simple to use once appropriate replicate weights are constructed.

By contrast, the “linearization” methods traditionally used by NASS for estimating variances can

be exceedingly complicated and cumbersome when applied to complex estimators strategies. The

advantage of the delete-a-group jackknife overthe traditional, delete-one-primary-sampling-unit-ata-time jackknife (see Rust 1985) is that the number of needed replicate weights per sample record

is kept manageable. 

A disadvantage of the delete-a-group jackknife over the delete-one jackknife is that it requires the

first-phase stratum sample sizes to be large ) at least five sample units per stratum. Otherwise, the

delete-a-group jackknife will be overly conservative; that is, higher, on average, than the true

variance it is measuring. As a result, when this jackknife is applied to estimators from the NASS

area frame, it will be biased upward. 

Like the delete-one jackknife, the delete-a-group jackknife is a nearly unbiased estimator of

variance only when the first-phase sampling fractions are small ) no more than 1/5 for most

records. Otherwise, the delete-a-group jackknife tends to be biased upward. This bias is likely to

be ignorable in most NASS applications. For the 1996 VCUS, however, it was so great that the

delete-a-group jackknife has to be modified. A potential modification is discussed in the text. It

is useful, but has a striking limitation: One set of replicate weights is needed when estimating the

variances of totals and another when estimating the variances of ratios. 
1

INTRODUCTION

This report addresses the construction of

delete-a-group jackknife variance estimators

for a variety of estimation strategies (an

estimation strategy is a sampling design

paired with an estimator). The emphasis

will be on computational formulae, which

will be rendered in fairly abstract form.

Relevant theoretical comments will be made

where appropriate, but most proofs are left for

the appendices. 

The sampling designs with which we will be

dealing may have any number of phases. At

each phase, one of the following selection

schemes is assumed to be used: 

1) stratified simple random sampling without

replacement, 

2) systematic probability sampling (usually

called systematic probability proportional to

size sampling; here we want to de-emphasize

the “size” measure),

3) the converse of systematic probability

sampling (what remains in a frame after a

systematic probability sample has been

removed), or

4) Poisson sampling (in which each element

is given its own selection probability, and the

sampling of one element has no impact on

whether another gets selected).

All stratum samples are assumed to be large

(contain at least five sampling units).

Violation of this assumption in the first-phase

of sampling can cause the delete-a-group

jackknife to be biased upward. This is shown

in Appendix A.

NASS currently incorporates two types of

calibration in its estimators and does not plan

to use any other types in the near future. 

“Calibration” is a general term for a

sampling-weight adjustment that forces the

estimates of certain item totals based on the

sample at one phase of sampling to equal the

same totals based on a previous phase or

frame (control) data. 

Ratio adjustments, the most common form of

calibration, were used repeatedly in the 1996

Agricultural Resource Management Study

(ARMS). Restricted regression, another

population form of calibration, was used in

both the 1997 Minnesota pilot Quarterly

Agriculture Survey (QAS) and the secondphase of the 1996 Vegetable Chemical Use

Survey (VCUS). Only these forms of

calibration are discussed in the text. 

Most of the results in this report are supported

with randomization-based (design-based)

analyses. As a consequence, all estimators of

population parameters are assumed to be

randomization consistent (i.e., have small

randomization mean squared errors and even

smaller randomization biases). A brief

discussion of the model-based properties of

the delete-a-group jackknife is reserved for a

separate section.

The concise term “variance estimation” will

be used throughout the text in place of the

more cumbersome “mean squared error

estimation.” It should be understood,

however, when the delete-a-group jackknife

is a good estimator for the variance of a

randomization-consistent estimator, it is also

a good estimator for its mean squared error.

For our purposes, the term “nearly unbiased”

will mean that the bias of the estimator in

question is an ignorably small fraction of its

mean squared error. The term “biased” will
2

be used to mean “not (necessarily) nearly

unbiased.” 

When first-phase stratum sample sizes are

large, the delete-a-group jackknife is

appropriate (has only a small potential for

bias) whenever the conventional,

randomization-based, delete-one jackknife is.

Kott and Stukel (1997) have extended the use

of the latter jackknife to two-phase estimators

with calibration in the second phase. This

report relies heavily on their results. Here,

however, systematic probability can be used

in the second design phase, even though Kott

and Stukel only treated strategies featuring

stratified simple random sampling in the

second phase. 

In most applications of the delete-a-group

jackknife at NASS the need for finite

population correction (fpc) is ignored. One

section of the text discusses a number of

those applications. 

A subsequent section takes up variance

estimation of totals and ratios when proper

fpc is a concern. Strictly speaking, the

variant of the delete-a-group jackknife that

captures fpc requires single-phase Poisson

sampling to be nearly unbiased.

Nevertheless, the practical application can be

broader, as we shall see. 

WHY USE THE DELETE-A-GROUP

JACKKNIFE?

The delete-a-group jackknife is simple to

compute once appropriate replicate weights

are constructed. The so-called “linearization” methods traditionally used by NASS

for estimating variances can be very

cumbersome when applied to estimators

based on multi-phase designs like the 1996

Vegetable Chemical Use Survey (VCUS)

1998) and components of the 1996 ARMS

(Kott and Fetter 1997). Estimators using

calibrated weights based on restricted

regression, like those calculated for the 1997

Minnesota pilot QuarterlyAgriculture Survey

(QAS), pose even greater practical problems

for linearization variance methods (a

multivariate regression coefficient would

need to be estimated for every item of

interest). 

It is also a relatively simple matter to apply

the delete-a-group jackknife to the composite

estimators associated with the ARMS. With

1996 survey data, for example, results from

the Phase II Corn Production Practices Report

(PPR) were composited with results from the

Phase II Corn-for-Grain Production Practices

and Costs Report (PPCR). In addition, results

from the Phase III Cost and Returns Report

(CRR) stand-alone (based on respondentsthat

were not in the Phase II PPCR sample) were

composited with results from the Phase III

CRR follow-on (based on respondents that

were). 

The advantage of the delete-a-group jackknife

over the traditional, delete-one-primarysampling-unit-at-a-time jackknife (see Rust

1985) is that the number of needed replicate

weights per sample record is kept

manageable. A common practice with the

delete-one jackknife for handling this

problem is to group primary sampling units

(PSU’s) into variance PSU’s. This practice

reduces the number of replicate weights

needed per record ) there is one for every

variance PSU. Nevertheless, NASS would

need at least 15 replicate weights per record

to compute variances for state estimators.

This would result in national variance

estimates employing several hundred

replicate weights per record.
3

COMPUTING A DELETE-A-GROUP

JACKKNIFE: AN OVERVIEW

Suppose we have a sampling design with any

number of phases and a randomizationconsistent estimator, t, we wish to apply to

the resultant sample. To compute a delete-agroup jackknife variance estimator for t, we

first divide the first-phase sample ) both

respondents and non-respondents ) into R

(jackknife) groups. Currently, R is 15 in

NASS applications. Consequently, we will

assume R = 15 in the text. By setting R at

15, we lengthen the traditional, normalitybased 95% confidence interval by ten percent.

To see why this is so, observe that the ratio

of the t-value at 0.975 for a Student’s t

distribution with 14 degrees of freedom and

the normal z-value at 0.975 is approximately

1.1.

Suppose we have a survey which may have

multiple phases. Let F be the sample selected

at the first phase of the sampling process.

The first-phase sample units may be

composed of distinct elements (e.g., farms) or

it may consist of clusters of elements (e.g.,

area segments). Many survey designs feature

a single phase of sample selection.

The delete-a-group jackknife begins by

dividing the first-phase sample F, into 15

groups. This can be done as follows: order F

in an appropriate manner (discussed below);

select the first, sixteenth, thirty-first, ... units

for the first group; select the second,

seventeenth, thirty-second, ..., units for the

second group; continue until all 15 groups are

created. Unless the number of units in F is

divisible by 15 (which is unlikely), the groups

will not all be of the same size.

Ordering in an “appropriate manner” depends

on the context. If F was drawn using

stratified random sampling, then order the

sample so that units in the same stratum are

listed together (i.e., contiguously). If samples

were drawn using Poisson sampling, order the

sample units randomly. 

 Let S denote the final respondent sample

used to compute t, and let wi denote the

sampling weight for element i in S. The

elements in S may be the same as the sample

units in F or they may be a subsample of

those units. The elements in S may also have

a different nature than the original sample

units in F; for example, they may be farms as

opposed to area segments or fields as opposed

to farms. In all such cases, however, each

element in S must be contained within an

original sample unit in F in a clearly defined

way. Let ei

 be the original sampling weight

of the unit containing i (which may be i

itself); that is, ei is the inverse of the unit’s

first-phase probability of selection. 

Let Sr denote that part of the final sample

originating in first-phase sample units

assigned to group r. The jackknife replicate

S(r) is the whole final sample S with Sr

removed. We similarly define F(r) as the set

of first-phase sample units not in r.

We need to create 15 sets ofreplicate weights

{wi(r)}, one for each r, in the following

manner: wi(r) = 0 for all elements in Sr; for

other elements, wi(r) will be close to (15/14)wi

but adjusted to satisfy calibration constraints

similar to those satisfied by wi

 (exactly how

to do this in a number of situations is the

subject matter of the following section).

Observe that a wi(r)-value has been assigned

to every element in S including those in Sr.

Now t is an estimate based on the sample S

calculated using the set of weights, {wi}. Let

t

(r) be the same estimate but with the member
4

of {wi(r)} replacing {wi}. The delete-a-group

jackknife variance estimator for t is 

vJ = (14/15) 3

15 (t(r) - t)2

. (1)

PARTICULAR CASES (IGNORING

FINITE POPULATION CORRECTION)

In this section, we see how the delete-a-group

jackknife can be fruitfully applied in a

number of estimation strategies where fpc

may be ignored; that is, when the first-phase

selection probabilities are all small (say less

than or equal to 1/5). 

One sampling design not discussed in detail

subsequently is stratified multi-stage

sampling, in which subsampling within each

primary (first-stage) sampling unit is

conducted independently of subsampling in

other primary sampling units. When the first

stage of sampling has ignorably small

selection probabilities, the conventional

variance estimator for a stratified multi-stage

sample looks exactly like that for a stratified

single-stage cluster sample with estimated

totals for primary sampling units used in

place of actual values. As a result, when a

delete-a-group jackknife is appropriate for an

estimator based on a stratified single-stage

sample, it is appropriate for an estimator

based on a stratified multi-stage sample. 

Stratified Simple Random Sampling

Suppose we have a single-phase stratified

simple random sample without any

nonresponse (handling nonresponse will be

discussed later). The original and final

sampling weight for a unit i in stratum h is ei

= wi

 = Nh /nh

, where Nh

 is the population size

of stratum h and nh is its sample size. 

Let us now consider the r’th set of replicate

weights. For a unit i in S(r) and stratum h,

ei(r) = (15/14)Nh /nh. By contrast, the

appropriate final r’th replicate weight for unit

i recognizes the calibration equations

inherent in the direct expansion estimator

(i.e., Nh = 3j0S(r)1h wj(r) for all h). It is wi(r) =

Nh

/nh(r) = (nh /nh(r))ei

, where nh(r) is the number

of sample units in both S(r) and h. Observe

that ei(r) = wi(r) only when nh is divisible by 15.

Stratified Systematic Probability Sampling 

Suppose we have a single-phase, stratified

systematic probability sample. The original

and final sampling weight for a unit i in

stratum h is ei = wi = Mh /(nhmi), where mi is

the measure of size of unit i in stratum h, Mh

is the sum of the mj across all units in stratum

h, and nh

 is the stratum sample size. 

Analogous to the simple random sampling

case, the appropriate final r’th replicate

weight for element i recognizes the

calibration equations inherent in the HorvitzThompson expansion estimator (i.e., Mh

 =

3j0S(r)1h wj(r)mj for all h). It is wi(r) =

(nh

/nh(r))ei

, where nh(r) is the number of sample

units in both S(r) and h. 

Stratified simple random sampling can be

viewed as equivalent to a special case of

systematic probability sampling from

randomly-order lists (one in which mi

 is

constant within strata). Appendix A provides

some theoretical justification for using the

delete-a-group jackknife as described above

with a stratified, single-phase systematic

probability sampling design under certain

conditions. One of those conditions is that

the systematic samples be drawn from

randomly-ordered lists. Variance estimation

can be problematic when systematic samples

are drawn from purposefully-ordered lists. 

Purposefully-ordered lists can reduce the

variance in estimators based on systematic
5

samples. Unfortunately, the reduction in

variance due to a well-designed ordering

usually can not be measured in an effective

manner.

Restricted Regression 

(A Form of Calibration) 

There are many versions of restricted

regression. Below is a description of a

method similar to what was used in the 1996

VCUS and 1997 Minnesota pilot QAS. The

version presented here will likely be used in

the future. 

Suppose, for exposition purposes, there are

two sampling phases. Suppose further that

the second phase sample is calibrated to a row

vector of totals, 0, based on estimates from

the first-phase sample or determined from the

frame itself. 

Let fj

 be the weight for element j after the first

phase of sampling, and let pj be the element’s

selection probability in the second sampling

phase. In the absence of non-response (again,

nonresponse will be dealt with later) in the

second sampling phase, a general form of the

calibrated weight for j under restricted

regression is

wj

 = fj /pj

 + 

 (0* ) 3i,S* [fi /pi]xi)

 ( 3i,S* [fi/pi]xi'xi)

-1 [fj /pj

]xj' (2)

 

for i 0 S*, and a predetermined value

otherwise (chosen so that wj is not too small

or too far from fj /pj), where S is the secondphase sample, S* a subset containing almost

all the elements of S, xi

 is a row vector of

covariates whose sum across all elements in

the population is either 0 or has been

previously estimated to be 0 ) that is, 0 =

3Ffixi, where F denotes the elements in the

first-phase sample; finally, 0* = 0 ) 3S-S* wixi

.

Let fj(r) be the r’th jackknife replicate weight

for unit j after the first sampling phase. The

r’th jackknife replicate weight for element j

is 0 when j0Sr ; otherwise, it is 

wj(r) = wj

[fj(r) /fj

] +

 (0(r) ) 3i,S(r) wi[fi(r) /fi]xi) (3)

 ( 3i,S(r) wi[fi(r) /fi]xi'xi)

-1 wj

 [fj(r)/fj]xj', 

where 0(r) = 0 when 0 has been determined

from frame; 0(r) = 3F fi(r)xi when 0 has been

estimated from the first-phase sample.

Equation (3) is not the standard way to

construct jackknife replicate weights. The

expression wk[fk(r) /fk] has been used in place

of the more common fk(r) /pk, with which it is

nearly equal (because wk . fk /pk

). Equation

(3)’s strength is that it forces the replicate

weights (for elements not in group r) to be

fairly close to the associated calibrated

weights. This appears to reduce the upward

bias that unexpected differences between the

two can cause. It should be noted that any

such upward bias is small; in fact, it is

asymptotically ignorable. We live, however,

in a finite world. 

Restricted-regression as described above can

be done at any phase of sampling. At the t’th

phase, fi in equation (2) becomes the weight

for element i at the t-1'th phase and pi

 the

element’s conditional selection probability at

the t’th phase. For a single-phase restrictedregression estimator, we can set all pi

 =1 in

equation (2). 

When the phase of sampling calibrated in this

manner contains more than a single stratum,

the jackknife can have an upward bias (see

Appendix B). In addition, for a single-phase

Poisson sample, xi8 = 1 must hold for some 8

(see the section on Poisson sampling and

Appendix D).
6

Ratio-Adjusted Weights

(Another Form of Calibration)

Consider, again, a two-phase sample with fi

and pi as above. A very common form of

calibration occurs when a vector of covariates

for element i, xi

, is defined in such a way that

only one component of the vector is non-zero

for each i. That is to say, the elements are

categorized into G mutually exclusive

calibration (or ratio-adjustment) groups, and

xig >0 only when element i is in group g;

otherwise, xig = 0. 

Under that structure, a ratio-adjusted weight

for an element j in group g is 

wj

 = 0g ( 3i,S [fi /pi

]xig)

-1[fj /pj

], (4)

and 0 = (01, ..., 0G). Similarly, the

corresponding replicate weight is 0 for

j0Sr , and 

 wj(r) = 0g(r) ( 3i,S(r) fi(r) /pi

]xig)

-1[fj(r) /pj

] (5)

otherwise, where 0(r) = (01(r), ..., 0G(r)). 

If the second-phase sample is stratified, and

more than one of these strata are contained

within a calibration group, then the jackknife

can have an upward bias (see Appendix B).

When the second-phase sample is unstratified

or the second-phase strata and ratioadjustment groups coincide, the delete-agroup jackknife is nearly unbiased. In the

1996 ARMS and 1996 VCUS, second-(and

later-)phase sampling was unstratified.

Extensions of these results to estimation

strategies with t > 2 phases are straightforward; the fi

 in equation (4) and fi(r) in

equation (5) become the weight and replicate

weight at the t-1'th phase. For a single-phase

sample, we can set all the pi

 equal to 1 in both

equations (4) and (5).

The establishment of the appropriateness of

the delete-a-group jackknife forratio-adjusted

estimators parallels that of restrictedregression estimators, which is outlined in

Appendix B.

NASS Applications of Ratio-Adjusted

Weighting

One way to handle nonresponse is to treat the

set of responding elements (at any phase of

the design) as a stratified simple random

subsample of the selected sample. This was

essentially what was done in the first-phase of

the 1996 VCUS. All the original sample

elements (respondents and nonrespondents)

were assigned to jackknife replicates, and

nonresponse was treated as a second phase of

sampling. The “second-phase” strata and

calibration groups coincided with the original

stratum definitions, and xig was set equal to 1

when i was in group g (0, otherwise). Since

fi

 was equal for all i in the same stratum, and

pi was likewise identical for each respondent

i in the stratum, wi in equation (4) collapsed

to the population size in the stratum

containing i divided by the respondent sample

size in that stratum. Equation (5) collapsed

similarly.

In the 1996 ARMS, a stratified simple

random screening sample of farms was

subsampled sequentially for several mutually

exclusive survey modules (see Kott and Fetter

1997). Farms were selected for the Phase II

SoybeanPPR in Nebraska, for example, using

an additional five phases of sampling (to be

selected for this module, a respondent farm

from the screening sampled had to avoid

being subsampled for one of the four modules

preceding it). Each of these phases employed

unstratified systematic probability sampling

from a purposefully-ordered list (the theory in

Appendix B is assumes a randomly-ordered

list; if anything, purposeful ordering should
7

reduce mean squared errors and contribute an

upward bias to the delete-a-group jackknife).

Finally, a field was randomly selected from

each sampled soybean farm. 

The separate-ratio estimator in equation (4)

was used twice in Phase II Soybean PPR

estimates. It was used to ratio adjust the

weights for the screening-sample respondents

to the frame total-value-of-sales within every

screening stratum (notice tha t

response/nonresponse on the screening survey

is treated here implicitly as another phase of

sampling). In addition, the soybean field

sample was divided into three size groups.

Here, 0g was the total soybean acres in

calibration (size) group g as estimated from

the screening sample with the weights

described above, and pi

 was the product of the

six (conditional) probabilities: the

probabilities that the farm containing field i

was not selected for the four modules

preceding soybeans, the probability that this

farm was selected for the production practices

module, and the probability that field i was

subsampled from the farm. 

We treated the fields from which we collected

Phase II PPR information as if they were a

stratified simple random subsample of the

selected fields, where the three calibration

groups served as strata. This had no practical

effect on the calculation of the pi (observe

that if all the pi

 in a group are multiplied by

the same factor, the computed weights in

equations (4) and (5) are unchanged). 

Composite Estimators

Consider a set of C distinct samples, each of

which can be used to estimate a common

target value. Let S denote the combined

sample, and wi

(c) denote the weight for

element i in original sample c. If i is not in

sample c, set wi

(c) = 0. A composite

estimator t uses the set of weights {wi},

where each wi

 = 3

C

8cwi

(c) and 3 8c

 = 1.

To estimate the variance of t, we can create

15 sets of replicate weights for every wi

(c) and

denote each by {wi(r)

(c)}. We then estimate the

t

(r) using wi(r) = 3

C

8cwi(r)

(c) and compute vJ

using equation (1). 

Composite estimation was used, for example,

to combine the Phase III Beef and Corn-forGrain CRR follow-on samples in the 1996

ARMS with the Phase III CRR stand-alone

sample. First the two enterprise CRR

samples were composited and then this

combined sample was composited with the

other CRR sample (see Kott and Fetter 1997).

Samples being combined need not correspond

to identical target populations. For example,

the population of list farms with corn for

grain in 1996 is not the same as the

population of list farms with ten weaned

calves in 1996 (the Beef CRR population).

When combining CRR samples, we also

combine target populations; in this case, to

the set of all list farms with either grain corn

or ten weaned calves in 1996. Only those

sample farms having both corn for grain and

at least 10 weaned calves are assigned

composite weights as described above. Other

farms in the combined sample retain their precomposite weights. 

Appendix C shows why the delete-a-group

jackknife works for the composite estimators

used in the ARMS in which the components

were separate modules based on the same

screening sampling. Composite estimation

was also used in the ARMS to combine

independently drawn samples like the Phase

II Soybean PPR sample and the National

Resource Inventory sample. Here, like a

conventional jackknife, when the delete-a-
8

group jackknife is appropriate for each

independent component, it is also appropriate

for any linear combination of the components.

SINGLE-PHASE POISSON SAMPLING

AND FINITE POPUL ATIO N

CORRECTION

In this section, we restrict our attention ) at

first ) to a single-phase Poisson sample of

elements. Let Bj

 be the selection probability

of element j. We assume there is no

nonresponse. 

The versions of the delete-a-group jackknife

developed in this section will contain finite

population corrections. The versions are

different for an estimator of a total and the

estimator of a ratio. This is a reflection of the

fact that a simple formula like equation (1)

does NOT work for all smooth

transformations of calibrated expansion

estimators when finite population correction

is an issue (note: a “smooth” transformation

has first, second, and third derivatives; most

statistics of interest are smooth

transformations of expansion estimations, the

major exception being percentiles). 

A Calibrated Estimator for a Total

Suppose we have a calibrated estimator for a

total, t = 3S wjyj, where 

wj

 = 1/Bj

 +

 (0* ) 3i,S* [1/Bi]xi) (6)

 ( 3i,S* [1/Bi

]xi

'xi

)

-1 [1/Bj

]xj

' 

 

for j 0 S*, and a predetermined value

otherwise (chosen so that wj $ 1 and, perhaps,

not too far from 1/Bj

), S is the sample, S* a

subset containing almost all the elements of

S, xi

 is a row vector of covariates whose sum

across all elements in the population is 0, and

0* = 0 ) 3S-S* wixi. There must also be a

vector 8 such that xj8 = 3⁄4(1 ) Bj) for all j (that

is to say, either a component of xj

 or a linear

combination of components must equal

3⁄4(1 ) Bj)). Since we are dealing with a

single-phase sample, (6) is simply equation

(2) with 1/Bk replacing fk /pk (i.e., fk in

equation (2) is 1, while pk is Bk). 

To estimate the variance of t, we use equation

(1) but replace t with t(v) = 3S wj

(v)yj

, and t(r)

with t(r)

(v) =3S wj(r)

(v)yj

, where 

 wj

(v) = wj3⁄4(1 ) 1/wj

), (7)

and

wj(r)

(v) = wj

(v){1 + (3S wi

(v)xi ) 3S(r) wi

(v)xi

)

 (3S(r) wi

(v)xi

'xi

)

-1xj

'} (8)

 

 

when j 0 S(r) and 0 otherwise. Appendix D

outlines why this works. 

Observe that wj

(v)

. wj3⁄4(1 ) Bj

), so that

wj

(v)

. wj when the selection probability for

element j is ignorably small. When all

element selection probabilities are very small,

there is little difference between this delete-agroup jackknife for a total estimator with

finite population correction, vJ(fpcT), and the

standard delete-a-group jackknife, vJ.

Moreover, the rather odd assumption that

there exists a 8 such that xj8 = 3⁄4(1 ) Bj)

becomes close to the more standard

assumption that either a component of xj

 or a

linear combination of components is a

constant (i.e., xj8 = 1 for some 8). 

In fact, if we were to ignore finite population

correction (which we can do for most

surveys, but not VCUS), we could estimate

the variance of t with equation (1), replacing

equation (8) with
9

wj(r) = wj{1 + (3S wixi ) 3S(r) wixi)

 (3S(r) wixi'xi)

-1xj

'} (8')

 

when j 0 S(r) and 0 otherwise as long as xj8 =

1 for some 8. This is what we did for the

1997 Minnesota pilot QAS (see Bailey and

Kott 1977).

An Estimator for a Ratio

Suppose tR is an estimator for a ratio of the

form, tR = 3S wjyj /3S wjzj , where wj is

calibrated as above. One can estimate the

variance of t with 

vJ(fpcR) = (3S wj

(v)zj /3S wj

zj)

2

 (14/15)3

15 (tR(r)(v) - tR(v))2

, (9)

where tR

(v) = 3S wj

(v)yj /3S wj

(v)zj

, and tR(r)

(v) =

3S wj(r)

(v)yj / 3S wj(r)

(v)zj

. This assumes xj8 =

3⁄4(1 ) Bj

) for some 8. Even without this

assumption holding, in fact, even without

calibration, vJ(fpcR) will likely be a reasonable

variance estimator; as we shall see.

Alternatively, we could estimate the variance

of tR

 ignoring finite population correction

with equation (1). We need not assume that

xj8 = 1 for some 8. In fact, the wi

 need not

even be calibrated in this case (to see why,

observe that multiplying all the weights in tR

by a fixed constant so that 3S wj equals the

population size has no effect on the

estimator; consequently, all ratio estimators

are effectively calibrated on xj

 = 1).

Some Explanations and Extensions

Consider a single-phase element sample that

is not necessarily Poisson. Suppose we wish

to estimate the variance of t = 3S wjyj, where

the wj

 satisfy equation (6). Let uk = yk )

xk(3U xi'xi)

-13U xi

'yi, where U denotes the

population. The variance of t is

approximately

V = 3U uk

2

(1 ) Bk)/Bk +

 3U(ki) ukui

(Bki ) BkBi

)/(BkBi

). (10)

Under Poisson sampling the joint selection

probability of k and i, Bki, is equal to the

product BkBi

, and so V collapses to

3U uk

2

(1 ) Bk)/Bk. This can, in principle, be

estimated by 3S uk

2

(1 ) Bk)/Bk

2

 , which is

approximately equal to 3S[wkuk3⁄4(1) 1/wk)]2 ,

which is what vJ(fpcT) estimates (see Appendix

D). 

A similar argument can be made for the ratio

estimator, tR = 3S wjyj /3S wjzj, except that now

V becomes approximately 

V

*

 = (3S wj

zj

)

-2[3U (uk*)2

(1 ) Bk

)/Bk

 +

 3U(ki) uk*ui*(Bki ) BkBi)/(BkBi)],

where 

uk* = uk

+

) xk (3P xi'xi)

-1 3P xi

'ui

+

, and

uk

+

 = yk ) (3U yi /3U zi

) zk .

Under Poisson sampling, V*

 collapses to 

(3Swjzj)

-2[3U

(uk*)2

(1 ) Bk)/Bk]. If we tried to

compute a delete-a-group jackknife with

equation (1) replacing t by tR

(v) and t(r) by

tR(r)

(v)

, we would get a reasonable estimator for

(3S wj

(v)zj

)

-2[3U (uk*)2

(1 ) Bk)/Bk] rather than

V

*

, hence the factor (3S wj

(v)zj /3Swj

zj)

2

 on the

right hand side of equation (9).

This factor is unnecessary if finite population

correction is ignored. In fact, since 3U ui

+

 = 0

(simplifying the proof in Appendix D), the

weights need not be calibrated for the deletea-group jackknife variance estimator for tR

 to

be nearly unbiased.

Calibrated estimators of totals were computed

in the 1997 Minnesota pilot QAS. Sampling

was not exactly Poisson due to the need to

combine some samples and subsample others

(see Bailey and Kott 1997). Nevertheless, it is
10

not unreasonable to assume that 

3U(ki) ukui

(Bki ) BkBi

)/(BkBi

) in the right hand

side of equation (10) is roughly zero and then

) ignoring finite population correction ) use

vJ to estimate variances. 

It is of interest to note that for systematic

probability sampling from an purposefullyordered list, Bki will often be zero when i and

k are listed sequentially in the ordering. If uk

and ui in equation (10) tend to have the same

sign when i and k are listed together, then it

is likely that 3U(ki) ukui(Bki ) BkBi)/(BkBi) will

be negative ) reducing the variance of t. The

delete-a-group jackknife does not capture this

variance-reducing phenomenon, however.

That is why it was claimed earlier that the

use of systematic unequal probability

sampling from an ordered list will, if

anything, bias the delete-a-group jackknife

upward. This presupposes that elements (or

units) listed together in the ordering are in

some sense similar. 

Remember the delete-a-group jackknife for

an estimated total with finite population

correction, vJ(fpcT), is only appropriate when

there is no nonresponse. Still, computing

vJ(fpcT) and vJ using imputed values in place of

real ones can provide a means of evaluating

the impact of high selection probabilities on

variance. There is one additional caveat.

When one does not require there exists a 8

such that xj8 = 3⁄4(1 ) Bj

) for all j, then vJ(fpcT)

may be biased downward. This possibility is

likely to be remote in practice (see Appendix

D).

We could have used vJ(fpcR) to estimate

variances from the 1996 VCUS. In theory,

this might not be appropriate since the

calibration in that survey was to first-phase

totals rather than to control totals. Moreover,

we did not require that there be a vector 8

such that xj8 = 3⁄4(1 ) Bj) for all j. It is unlikely

that either failing would cause much bias in

vJ(fpcR). This is because calibration does little

to reduce the variance of t in the VCUS.

Moreover, it is likely that 3S wi

(v)ui* (or

3S wi

(v)ui

+

 when vJ(fpcR) is used with an uncalibrated VCUS estimator) is close to zero

even when there is no 8 such that xj8 = 3⁄4(1 )

Bj

). Appendix D explains why 3S wi

(v)ui

*

must be near zero. 

SUMMARY OF NASS USES (SO FAR)

The delete-a-group jackknife was used to

estimate variances for the 1996 ARMS, 1996

VCUS, and 1997 Minnesota pilot QAS. It has

also been used in some of NASS’s foreign

consulting work, but that is beyond the scope

of this report. 

Bailey and Kott (1997) describes the sample

design used in the Minnesota pilot QAS. 

Since NASS imputes for nonresponse on the

QAS, there was essentially a single-phase

sample in Minnesota. Equation (2), with all pj

set equal to 1 and fj

 computed as described in

Bailey and Kott, was used to generate most of

the calibration weights. The vector xi had 20

components including a constant term. 

When a wj calculated with equation (2) would

have been less than 1, farm j was removed

from S*, and wj

 was set equal to 1. Sampled

farms were randomly assigned to jackknife

groups, and replicate weights were calculated

using the more conventional 

wj(r) = [15/14]fj

 + 

 (0(r) ) 3i,S(r) [15/14]fixi)

 ( 3i,S(r) [15/14]fixi'xi)

-1 [15/14]fjxj

' 

 = fj + (0(r) ) 3i,S(r) fixi)

 ( 3i,S(r) fixi

'xi

)

-1 fjxj

' 
11

instead of equation (3). This was because

the advantages of using the latter equation

was not clear at the time.

The 1996 VCUS had a two-phase design.

The first phase was stratified simple random

sampling. Sampled units were randomly

assigned to jackknife groups within firstphase strata. Nonresponse to the first design

phase of the VCUS was treated as an

additional phase of stratified simple random

sampling where the strata were the same as

the first-phase. 

The second design phase of the VCUS used

systematic unequal probability sampling.

Nonresponse to this phase was treated as an

additional phase of simple random sample.

Equation (2) was used to compute calibrated

weights. The “first-phase” weight, fj

, was

actually the population size of the first-phase

stratum containing i divided by the number

of first-phase usables in the stratum; 0 was a

vector of estimated planted-acre totals for inscope vegetables based on the first-phase

sample adjusted for nonresponse; pi was the

second-phase probability of selection

multiplied by the number of second-phase

usables and divided by the number of

second-phase sample farms. Replicate

weights were computed using 

 

wj(r) = wj

 + (0(r) ) 3i,S(r) wixi

)

 ( 3i,S(r) wixi

'xi

)

-1 wjxj

', 

which turned out to have slightly better

empirical properties (less negative values;

values closer to wj) in this context than those

produced by equation (3) for some reason . 

The many uses of ratio-adjustment and

composite estimation in list-based estimates

from the 1996 ARMS are discussed

thoroughly in Kott and Fetter (1997). The

original screening sample was randomly

allocated into jackknife groups on a stratumby-stratum basis. The text provides some

details for a couple of examples. See Kott and

Fetter for more. 

The delete-a-group jackknife was also used

for the non-overlap (area) portion of the 1996

ARMS for economic statistics. The area

design had effectively three-phases: 1) a

stratified simple random sample of area

segments, 2) a restratified simple random

subsample of farms; and 3) a stratified (using

the first-phase strata) simple random

subsample of respondents. Using the delete-agroup jackknife in this context treats the threephase sample as if it were a three-stage

sample. As a result, the variance estimator

can be biased upward (see Kott 1990). The

problem here is that the second-phase sample

is not calibrated in any way. 

There is an additional source of upward bias

in the delete-a-group jackknife applied to the

1996 ARMS non-overlap sample. Some area

substrata have very small samples sizes (less

than five areas segments). Collapsing 

substrata into land-use strata helped some, but

on occasion even land-use strata had small

sample sizes. Appendix A shows why this

can cause a bias in the delete-a-group

jackknife.

A DIGRESSION ON MODEL-BASED

INFERENCE

The delete-a-group jackknife can be applied 

to estimate variances in a reasonable fashion

under a variety of complex estimation

strategies. Both the text so far and the

appendices rely exclusively on the principles

of randomization-based inference. As a result

of this, we were forced to assume two number

of questionable or erroneous assumptions:
12

1) systematic probability sampling is

conducted by NASS from randomly-ordered

lists, and

2) farm in the same ratio-adjustment

(calibration) group are equally likely to

respond to a survey. 

These assumptions would not be necessary if

we replace them by the model assumptions

behind calibration; namely; 

yi = xi$ + ,i,

where the ,i have zero mean, bounded

variances, and are uncorrelated ) at least

across first-phase sampling units. 

For example, considerthe difference between

the calibration estimator, 

t = 3S wiyi

, and its target, T = 3U yi : 

t ) T = 3S wiyi ) 3U yi = 3S wi,i ) 3U ,i

 

 

 = 3U (wi

Ii ) 1),i

,

where Ii = 1 when i 0 S, and Ii = 0 otherwise.

Now

(t ) T)2

 = 3U (wiIi ) 1)2

,i

2

 +

 3U(ik) (wi

Ii ) 1)(wk

Ik ) 1),i,k

.

If ,i and ,k are uncorrelated, the model

variance of t as an estimator for T is

E,[(t ) T)2

] = 3U (wi

2

Ii ) 2wiIi + 1)E(,i

2

)

 = 3S (wi

2

) wi)E(,i

2

)

 ) [3S wi E(,i 

2

) ) 3U E(,i

2

)]

no matter what the sampling design.

Moreover, if E(,i

2

) = xi( for some (, then the

model variance of t as an estimator for T

collapses further to 3S (wi

2

) wi)E(,i

2

), which

is what the delete-a-group jackknife for a

total with finite population correction

estimates. (Note: even if E(,i

2

) does not equal

xi( for some (, we know that 3S wi E(,i 

2

) .

3S(1/Bi )E(,i 

2

) . 3U E(,i

2

) for randomizationbased reasons). 

 

Similar arguments can be made for calibration

in the second (or later) phase of sampling. 

Kott (1997) contains a treatment of this topic

for the conventional delete-one-primarysampling-unit jackknife. The interested reader

may also want to look at the expression for

Var2d in Appendix B and replace each uk with

,k. Similar sub-stitutions can profitably be

made in Appendices C and D as well.

In the real world, models fail, which is the

reason NASS insists on using randomizationconsistent estimators where possible. The

impact of model failure is typically greater on

bias than on variance. This is because model

failure is usually small and subtle but can

nonetheless lead to a bias in a nonrandomization-consistent estimator that is not

asymptotically ignorable. Once the potential

for asymptotic bias is removed by using a

randomization-consistent estimator, a model

can often be safely invoked to estimate

variance. 

The situation can be reversed when ratioadjustment is used (in part) to handle

nonresponse as in the 1996 ARMS and

VCUS. The model assumption that the

expected value for an unknown y-value is a

fixed multiple of a known x-value within a

ratio-adjustment group is usually more

reasonable than the quasi-randomization

assumption that all farms in the group are

equally likely to be survey respondents. In

this situation, the assumption of the linear

model offers some protection against a

systematic bias in an estimated value due to

the failure of the quasi-randomization

assumption.
13

CONCLUDING REMARKS

The delete-a-group jackknife is remarkably

simple to compute once appropriate replicate

weights are determined. We have seen how

this variance estimation method can be

meaningfully applied to a number of complex

estimation strategies. These include the 1996

ARMS (with multiple phases and ratio

adjustments), the 1997 Minnesota pilot QAS

(restricted regression and Poisson sampling),

and the 1996 VCUS (two phases, calibration

of the second phase to the first, and finite

population correction problems). 

Like any variance estimator, the delete-agroup jackknife is not necessarily nearly 

unbiased when any phase of the sample is

drawn systematically from a purposefullyordered list (as is the case in latter phases of

the ARMS). If anything, however, the

deletea-group jackknife will usually be

conservative (biased upward) in this

circumstance. In addition, the delete-a-group

jackknife requires the following to be nearly

unbiased:

1) results from each phase of a survey )

including the first phase ) be calibrated for

some key items of interest on results from

either an earlier same phase or the frame (for

example, the estimated number of farm

names on the list frame is often forced to

equal the actual number of farm names on the

list frame); and

2) the sample size of at every follow-on

phase of a non-nested (not multi-stage) multiphase design be large (contain at least five

sample units per stratum at that phase).

These are not difficult requirements, and

NASS need keep them in mind when

developing estimators in the future. 

A disadvantage of the delete-a-group

jackknife over potential competitors is that it

requires the first-phase stratum sample sizesto

be large (at least five sample units per

stratum). Otherwise, the delete-a-group

jackknife can be overly conservative. As a

result, when this jackknife is applied to

estimators from the NASS area frame ) as it

was with the non-overlap component of the

1996 Phase III CRR, it has an upward bias.

NASS needs to assess how big a problem this

constitutes in practice. 

JULY 2001 UPDATE

NASS made the Minnesota QAS pilot

operational in all states in 2000. A slightly

different form of restricted regression is used.

Variances are estimated ignoring finite

population correction. For more details, see

Kott and Bailey (2000).
14

REFERENCES

Bailey, J.T. and Kott, P.S. (1997). An

Application ofMultiple List Frame Sampling

for Multi-Purpose Surveys. ASA

Proceedings of the Section on Survey

Research Methods, 496-500.

Harley, H.O., and Rao, J.N.K. (1962).

Sampling with Unequal Probabilities and

Without Replacement. The Annals of

Mathematical Statistics, 350-74.

Kott, P.S. (1990). Variance Estimation when

a First Phase Area Sample is Restratified.

Survey Methodology, 99-104.

Kott, P.S. (1997). A (Partially) Model-based

Look at Jackknife Variance Estimation with

Two-Phase Samples, Statistical Society of

Canada Proceeding of the Survey Methods

Section, 37-45. 

Kott, P.S. and Bailey J.T. (2000). The

Theory and Practice of Maximal Brewer

Selection with Poisson PRN Sampling. ICES

II: The Second International Conference on

Establishment Surveys, 269-78. 

Kott, P.S. and Fetter, M. (1999). A MultiPhase Design to Co-ordinate Surveys and

Limit Response Burden. ASA Proceedings of

the Section on Survey Research Methods, 

33-7. 

Kott, P.S. and Stukel, D.M. (1997). Can the

Jackknife Be Used With a Two-Phase

Sample? Survey Methodology, 81-9.

Rust, Keith (1985). Variance Estimation for

Complex Estimators in Sample Surveys. 

Journal of Official Statistics, 1, 381-397.
15

 APPENDIX A: Justifying the Delete-a-Group Jackknife

Under a Single-Phase, Stratified Sampling Design

Suppose we have a probability sample design with H strata and nh sampled units within each

stratum h. Let us assume that the sample was selected without replacement but the selection

probabilities are all so small, and the joint selection probabilities are such, that using the withreplacement variance estimator is appropriate (this rules out systematic sampling from a

purposefully-ordered lists). In particular, let us assume that the estimator itself can be written in

the form:

 H nh

 

 t = 3 3 thj.

 

 h=1 j=1

 

 nh

Let qhj = thj ) 3t

hg /nh

. The randomization variance of t is Var(t) = 3

H Var(3 th+), where th+ = 3j t

hj.

Now Var(th+) can be estimated in a (nearly) unbiased fashion by 

 nh

var(th+) = (nh /[nh ) 1]) 3 qhj

2

 j=1

(“nearly” because we are ignoring finite population correction). 

In order to estimate Var(t) with a delete-a-group jackknife as suggested in the text, we first order

the strata in some fashion and then order the units within each stratum randomly. The sample is

partitioned into R (i.e., 15) systematic samples using the resulting ordered list. Let Sr denote one

such systematic sample, Shr the set of nhr units in both Sr and stratum h, and Sh(r) the set of nh(r) units

in stratum h and not in r. 

The jackknife replicate estimator t(r) is

 H

 t(r) = 3 (nh /nh(r)) 3 thj.

 h=1 j0Sh(r)

Now 

 H

 t(r) ) t = 3 [(nh /nh(r)) 3 thj ) th+].

 h=1 j0Sh(r) 

 

Treating each Sh(r) as a simple random subsample of the sample in stratum h, we have 

 

 

 H

 E2[(t(r) ) t)2

] = 3 Var2([nh /nh(r)] 3 thj) 

 

 

 j0Sh(r) 

 
16

 

 

 H nh

 = 3 (nh

2

/nh(r))[1 ) (nh(r)/nh

)] 3 qhj

2

/(nh ) 1)

 H 

 

 nh

 = 3 (nh/[nh ) 1])(nhr /nh(r)) 3 qhj

2

 H

 = 3 (nhr /nh(r)) var(th+), 

where E2 denotes expectation with respect to the subsampling. 

Observe that for strata where nh < R, nhr /nh(r) is either zero because there are no units in both r and

h or nhr /nh(r) is 1/(nh ) 1) because there is one unit in both r and h. Since the latter situation occurs

in exactly nh

 replicates, 3

R nhr /nh(r) = nh /(nh ) 1).

For strata where nh $ R, nhr /nh(r) = O(1/R) and 3

R nhr /nh(r) . 1 + O(1/R). (Technical note: z = O(1/R)

means limR64R|z| is a constant). In fact, when nh /R is an integer, nhr /nh(r) exactly equals 1/(R ) 1),

and 3

R nhr /nh(r) = R/[R ) 1] . 

Since Var(t) can itself be estimated in an approximately unbiased fashion by var(t) = 

3

H

(nh

/[nh

 ) 1]) 3j

 qhj

2

, it is not difficult to see that the delete-a-group jackknife variance estimator,

vJ

 = ([R ) 1]/R) 3

R (t(r) ) t)2

 is approximately unbiased for var(t) and thus for Var(t) when all strata

are such that nh $ R and is biased upward otherwise. Moreover, the relative upward bias is bounded

by ([R ) 1]/R)minh {1/(nh ) 1)}. 
17

APPENDIX B: Justifying the Delete-a-Group Jackknife

for a Restricted-Regression Estimator Under a Two-Phase Sample

(Including When the “Second” Phase is a String of Phases)

Consider the estimator, t = 3S wiyi, where wi have the same definition as in equation (2), and yi is

a value of interest for element i. For ease of exposition, we will let fi

 be the inverse of the firstphase selection probability of element i, which we assume to be large for all i. It is a simple matter

to use induction to cover the situation where fi is itself the result of several phases and calibrations.

We also assume that neither phase sample is Poisson. Poisson sampling in the first phase is

discussed in Appendix D. Poisson sampling in a later phase is identical to an additional stage of

sampling because Poisson sampling is independent from one selection to the next. 

Let B = (3U xi'xi)

-1 3U xi

'yi, where U is the set of all elements in the population, and ui = yi ) xiB.

Equation (2) allows us to rewrite t as t = 0B + 3S wiui, and the variance of t as Var(t) . Var(0B) +

Var(3S wiui ) + 2Cov(0B, 3S wiui). Now 0 has no variance when it comes from the frame, and

Var(t) collapses to Var(3S wiui) .

Let O and o define asymptotic orders (z = O(m) means limm64

 |z|/m is a constant; z = o(m) means

limm64 |z|/m = 0). We assume that equation (2) holds for almost all elements in the sample (i.e., it

fails at most oP(m) times, where m is the size of S). As a result, 3S wiui . 3S (fi /pi)ui under mild

conditions, we assume to hold (this is because, treating each (fi /pi

) as O(1), (0* ) 3i,S* [fi /pi

]xi

)

( 3i,S* [fi /pi]xi'xi)

-13S* [fj /pj

]xj'uj = OP(3⁄4m)OP(1/m)OP(3⁄4m) is ignorably small compared to t = OP(m);

note that the equality 3U xj

'uj

 = 0 has a role in making this contention viable). Thus, Var(3S wiui

)

is approximately the variance of a double-expansion estimator, 3S (fi /pi)ui. Assuming the secondphase samples within each second-phase stratum are large, results in Kott (1990, p. 103) show the

single-phase variance estimator with estimated primary sampling unit values put in place of actual

values will over-estimate the variance of a double expansion estimator unless the sum of the fiui in

the second-phase strata before subsampling are equal to zero (note that since both md

 and nh in

equation (B) of Kott are large, only ed..

2

 matters). 

Kott assumed stratified simple random sampling in both phases, but extensions to stratified

systematic probability sampling from randomly-ordered lists are straight-forward. For the firstphase sample all the fi

 must be large (as in the simple random sampling case), so that the withreplacement expression for variance is appropriate. For the second-phase sample, the selection

probabilities and population must be such that the approximation pik . (md ) 1)pipk /md

 holds (see

Hartley and Rao 1962), where pik is the second-phase joint selection probability of two elements,

i and k, from second-phase stratum d, and md is the number of sampled elements in that stratum.

In most NASS applications, the second-phase of selection is unstratified, which is equivalent to d

being all the elements in the first-phase sample. 

The second-phase variance of 3Swiui originating from second-phase stratum d can be expressed by

(we are assuming md is large) 
18

Var2d = 3 (fiui )

2

(1 ) pi)/pi + 3 (fiui)(fkuk)(pik ) pipk)/(pipk)

 . 3 (fiui )

2

(1 ) pi)/pi ) 3 (fiui)(fkuk)pipk /md

 = 3 (fiui )

2

(1 ) [(md ) 1)/md

]pi

)/pi

 ) (3 fiui

)

2

/md

 . 3 (fiui)

2

(1 ) pi)/pi ) (3 fiui)

2

/md

where the summations are over all elements in second-phase stratum d before the second-phase of

sampling takes place. This is analogous to equation (3) in Kott. 

Note that Var2d # 3 (fiui

)

2

(1 ) pi

)/pi

, where equality holds only when 3 fiui

 = 0. This turns out to

be the reason (not directly proven here) why the delete-a-group jackknife over-estimates the

variance of t when the sum of the fiui

 within all second-phase strata are not approximately zero. 

Observe that given any column vector 8 of the same dimension as xi', 3F 8'xi'fiui .3U 8'xi'ui =

3U 8'xi'[yi ) xi(3U xi'xi)

-1 3U xi

'yi] = 0. Since 3F 8'xi'fiui .0 for any 8, when there exists a 8d for every

second-phase stratum d such that xi8d = 8'

dxi

' equals 1 when i is in d and 0 otherwise, then the sum

of the fiui in any second-phase strata before subsampling is approximately (asymptotically) zero.

Applying the weights defined by equation (3) to t, we get t(r) . 0(r)B + 3S(r) [fj(r) /fj]wjuj .

3S(r) [fj(r) /pj]uj. The second part of the first s approximation makes use of the facts that the

components of (0(r) ) 3i,S(r) [fj(r) /fj

]wixi

) are OP

(m/R), while the diagonal components of

3i,S(r) [fi(r) /fi

]wixi

'xi

 are OP

(m) under mild conditions. 

When 0 = 0(r) is a frame value, t(r) ) t . 3S(r) [fj(r) /pj

]uj ) 3S [fj /pj

]uj

. It is straight-forward to show

that the delete-a-group jackknife estimates Var(t) = Var(3S wiui

) . Var(3S [fj /pj

]uj

) fairly well with

the possibility of being upwardly biased when the sum of the fiui

 before subsampling in one or more

of the second-phase strata is not equal to zero. 

When 0 = 3F fixi, then 0B =3F fixiB, and t can be rewritten as t = 3F fiyi + (3S wiui ) 3F fiui) .

3Ffiyi + (3S [fi /pi]ui ) 3F fiui) = 3F fi(yi + {[Ii /pi] ) 1}ui), where Ii = 1 when i is in S and zero

otherwise . In a similar fashion, t(r) . 3F fi(r)(yi

 + {[Ii /pi

] ) 1}ui

). It is straight-forward to show that

the delete-a-group jackknife estimates the conventional multi-stage variance estimator ignoring fpc

at the first stage, which in turn estimates Var(t) fairly well but has the possibility of being upwardly

biased when the sum of the fiui before subsampling in one or more of the second-phase strata is not

equal to zero (see Kott and Stukel [1997] for some missing details). 

Extension of the above result to a sample design where the “second-phase” sample is itself the

result of a string of phases, all within the same second-phase strata, is a simple matter. We need

only assume that pik* . "pi*pk* where pj* (pik*) denotes the appropriate product of conditional

(joint) selection probabilities, and " = 1 ) O(1/md). Appendix C has more on the sequence-ofsampling-phases methodology used in the ARMS design.
19

APPENDIX C: Justifying the Delete-a-Group Jackknife

for Certain Composite Estimators in the ARMS

We restrict attention in this appendix to the unusual composite estimators used with the ARMS

surveys. 

To show that the delete-a-group jackknife works for a composite estimator like the Phase III Cornfor-Grain/Beef CRR described in the text, one needs to show that it works when estimating a total

for: 

1) the intersection of the two original target populations (list farms with grain corn and at least 10

weaned calves), 

2) each of the two “rump” populations that contain elements in one population but not the other,

and, 

3) the union of the two rumps and the intersection. 

The delete-a-group jackknife works for estimates of a rump total because it works for domains (by

defining item values within one target population as zero for farms outside the domain), and it

works for estimated totals in the union ) assuming it works for estimated totals in the intersection

) because it works for functions of estimators ( like the sum of the totals in the two rumps and the

intersection). We discuss intersections below.

Let us call the two samples we are compositing A and B. In principle, we can estimate an item total

in the intersection of the target population using either sample. Let tC = 3C wi

C

yi be the estimated

total calculated using sample C ( = A or B), and let t = 8t

A

 + (1-8)tB be the composite total. Note

the yi

 is defined to be zero for farms outside the intersection. 

The ARMS samples are drawn sequentially to avoid overlap using an unstratified variant of

systematic unequal probability sampling at every phase after the initial screening phase. Let Bi

t

 be

the probability of selecting farm i for sample t given that it is available for sampling after sample

t-1 is drawn. Let t=1 denote the first sample drawn after the screening sample. Finally, let pi

t

 =

(1 ) Bi

1

) ... (1 ) Bi

t-1)Bi

t

. Note that Bi

s

 = 0 when farm i is not in the target population for sample s.

 Without loss of generality, we will assume sample A was selected before B, and that A, B, and

their intersection are of size O(m).

Using arguments similar to those in the previous appendix, we can see that 

t

C

. 3F fiyi

 + (3C [fi /pi

C

]ui

C

) 3F fiui

C

), 

where ui

C

 = yi ) (3 yk /3 xk

C

)xi

C

, the summations being over the farms in the population that are in

the same calibration group as i when computing tC

, and xk

C

 is the x-value of farm k when computing

t

C

. Observe that the first-phase sample applies to both A and B since there is one ARMS screening
20

sample for all purposes. By contrast x and u-values as well as calibration group memberships may

differ across samples for the same i. This happens when the 1996 Phase II Corn-for-Grain PPCR

is composited with the Phase II Corn PPR; xi

A

 is corn-for-grain acres for farm i, while xi

B

 is general

corn acres. 

We can now see that 

Var(t) = Var1( 3F fiyi) + E1{Var2( 3A [fi /pi

A

]ui

A

) +Var2( 3B [fi /pi

B

]ui

B

) +

 2Cov2( 3A [fi /pi

A

]ui

A

, 3B [fi /pi

B

]ui

B

)}.

Now 

Cov2( 3A [fi /pi

A

]ui

A

, 3B [fi /pi

B

]ui

B

) = 3i0F(A) 3k0F(B) (fiui

A

)(fkuk

B

){[(pi

A

k

B

)/(pi

A

pk

B

)] ) 1},

where F(C) denotes that part of the first-phase sample in the target population for sample C, and

pi

A

k

B

 = Bi*k*

1 ... Bi*k*A-1Bik*A

(1 ) Bk

A+1) ... (1 ) BkB-1)Bk

, when Bi*k*

t

 is the probability of selecting neither

farm i nor k for sample t providing both are available after sample t-1, and Bik*

A

 is the probability

of selecting farm i but not k for sample A given that both are available after A-1. We will assume

that the design is such that given ki, pi

A

k

B

. (1 + "k

)pi

A

pk

B

, where "k

 = O(1/m) (if Bi*j*

s

/Bi*

s

Bj*

s

.

1 for all s < A, then the assumption is equivalent to Bik*

A

 = Bk*

A

 Prob(i chosen for A|k not chosen

for A) = Bk*

ABi

A

(1 + "k)). Since 3F(B) fiui

B

. 0 and pi

A

i

B

 = 0, summing the left hand size of the last

expression for Cov2

 over i yields a term of order 1/m. Summing then over k yields a term of order

1. Since Var1(tA) is O(m), the covariance term is asymptotically ignorable. 

 It is now not hard to show using arguments developed here and in the previous appendices that the

delete-a-group jackknife is unbiased for t under conditions we assume to hold.
21

APPENDIX D: Justifying the Delete-a-Group Jackknife

with Finite Population Correction for a Single-Phase Poisson Sample

Suppose we have a calibrated estimator for a total, t = 3S wjyj, where the wj satisfy equation (6): 

 wj

 = 1/Bj

 + (0* ) 3i,S* [1/Bi

]xi

)( 3i,S* [1/Bi

]xi

'xi

)

-1 [1/Bj

]xj

' (6)

 

for j 0 S*, and a predetermined value otherwise. In addition, there is a vector 8 such that

xj8 = 3⁄4(1 ) Bj

) for all j. 

Let B = (3U xi'xi)

-1 3U xi

'yi, where U is the set of all elements in the population, and ui = yi ) xiB. 

Now t =3S wjyj = 3S wj(xjB + uj) = 0 + 3S wjuj. Consequently, Var(t) = Var( 3S wjuj) .

Var( 3S uj /Bj

) under conditions we assume to hold (see Appendix B).

If the sample is Poisson, we have Var(t) . 3U uj

2

(1 ) Bj)/Bj, which, in principle, can be estimated

in a nearly unbiased fashion by var(t) = 3S uj

2

(1 ) Bj

)/Bj

2

. 3S

 wj

2

(1 ) 1/wj

)uj

2

 = 3S

 (wj

(v))2

uj

2

, where

wj

(v) = wj3⁄4(1 ) 1/wj

) (see equation (7)).

Using the definition of wj(r)

(v) in equation (8), we have t(r)

(v) =3S wj(r)

(v)yj . 3S wj

(v)xjB + 3S(r) wjr

(v)uj

.

Consequently, t(r)

(v)

) t(v) = 3S(r) wjr

(v)uj

 ) 3S wj

(v)uj

. Now 3S wi

(v)ui /3⁄4n = 3U (1 ) 1/wi

)

1/2ui /3⁄4n .

3U(1 ) Bi)

1/2ui 3⁄4n = 3U 8'xi

'ui /3⁄4n = 0, since 8'xj' = xj8 =3⁄4(1 ) Bj) for some 8, while 3U xi'ui = 0.

Similarly, 3S wj

(v)uj /3⁄4n . 0. 

The the n(r) members of S(r) can be viewed as a simple random subsample of the n members of S.

Since n/n(r) = 1 +OP

(1/R), t(r)

(v)

) t(v)

. (n/n(r))3S(r) wj(r)

(v)uj

. Using arguments similar to ones made

in Appendix A, we have E2

[(t(r)

(v)

) t(v))

2

] . (n/n(r))(1 ) n(r) /n)[ 3S

 (wj

(v))2

uj

2

) (3S wj

(v)uj

)

2

/n]. So,

E2[(t(r)

(v)

) t(v))

2

] . (n/n(r))(1 ) n(r) /n) 3S (wj

(v))

2

uj

2

 = (nr /n(r)) 3S (wj

(v))

2

uj

2

, where nr is the size of Sr.

From here it is easy to see that vJ(fpcT) is nearly unbiased for 3S

 (wj

(v))

2

uj

2

, which is turn is nearly

unbiased for Var(t). 

Observe that when 3U (1 ) Bi)

1/2ui  0, the jackknife is biased, although the bias depends on the size

of the Bi. In practice, this may be of little importance because if we felt that 3U (1 ) Bi)

1/2ui /3⁄4n had

an absolute value far from zero, we would include 3⁄4(1 ) Bj) as a component of xj.