Comparison of The Random Group Variance and 

The Stratified Variance Estimators 

Stephen M. Miller, Kenneth Robertson, and Shail Butani; Bureau of Labor Statistics 

2 Mass. Ave.N.E., Postal Square Building Suite 4985, Washington DC 20212 

1. Introduction 

The office of Employment and Unemployment Statistics 

of the Bureau of Labor Statistics (BLS) frequently 

conducts quick response establishment surveys. These 

surveys are ~ial one time surveys mandated by the 

Congress. Quick restxmse surveys are normally National 

in scope, with a sample size of 7,500 to 10,000 

establishments. Stratification for these surveys is by 

industry and size class. Quick response surveys are 

designed to measure characteristics such as the number of 

employees providing child care facilities, the number of 

employees having a drug testing program, and the number 

of employers who have job openings for specific 

occupations. 

The characteristics estimated by quick response surveys 

are, in may cases, rare. Because of its ease of use the 

random group estimator is commonly employed. It is felt, 

from years of empirical experience, that for rare 

characteristics (p<0.05) the random group variance 

estimator overestimates the variance especially when the 

relative standard error of the estimate is large. The result 

of limiting a characteristic to one or two groups is a pvalue which is high in these random groups and zero in 

the remaining random groups. In order to determine if the 

random group variance estimator is overestimating the 

variance, we decided to compare the random group 

variance estimator with the standard stratified variance 

estimator. This paper reviews the research done to 

compare these two variance estimators. 

Section 2 details the theoretical research; Section 3 

provides a description of the data and statistics used for 

our empirical investigation; Section 4 gives the empirical 

results, and Section 5 gives our conclusions and future 

research plans. 

2. Theoretical Research 

In this section we present a theoretical background for 

our investigation of the random group variance estimator 

and the standard stratified variance estimator. We will 

assume the following stratified sampling setup. Let there 

be L strata each indexed by h such that h=l ..... L. 

Within each strata let the population values be indexed as 

L 

X~ for i=l ..... N h . We will let N=~N h denote the 

h=l 

total population size across all strata. Within each strata 

let a simple random sample with replacement (SRSWR) 

be selected, where the sample units are arranged by 

random groups k = 1 ..... K with m h units falling in the 

k-th random group in stratum h. Notice that for 

convenience we are assuming that the same number of 

sample units are allocated to each random group within a 

given strata. We also assume that the sample units are 

allocated to the groups at random. We are interested in 

estimating the total T by the estimator "I" as follows, 

t. x =, t. ,¢, 

h=l k=l iffil hffil ill 

where w h =n~tNh is the sampling weight. We are 

interested in estimating the variance of "i" by the method of 

random groups, and also by the usual stratified variance 

estimator. In order to define these two estimators with a 

common notation, it is convenient to define 

(2) a~ = w~x~. 

In the case of simple random sampling without 

replacement (SRSWOR) we can define d~. = ~/1- fh whx~ 

where fh = N-htnh is the sampling fraction. The random 

group variance estimator VRG , and the usual stratified 

variance estimator Vs, can be written as 

K 

(3) vR~=(K-1)-'K~_~(a+,÷-d÷.+) 2 

k=l 

L K ma 

= E(,,,- 1) ',,.E E(d - d..) 

h=! k=l i=l 

where n h =Km h , and 

L m~ K 

(4) d+,+ = EZdu/ d+.+ = K-'Zd+k + 

h=l i=1 k=l 

K m k 

k=l i=1 

We are interested in the joint behavior of the two 

variance estimators. We begin by defining a few 

population values. Let 

N~ 

i=l 

tv k 

- N;'E 

i=1 

Nlt 

i=l 

Notice that flh is the population kurtosis within stratum h 

Under the SRSWR sampling scheme, each dh~ is 

distributed independently with mean whX h , variance 

%crh2 2 , and kurtosis flh • In addition, the d~ are 

746 
identically distributed within each stratum. 

SRSWR sampling scheme it follows that 

Under the 

L 

(6) V{'I'} ~ 2 2 

--- nhWhO" h 

h=l 

We begin by presenting Result 1, which gives 

expressions for the means, as well as variances and 

covariances for the two variance estimators. 

Result 1. Under SRSWR, 

E{vs,} = E{vRG} = V{rF} 

L [ 2,] Vl:v~,} = Y__, nhWhCrh "" (~h -- 3) + 

h=l n h - 1 

nhWhl:~h([~ h _ 3)+2(K _ 1) -1 nhwhl~ h 

h=l h=l 

C{v~,,v.~ } = V{v~,} 

L 

Corr{vs,,V~c ] : h~nhw~a:[ (~h-3)+2n'(nh-1)-~ 

y n, w, cr~(~,-3)+2(K-1) n, whcr h 

ll,,1 

Proof. See Appendix. 

The expressions for the means and variances can be 

found in other sources such as Wolter, but the result on 

covariance and correlation between the two estimators 

appears to be new. It is interesting to note that the 

covariance between the two estimators is the same as the 

variance of v,, . This fact will be exploited in Result 2 

below. The correlation expression of Result 1 has some 

interesting interpretations. We begin by presenting a 

Table below which gives kurtosis values for three 

distributions. 

Table 1. Kurtosis Values for Various Distributions 

~Volter, 1985) ..... 

Distribution Kurtosis 13 Parameter 

Normal 3 

Bernoulli 

Poisson 

3 +[1-6p(1-p)] 

p(1-p) 

3+/1. -1 

p: Success 

)~:Expected 

Value 

For a normally distributed population, the kurtosis is 3 

which causes the first term in both the numerator and 

denominator of the correlation expression to disappear. In 

this case if the sample size is large then the correlation 

will be small and approach zero asymptotically. The 

situation is slightly different with the other two 

distributions. For both the Bernoulli and Poisson 

distributions the first term does not disappear from the 

numerator and denominator. In fact, if the populations are 

rare (p close to 0 for the Bernoulli, and /q. close to zero 

for the Poisson) then the kurtosis is large and the first 

term in both the numerator and denominator can dominate 

the correlation expression which causes it to approach 1. 

Therefore, the two variance estimators should be more 

highly correlated when the populations are rare. 

In the next result we express the random group variance 

estimator in terms of the usual stratified variance estimator 

plus an error. 

Result 2. Under SRSWR, we can write 

V RG -- V St -t" error 

where 

E{error} = O, C{vst,error } : 0 

/ L 2 2 12 V{error} = 2(K- 1) -1 ~nhWhCr h 

h=l 

L 

__2 Znh2Wh4 4(nh_l)-lo. h 

h=l 

Proof. We can write error=vRa-Vst, and Result 2 

follows from Result 1. 

Result 2 can be thought of as a regression like result. It 

says that if we were to regress the random group estimator 

on the usual stratified variance estimator that we would 

get an intercept of 0 and slope coefficient of 1. In addition 

the regression error is heteroscedastic, with a variance 

which increases with the variance of the estimated total. 

In the next result we examine the approximate 

expectation of the ratio of the two variance estimators. 

Result 3. If we approximate the ratio of variance 

estimates as 

VSt 

(~.~ - vttl)(v,,- vtt~)+~(~,,- v{~) ~ v~{t} 

+remainder. 

Then the expectation of the approximation, ignoring the 

remainder, is EIvRGI=I 

LVs, J 

Proof. The result follows from applying Result 1. 

The interesting thing about this result is the expectation 

of the second order terms in the approximation is zero. In 

Result 4 below, we examine the approximate variance of 

the ratio of the variances. 

747 
Result 4. If we approximate the ratio of variance 

estimates as 

~=='+, ,, v m '""('=-vc*~)-v~(~"-vc~) 

+remainder 

Then the variance of the approximation, ignoring the 

remainder, is 

L 

2 2Zn~w:cr:(n'-l)-' vI .ol= ,., 

[Vs, J K-1 

Proof. The result follows by applying Result 1. 

We will have more to say about Result 4 after we 

present Result 5. In Result 5 below, we present some 

asymptotic theory about the behavior of the variance ratio. 

Result 5. Assume an infinite sequence of sample 

designs and populations such that, 

d+j,+ - K-'T 

~K-'V{t} L > N(O,1), and Vs, - V{q'} P )0. 

Then, va-----q-a " > F x-' 

Vs, 

where F x-' denotes an F random variable with numerator 

degrees-of-freedom K-1 and denominator degrees-offreedom .o. 

Proof. Let 

gk -- 

d+j,+ - K-'T 

4K-'Vit} 

K 

then VRG = v{'r}(K-1)-'~(Rk--R) 2. 

k=l 

By the first assumption, 

K 2 

k=l 

so the result follows from the second assumption by 

Slutsky's theorem. 

The assumptions of Result 5 are mild, and should be 

satisfied by most designs used in practice. Result 5 

implies that under the asymptotic theory, 

(7) e{vRo < - Vs, } ---) P{ Zx-1 2 < K- 1}. 

In Table 2 below, we give some of the probability values 

for various numbers of random groups. 

Table 2. Asymptotic Values for F_,q. ~7) 

Random Groups p{vR ° < Vst} 

0.5940 

6 0.5841 

7 0.5768 

8 0.5711 

9 0.5665 

10 0.5627 

11 0.5595 

12 0.5567 

13 0.5543 

Asymptotically, we would expect the random group 

variance estimator to be less than the usual stratified 

variance estimator on average, despite the fact that they 

are both unbiased. Result 5 also implies that 

(8) v~- Us, -v{~}(~ -~- ~), 

where N stands for approximately distributed as. That 

is, if we examine the difference between the two variance 

estimator, then .asymptotically the distribution of the 

difference has a skewed distribution which is a multiple of 

a translated F-distribution. This relates back to Result 2 

which gave expressions for the error, Result 5 tells us 

about the asymptotic behavior of the distribution of error. 

Finally, notice that an F random variable has mean and 

variance given by 

(9) E{F. x-'} = 1 V{F2-'} = 2(K-1)-' 

We can compare these values to those of Result 3 and 

Result 4. Note that our approximate expectation and the 

mean of the asymptotic distribution are both l, but 

expressions for the variances differ slightly. In particular, 

the approximate variance is smaller than that of the 

variance of the asymptotic distribution. Later we will 

examine which variance expression holds for our 

empirical data, and we will say something about how well 

the asymptotic theory works with rare populations. 

3. ETJO Data 

At the time we began this investigation, we had recently 

completed a special survey, a pilot study on employee 

Tumover and Job Openings (ETJO). The ETJO survey 

was conducted in response to a Congressional mandate to 

the Department of Labor to study the feasibility of 

measuring national labor shortage data and the associated 

cost.. 

The ETJO survey employed a stratified (8 industries x 3 

employment size classes) probability sampling plan. 

These 24 strata are called ETJO sampling cells. Because 

occupations vary by industry, a separate questionnaire was 

required for each industry. Therefore, the scope of the 

survey was limited to eight industries. The sample 

consisted of approximately 3300 units selected from a 

universe of about 1.5 million establishments. The universe 

covered private establishments with one or more 

employees during the first quarter of 1989 in the 50 States 

and the District of Columbia. Prior to sample selection, 

the total national frame was sorted within each ETJO cell 

748 
by 4-digit SIC/ Employment/ and State. This sort 

permitted further stratification in the sample selection 

process. Once this sort was completed, a random 

systematic start sample was selected within each ETJO 

sampling cell. The total sample was divided into three 

monthly panels. Each panel was surveyed 1 month, out of 

sample 2 months, and then back in the survey 1 month (ie. 

1- 2-1). Random Group assignments were made by 

systematic assignment after a random start. The reference 

data collection period for the survey was November 1990 

to April 1991. 

The data were collected by mail. There was one follow 

up mailing, after which nonrespondents were contacted 

using a Computer Assisted Telephone Interview (CATI) 

instntment. The overall usable response rate for the first 

and second collection periods, respectively, was 70 and 75 

percent. Five data items were collected by detailed 

occupation: job separations, new hires, average wage of 

new hires, job openings, and the duration of job openings. 

Estimates were produced for each of the five 

characteristics by occupation for each of the eight 

industries surveyed. Additionally, estimates were 

produced by occupation at the National level, or by 

ignoring industry and grouping by occupation only. The 

ETJO survey variance estimates were produced using a 

Random Group Variance (RGV) estimator. This estimator 

was calculated as follows 

K 2 

VRa(f(~)= (K-1) I K~(X, - X.) 

1:=1 

where X~ is the estimate for characteristic X in industry j 

for occupation d. In addition K refers to the number of 

random groups, and X~, is the estimate based upon the k-th 

6 3 

random group where, X k = E E Z Wi~u, NXo~k ' and 

/=1 c=l i 

K 

2". = K-I~ X, where 1 is the month, c is the size 

k=l 

class, i is the establishment, and NX00~k = the number 

reported for characteristic X by unit i for occupation d. 

1FWo, u,~/1 fo.,( 3 ) 

where FW,~j, is the sampling weight, which is the inverse 

of the probability of selection adjusted for nonresponse 

and changes in the frame, and (1-fj,~)is the finite 

population correction factor. Notice that a divisor of 6 

was used to obtain an average over six months of data, and 

multiplied by (3/1000) to produce an estimate of monthly 

total in units of one thousand. 

The standard stratified variance estimator was computed 

as follows 

Vs'(2~) ( ) (1- x.) = f nj,'- 1 

¢:i i=l 

where hi, is the number of establishments in industry j 

and employment size class c, and 

x,.:.;, x,,. . x,,. : %--g ) 

i=1 

In our investigation we decided to use three of the 

characteristics from the survey. These characteristics were 

Separations, Job Openings, and New Hires. These 

statistics were produced for each SIC/Occupation group. 

Additionally, these statistics were produced using 6, 9, 

and 12 random groups. This gave us a total of 81 sets of 

statistics to review, with each set containing a varying 

number of statistics according to the number of 

occupations in the industry. In addition, the computer 

package PC CARP was used to compute the standard 

stratified variance estimates. The statistics for each set 

were grouped by size to produce frequency distributions. 

Since most of these differences were small, we decided to 

use the interval from zero to three in units of 0.2. 

4. Empirical Results 

In this section we describe our empirical research using 

the ETJO survey data. 

We begin by presenting plots of variance ratios. The 

chart labelled "Theoretical vs. Empirical Distribution" was 

constructed using Separations data for 6, 9, and 12 

random groups. This chart shows three histograms of 

ratios of variance estimates of total separations for the 8 

SICs and various occupations. The variance ratios were 

defined to be v Ra/v,, . The histograms were normalized 

by the bin width and total number of observations, so that 

they are an estimate of the density of the variance ratio. 

We also plotted the density for the associated F random 

variable. This is the density of the asymptotic distribution 

implied by Result 5. It is interesting to note that the 

empirical histogram density estimator appears to be shifted 

more to the right than the theoretical density estimator. 

This indicates that the random group estimator tends to be 

larger relative to the standard stratified estimator than the 

theory would predict. Theoretically we would expect that 

both variance estimators would be unbiased, that the 

random group variance estimator would be more variable, 

and that, on average, the random group variance estimator 

would be smaller than the standard stratified variance 

estimator. In fact, the theory says that 

P{vR~ < Vs, } = P{Z 2 < 8]- 0.567 . 

The empirical histogram estimator shows that for 9 

random groups 42 % of the distribution falls to the left of 

1.0 which indicates that the distribution has moved 

substantially to the right. 

The theoretical results can be made to more closely 

match the empirical results by allowing the noncentrality 

parameters to be greater than zero. In general, the 

749 
noncentrality parameters cause the distributions to move 

to the right, and pack more tightly around 1.0. 

5. Conclusions 

Our hypothesis at the beginning of this paper was that 

the random group variance estimator tends to overestimate 

the variance when the value to be estimated is a rare 

characteristic in the surveyed population. We have found 

empirical evidence that the random group variance 

estimator tends to overestimate the variance relative to the 

standard stratified variance estimator. We speculated that 

this could be caused by the sorting and systematic 

selection of sample units, as well as the systematic 

allotment of random group assignments. We performed 

small scale simulations using the sample as the sampling 

frame. Whether we used systematic sampling or random 

sampling in these simulations we still observed the 

random group variance estimator overestimating the 

variance relative to the standard stratified variance 

estimator. 

Theoretical results show that for a normal population 

E(VRG ) = E(Vst ). Our research also specified an F 

distribution for the ratio of these two estimators. This F 

distribution, for a normal population, shows that P(VRG 

< VSt) > 1/2. It was also shown that for a population with 

large kurtosis, which can be caused by rare characteristics, 

that this distribution can change shape rather dramatically. 

The theoretical section showed that for a normal 

population the kurtosis has no affect on the correlation 

between these two variance estimators. However, as the 

population begins to depart from normality (or as the 

kurtosis becomes larger than 3) the kurtosis does influence 

the correlation between the estimators. When the kurtosis 

of the population is large this figure becomes the 

dominant one in the correlation equation and the 

correlation between these variance estimators begins to 

approach 1. 

Our empirical findings revealed several interesting 

things. First, as expected, when the number of random 

groups is increased the variance of the ratio VRG/VSt 

decreases. This reduction in the variance of the ratio 

causes a larger percentage of the distribution to be close to 

1. This result is as described by previously established 

theory. 

Second, as the number of random groups is increased, it 

appears that the portion of the distribution most heavily 

affected was those observations where VRG/VSt < 1. That 

is, the observations less than 1 move towards 1 more 

rapidly than those observations greater than 1 as the 

number of random groups is increased. In cases where the 

characteristic to be measured is rare the random group 

variance estimator underestimates the variance less often 

than theory predicts. The observations where VRG/VSt > 1 

also move towards 1 as the number of random groups is 

increased. However, for the numbers of random groups 

tested this part of the distribution conforms more closely 

to the theoretical distributions than the part of the 

distribution less than 1. 

Third, the empirical results show that the random group 

variance estimator does tend to overestimate the variance, 

in cases where the characteristic to be measured is rare, 

more often than the theory would suggest. This 

overestimation tends to manifest itself by clustering 

around the point where the ratio of VRG/VSt = 1. As the 

number of random groups is increased, this clustering 

becomes more apparent. It is also apparent that most of 

these observations clustering around 1 are coming from 

the part of the distribution which theory suggests should 

be < 1. This means that in many cases, when estimating 

rare characteristics, the random group variance estimator 

is a better estimator than the theory would suggest for two 

reasons. First, in general practice we would rather have a 

conservative variance estimator than one that continually 

underestimates the variance. It appears that the random 

group variance estimator is conservative when the 

population characteristic to be estimated is rare. Secon~l, 

the estimated variance is approximately equal to the 'true' 

variance much more often than theory would suggest. 

Future research plans will include, but not be limited to, 

an investigation of this nature into the properties of the 

Jackknife variance estimator. 

References 

Fuller, W.A., Kennedy, W.J., Schnell, D., Sullivan, G., 

and Park, H.J. (1986). PC CARP. Statistical Laboratory, 

Iowa State University, Ames, Iowa. 

Koop, J.C. (1971). On Splitting a Systematic Sample for 

Variance Estimation. Annals of Mathematical Statistics 

42 1084-1087. 

Krewski, Daniel (1978). On the Stability of Some 

Replication Variance Estimators in the Linear Case. 

Journal of Statistical Planning and Inference 2 45-51. 

Valliant, Richard (1987). Some Prediction Properties of 

Balanced Half-sample Variance Estimators in Single-stage 

Sampling. Journal of the Royal Statistical Society Set. B 

49 68-81. 

Vallaint, Richard (1990). Comparison of Variance 

Estimators in Stratified Random and Systematic 

Sampling. Journal of Official Statistics 6 115-131. 

Wolter, Kirk (1984). An investigation of some estimators 

of variance for systematic sampling. Journal of the 

American Statistical Association 79 781-790. 

Wolter, Kirk (1985). Introduction to Variance 

Estimation. Springer-Verlag New York. 

Wu, C.F.J. (1991). Balanced repeated replications based 

on mixed orthogonal arrays. Biometrika 78 181-188. 

750 
Theoretical vs. Empirical Distribution 

Lt3