Investigation of Variance Estimators for the Survey of Business 

Owners (SBO)

Marilyn Balogh1, Sandy Peterson

U.S. Census Bureau

4600 Silver Hill Rd, Washington, D.C. 20233

Abstract

The Survey of Business Owners and Self-Employed Persons (SBO) provides, on a quinquennial basis, selected 

economic and demographic characteristics for businesses and business owners by gender, ethnicity, race, and 

veteran status. The SBO sample is a stratified systematic sample that employs the random group variance estimator 

to estimate sampling error. This paper reports the results of a simulation study conducted to compare the random 

group variance estimator to the delete-a-group jackknife variance estimator and the stratified jackknife variance 

estimator. The methods were compared by examining the relative bias and the coefficient of variation of each 

variance estimate for five states (Florida, Georgia, Kansas, New York, and North Dakota) for different characteristic 

data (public ownership, race, ethnicity, sex, and veteran status). Another factor taken into consideration was 

computer processing time, which is not trivial for the stratified jackknife applied to a large survey. The results of this 

study suggest that the delete-a-group jackknife variance estimator may provide an improvement over the random

group estimator, with little differences in processing time.

Keywords: variance estimation, stratified jackknife variance estimator, delete-a-group jackknife variance estimator, 

random group variance estimator

1 Introduction

The Survey of Business Owners and Self-Employed Persons (SBO) provides, on a quinquennial basis, selected 

economic and demographic characteristics for businesses and business owners. Currently, SBO employs the random 

group variance estimator to estimate sampling error. The primary purpose of this study was to determine if there is a 

more efficient variance estimator compared to the current method. This study compares the random group variance 

estimator to the delete-a-group jackknife variance estimator and the stratified jackknife variance estimator to 

determine which is best for estimating SBO sampling error. 

We compared variance estimators by simulating data using the 2007 SBO universe, and examining the relative bias

and the coefficient of variation (CV) of each variance estimate for five states (Florida, Georgia, Kansas, New York, 

and North Dakota) for different characteristic data (public ownership, race, ethnicity, sex, and veteran status). 

2 Background on SBO

The SBO provides economic and demographic data for businesses and business owners as part of the Economic 

Census every five years, for years ending in “2” and “7,” by Title 13 of the United States Code. The SBO samples

and publishes data on a firm basis where “a firm is a business consisting of one or more domestic establishments 

that the reporting firm specified under its ownership or control” (Survey of Business Owners2

). The SBO publishes 

the number of employer and nonemployer firms, sales and receipts, annual payroll, and employment by gender, 

ethnicity, race, and veteran status of the majority business owners. 

The SBO uses administrative data to place each firm into one of nine sampling frames: American Indian, Asian, 

Black or African American, Hispanic, Non-Hispanic white men, Native Hawaiian and Other Pacific Islander, Other, 

1 This report is released to inform interested parties of (ongoing) research and to encourage discussion (of work in 

progress). Any views expressed on statistical, methodological, technical, or operational issues are those of the 

author(s) and not necessarily those of the U.S. Census Bureau.

2

http://www.census.gov/econ/sbo/ 

1 

 




Table 1: Sign Test Comparing the Coefficient of Variations between the Different Variance Estimators 

Estimate Variance Estimators T p-value Superior Method

Firm 

Count

RG_S - RG_ST 148 0.0061 RG_ST

RG_S - DAG_S 126 0.3508 None

RG_S - DAG_ST 198 <0.0001 DAG_ST

RG_S - SJK 253 <0.0001 SJK

RG_ST - DAG_S 107 0.0061 RG_ST

RG_ST - DAG_ST 111 0.0225 RG_ST

RG_ST - SJK 249 <0.0001 SJK

DAG_S - DAG_ST 198 <0.0001 DAG_ST

DAG_S - SJK 253 <0.0001 SJK

DAG_ST - SJK 250 <0.0001 SJK

Total 

Receipts

RG_S - RG_ST 171 <0.0001 RG_ST

RG_S - DAG_S 105 0.0518 None

RG_S - DAG_ST 150 0.0019 DAG_ST

RG_S - SJK 249 <0.0001 SJK

RG_ST - DAG_S 84 <0.0001 RG_ST

RG_ST - DAG_ST 93 <0.0001 RG_ST

RG_ST - SJK 221 <0.0001 SJK

DAG_S - DAG_ST 150 0.0019 DAG_ST

DAG_S - SJK 249 <0.0001 SJK

DAG_ST - SJK 250 <0.0001 SJK

The median of the CVs of the SJK method are smaller than the median of all other methods. The RG_ST median 

CVs are smaller than all other methods except SJK, and the DAG_ST median CVs are smaller than both simple 

methods. The median CVs of the stratum-specific method are smaller than the simple methods. 

One limitation of the sign test is that it just looks at the sign instead of the magnitude of the difference. To find a 

superior method, we also looked at the actual values of the relative biases and the CVs for each of the variance 

estimation methods across all domains. 

Table 2 shows the relative bias of the five variance estimation methods for New York across the 17 demographic 

characteristics for all firms (firms with and without paid employees) by firm count. The other states and employer 

status categories show similar results.

 

6 
Table 2: New York total firm count relative biases by demographic characteristic and variance estimator

Demographic Characteristic

Relative Bias by Variance Estimator

RG_S RG_ST DAG_S DAG_ST SJK

All firms 0.174 31.417 0.174 -0.997 -0.988

Female -0.032 0.048 -0.032 -0.040 -0.012

Male -0.069 0.222 -0.069 -0.086 -0.076

Equally male/female -0.068 -0.048 -0.068 -0.068 -0.055

Hispanic -0.050 -0.049 -0.050 -0.058 -0.047

Non-Hispanic -0.054 1.664 -0.054 -0.136 -0.146

Equally Hispanic/non-Hispanic -0.132 -0.133 -0.132 -0.132 -0.104

White -0.245 0.414 -0.245 -0.281 -0.276

Black or African American -0.095 -0.074 -0.095 -0.107 -0.098

American Indian and Alaska Native -0.078 -0.064 -0.078 -0.079 -0.049

Asian -0.552 -0.544 -0.552 -0.555 -0.560

Native Hawaiian and Other Pacific Islander -0.011 -0.018 -0.011 -0.011 -0.020

Some other race -0.002 -0.002 -0.002 -0.002 -0.021

Veteran 0.030 0.039 0.030 0.028 0.030

Nonveteran -0.106 0.864 -0.106 -0.149 -0.141

Equally veteran/nonveteran -0.040 -0.039 -0.040 -0.040 -0.052

Publicly held -0.079 0.070 -0.079 -0.085 -0.096

In certain instances (all firms, male, non-Hispanic, White, and nonveteran), the relative bias of the RG_ST method is 

very high compared to the other methods. For example, for “all firms” the relative bias for the RG_ST method is 

31.417, that is, the average of the RG_ST variance estimate is much bigger than the true variance. There is little

difference between the other four methods (RG_S, DAG_S, DAG_ST, and SJK). All five methods tend to 

underestimate the true variance. The relative biases for the total receipts estimates found no differences between any

of the estimators. 

Tables 3 and 4 show the CVs of the five variance estimation methods for New York across the 17 demographic 

characteristics for all firms (firms with and without paid employees) by estimate type (firm counts and total 

receipts). The other states and employer status categories show similar results.

 

7 
Table 3: New York total firm count CVs by demographic characteristic and variance estimator

Demographic Characteristic

Coefficient of Variation by Variance Estimator

RG_S RG_ST DAG_S DAG_ST SJK

All firms 0.643 31.419 0.643 0.997 0.988

Female 0.449 0.445 0.449 0.444 0.014

Male 0.446 0.487 0.446 0.443 0.077

Equally male/female 0.441 0.437 0.441 0.440 0.056

Hispanic 0.454 0.451 0.454 0.452 0.048

Non-Hispanic 0.449 1.714 0.449 0.429 0.146

Equally Hispanic/non-Hispanic 0.428 0.427 0.428 0.428 0.112

White 0.429 0.535 0.429 0.439 0.276

Black or African American 0.442 0.430 0.442 0.438 0.098

American Indian and Alaska Native 0.451 0.445 0.451 0.452 0.065

Asian 0.591 0.582 0.591 0.593 0.560

Native Hawaiian and Other Pacific Islander 0.490 0.483 0.490 0.491 0.124

Some other race 0.472 0.470 0.472 0.473 0.050

Veteran 0.490 0.489 0.490 0.489 0.033

Nonveteran 0.438 0.953 0.438 0.429 0.142

Equally veteran/nonveteran 0.458 0.457 0.458 0.458 0.058

Publicly held 0.442 0.435 0.442 0.441 0.097

Across almost all characteristics, the SJK method has the smallest CVs for the firm count, almost by half in some 

instances. As with the firm relative biases in Table 2, the RG_ST method tends to have much higher CVs for firm 

estimates in certain groups (all firms, male, non-Hispanic, White, and nonveteran). There is little difference among 

the other three methods (RG_S, DAG_S, DAG_ST) for the firm counts. 

Table 4: New York total receipts CVs by demographic characteristic and variance estimator 

Demographic Characteristic

Coefficient of Variation by Variance Estimator

RG_S RG_ST DAG_S DAG_ST SJK

All firms 0.698 0.695 0.698 0.667 0.352

Female 0.830 0.831 0.830 0.829 0.459

Male 0.506 0.458 0.506 0.498 0.259

Equally male/female 0.559 0.560 0.559 0.560 0.351

Hispanic 0.953 0.712 0.953 0.957 0.771

Non-Hispanic 0.599 0.557 0.599 0.583 0.297

Equally Hispanic/non-Hispanic 1.887 1.831 1.887 1.886 1.899

White 0.608 0.554 0.608 0.592 0.313

Black or African American 0.931 0.872 0.931 0.929 0.812

American Indian and Alaska Native 1.171 1.192 1.171 1.173 1.034

Asian 0.549 0.529 0.549 0.547 0.321

Native Hawaiian and Other Pacific Islander 1.622 1.592 1.622 1.619 1.550

Some other race 0.946 0.944 0.946 0.947 0.817

Veteran 0.539 0.532 0.539 0.540 0.316

Nonveteran 0.525 0.478 0.525 0.515 0.249

Equally veteran/nonveteran 0.670 0.674 0.670 0.673 0.363

Publicly held 0.670 0.524 0.670 0.647 0.357

Across almost all characteristics, the SJK method has the smallest CVs for the total receipts, almost by half in some 

instances. There is little difference among the other four methods (RG_S, RG_ST, DAG_S, DAG_ST).

8 

Expenditures Survey," Proceedings of the Research Conference of the Joint Statistical Meetings, 

San Francisco, CA, August 3-7, 2003, 4226-4233.

Thompson, K. (2005). “An Emperical Investigation into the Effects of Replicate Reweighting on Variance 

Estimates for the Annual Capital Expenditures Survey, Proceedings of the Research Conference of 

the Federal Committee on Statistical Methodology, Washington, DC, November 2005.

Wolter, K. M. (1985). Introduction to Variance Estimation. New York: SpringerVerlag.

“Survey of Business Owners (SBO).” Survey of Business Owners. US Census Bureau, n.d. Web. 12 Aug. 

2013.

10