How Can We Best Measure an Establishment's Size?An Evaluation of Universe Extraction for Select U.S. Census Bureau Business Surveys

November 2, 2021 

Presented by: Kenishea Donaldson 

Additional Authors: Katrina Washington, Erica Wong, Olivia Brozek, Jacob Sandruck, Micah Tyler

U.S. Census Bureau 

Any views expressed are those of the author(s) and not necessarily those of the U.S.  Census Bureau. The Census Bureau has reviewed this data product for unauthorized disclosure of confidential information and has approved the disclosure avoidance applied. (Approval ID:CBDRB-FY22-ESMD005-002)

1

Topics Covered

Purpose

Introduction to key terms

Current Methodology

Annualization of Input Data

Editing of Input Data

Research Findings and Results

Improvements and Recommendations

2

2

Purpose

The objective of this research is to improve upon the methodology used to determine and validate the measure of size estimates for employer establishments in preparation for the next sample revision for select U.S. Census Bureau economic surveys

3

3

Terms to Remember 

Business Sample Revision (BSR)

Follows completion of the U.S. Census Bureau’s Economic Census

Sample is reselected 

Monthly, quarterly and annual Retail, Wholesale and Service surveys

Business Register (BR)

Database of domestic businesses and establishments 

Compilation of data from censuses, surveys, and administrative records 

Contains data at the establishment, EIN, and enterprise/firm/company level

Singleunit establishment

Multiunit establishment

4

4

Terms to Remember

Universe Extraction (UE)

Process used to build a frame of all in-scope establishments 

Economic Census and BR data used to determine revenue Measure of Size (MOS) for each establishment 

Run edits on the establishment data used in MOS calculation to define impact on the sample size 

5

5

Gimme All Your Money...Data

Universe Extraction creates universe of all ‘in-scope’ establishments which may be selected 

Pulling current data from the BR

BIRTHDAY (month, day, year) establishment became active

Receipts and payroll from the Economic Census 

Administrative receipts and payroll (most recent 3 years)

‘in-scope’

establishment

Business 

Register

(BR)

UE Frame

BIRTHDAY

2012 Econ Census Data 

Admin Receipts & Payroll

Annualized

 2012 Econ Census Data

Annualized

Admin Receipts & Payroll

6

6

Data Annualization	

BIRTHMONTH (BM) and BIRTHYEAR are derived from BIRTHDAY and used to annualize administrative receipts 

7

For administrative data: 

For Economic Census data: 

7

Improvements to Data Annualization	

Issues with the annualizing variable – BIRTHDAY

Based on posting date

Assigned at EIN level

Annual filers always receive January as tax month 

Immediate goal became finding a variable that is as populated as BIRTHDAY but is a more accurate representation of when an establishment first came into business as an employer

8

8

Data Annualization 

Includes two recommended Social Security Administration dates

Needed a method for verifying accuracy of variables  

9
|              | Singleunits             |        |           |       | Multiunits              |        |           |       |
|--------------|-------------------------|--------|-----------|-------|-------------------------|--------|-----------|-------|
|              | Total Count = 4,177,000 |        |           |       | Total Count = 1,745,000 |        |           |       |
| New Variable | Populated               |        | Missing   |       | Populated               |        | Missing   |       |
| BIRTHDAY     | 4,177,000               | 100.0% | <15    | 0.0%  | 1,745,000               | 100.0% | <15    | 0.0%  |
| BIRTH_PAY    | 4,176,000               | 100.0% | 700       | 0.0%  | 1,686,000               | 96.6%  | 60,000    | 3.4%  |
| BIRTH_RCPT   | 3,810,000               | 91.2%  | 367,000   | 8.8%  | 1,213,000               | 69.5%  | 532,000   | 30.5% |
| BIRTH_START  | 2,196,000               | 52.6%  | 1,981,000 | 47.4% | 273,000                 | 15.6%  | 1,472,000 | 84.4% |
| BIRTH_WAGES  | 1,421,000               | 34.0%  | 2,756,000 | 66.0% | 199,000                 | 11.4%  | 1,546,000 | 88.6% |

Availability of comparison variables pulled from the BR

Data reflect the BR as of May 2020

9

Verifying Variables

PAYFIRST was created to represent the first occurrence of non-zero payroll on the BR

Only used non-missing and non-zero estimates and comparison dates from 2012 or later 
|                          | Singleunits |                 |                |
|--------------------------|-------------|-----------------|----------------|
| Comparison Date Variable | Equal       | PAYFIRST before | PAYFIRST after |
| BIRTHDAY                 | 94.6%       | 5.1%            | 0.4%           |
| BIRTH_PAY                | 94.5%       | 5.3%            | 0.4%           |
| BIRTH_RCPT               | 18.5%       | 59.2%           | 22.2%          |
| BIRTH_START              | 28.6%       | 2.8%            | 68.8%          |
| BIRTH_WAGES              | 53.7%       | 9.9%            | 36.2%          |

10

Comparison of date variable vs. first occurrence of payroll data

Data reflect the BR as of May 2020

10

Creation of a Hierarchy 

BIRTH_WAGES

BIRTH_PAY

BIRTH_START

BIRTHDAY

11

BEST 

BUSINESS START DATE

11

Validating the Hierarchy 

The current methodology using BIRTHDAY is more effective than the proposed hierarchy 
|                          | Singleunits |                |                 |
|--------------------------|-------------|----------------|-----------------|
| Comparison Date Variable | Equal       | PAYFIRST after | PAYFIRST before |
| BIRTHDAY                 | 94.6%       | 0.4%           | 5.1%            |
| BIRTH_PAY                | 94.5%       | 0.4%           | 5.3%            |
| BIRTH_START              | 28.6%       | 68.8%          | 2.8%            |
| BIRTH_WAGES              | 53.7%       | 36.2%          | 10.0%           |
| HIERARCHY                | 75.1%       | 17.3%          | 7.5%            |

12

Updated comparison of date variable vs first occurrence of payroll data

12

13

Creation of MOS

Six established MOS formulas used in BSR-17

Estimates annual receipts for given year

Uses annualized data from BR administrative sources, Census parameters and surveys

Creation of MOS:  BSR-17 Formulas 

MOS6 = Beta Factor * 2014 Admin Payroll 

MOS5 = 

MOS3 

= 2012 Census Receipts * 

MOS1† = 2014 Admin Receipts 

MOS2† 

= 2013 Admin Receipts * 

MOS4 

= 2012 Census Receipts * 

† For singleunits only

14

14

MOS Editing 

Ratios using the MOS input data are created and tested against edit boundaries

The 1st and 99th percentiles were set as edit boundaries for each ratio test

The ‘best’ MOS is set based on which ratios pass edits

The number portion of the MOS indicates optimal choice of method

15

15

| SU/MU | Measure of Size |                                    | Corresponding Ratio Edit   |                                                                                                                                                                                               |
|-------|-----------------|------------------------------------|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| SU    | MOS2 =          | 2013 Annualized Admin Receipts *   | RAT4 =  RAT6 =             | 2014 Annualized Admin Payroll 2013 Annualized Admin Payroll 2013 Annualized Admin Receipts 2013 Annualized Admin Payroll                                                                      |
| Both  | MOS3 =          | 2012 Annualized Census Receipts *  | RAT1 =  RAT2 =  RAT3 =     | 2014 Annualized Admin Payroll  2012 Annualized Census Receipts 2012 Annualized Census Payroll 2014 Annualized Admin Payroll  2012 Annualized Census Payroll 2012 Annualized Census Receipts   |
| Both  | MOS4 =          | 2012 Annualized Census Receipts  * | RAT1 =   RAT5 =            | 2014 Annualized Admin Payroll 2012 Annualized Census Receipts 2012 Annualized Census Receipts2012 Annualized Admin Payroll                                                                    |

BSR-17 MOS ratio tests for singleunits and multiunits

16

16

Input Data Issues

Approximately 2% of the administrative receipts data values truncated

Affected only the 2014 and/or 2013 administrative receipts data

Used in MOS calculations for MOS1 and MOS2 

Affects MOS calculations for singleunit establishments only

MOS1 truncated data used

For MOS2, a ratio edit existed (RAT6) that potentially flagged the truncated data

Ratio Edit 6 Results by 2013 administrative receipts data disposition for singleunits

17

17

Improvements to MOS Editing 

Data truncation issue went largely undetected 

Need method to more effectively edit the input data used in MOS calculations

Tested several methods of outlier detection methodology

Method should be more robust than the ratio edits used currently

18

18

Outlier Detection Methodology 

Traditional Ratio Edits

Detect if ratio of two variables for a given unit differs markedly from the ratios of the remaining units

Does not account for size differences between units

Many small estimates flagged as outliers, unlike large estimates

Hidiroglou-Berthelot (HB) outlier detection methodology

Incorporates size component to reduce “size masking effect” and transforms ratios to ensure symmetry

Acceptance boundaries vary according to size of unit 

Finds both large and small outliers, on both tails of distribution 

19

19

Result of Ratio Edits on 2013 Administrative Receipts 
| Ratio 6 Flag      | 2013 Not Truncated |        | 2013 Truncated |        |
|-------------------|--------------------|--------|----------------|--------|
| Method            | 1st/99th           | HB     | 1st/99th       | HB     |
| Total Singleunits | 4,093,000          |        | 84,000         |        |
| RAT6 flagged      | 57,000             | 40,000 | 5,600          | 10,000 |
| Percent Flagged   | 1.4%               | 1.0%   | 6.7%           | 11.9%  |

Comparison of RAT6 results by method for 

2013 administrative receipts data disposition for singleunits

20

20

New Ratio Edit on 2014 Administrative Receipts 
|                                   | 2014 Truncation |           |
|-----------------------------------|-----------------|-----------|
| RAT7 Flag                         | Not Truncated   | Truncated |
| RAT7 not flagged                  | 4,112,000       | 7,000     |
| RAT7 flagged                      | 45,500          | 12,500    |
| Total                             | 4,157,000       | 19,500    |
| Total Truncations Flagged = 64.1% |                 |           |

Singleunits with non-missing 2014 administrative receipts using HB edit on RAT7

2014 Annualized Admin Receipts 

2014 Annualized Admin Payroll

RAT7 = 

21

21

Assignment of “best” MOS for singleunits with 2013 truncated data
|                           |       | NEW MOS (HB Outlier Detection) |        |        |        |        |        |        |
|---------------------------|-------|--------------------------------|--------|--------|--------|--------|--------|--------|
|                           |       | 1                              | 2      | 3      | 4      | 5      | 6      | Total  |
| OLD MOS(1st/99th Percent) | 1     | 65,000                         | 1,600  | 250    | 40     | 30     | 150    | 67,500 |
|                           | 2     | <15                         | 13,500 | 1,000  | 50     | <15 | 40     | 14,500 |
|                           | 3     | <15                         | 250    | 1,000  | 20     | <15 | <15 | 1,200  |
|                           | 4     | <15                         | 30     | 40     | 40     | <15 | <15 | 100    |
|                           | 5     | <15                         | 60     | 50     | 70     | <15 | <15 | 200    |
|                           | 6     | <15                         | 60     | <15 | <15 | <15 | 450    | 550    |
|                           | Total | 65,000                         | 15,500 | 2,300  | 200    | 60     | 650    | 84,000 |
|                           |       | NEW MOS (HB Outlier Detection) |        |        |        |        |        |        |
|---------------------------|-------|--------------------------------|--------|--------|--------|--------|--------|--------|
|                           |       | 1                              | 2      | 3      | 4      | 5      | 6      | Total  |
| OLD MOS(1st/99th Percent) | 1     | 6,800                          | 11,500 | 300    | 20     | 40     | 150    | 18,500 |
|                           | 2     | <15                         | 1,000  | <15 | <15 | <15 | <15 | 1,000  |
|                           | 3     | <15                         | 20     | 50     | <15 | <15 | <15 | 70     |
|                           | 4     | <15                         | <15 | <15 | <15 | <15 | <15 | <15 |
|                           | 5     | <15                         | <15 | <15 | <15 | <15 | <15 | 20     |
|                           | 6     | <15                         | <15 | <15 | <15 | <15 | 30     | 30     |
|                           | Total | 6,800                          | 12,500 | 350    | 20     | 40     | 150    | 19,500 |

Assignment of “best” MOS for singleunits with 2014 truncated data

22

22

Summary

Improvements to Data Annualization

Sought out to resolve issues with annualizing variable, BIRTHDAY

Creation of a hierarchy to use more BR data 

Proposal of BIRTH_PAY as a replacement 

Improvements to MOS Editing

Review of current BSR-17 methodology which uses 1st and 99th percentiles 

Research HB outlier detection as a more robust option

Creation of additional test ratio (RAT7) to edit input data for MOS1 creation

23

23

Acknowledgements 

Thank you to everyone who helped during this entire research process, including James Burton, James Hunt, Brandy Yarbrough and Joseph Barth. This paper would not be possible without your knowledge, feedback, and support.

24

24

For any further questions/comments/feedback, 

please contact the author at

Kenishea.Donaldson@census.gov

25

25

# image5.png

# image7.png

# image8.png

# image9.png

# image10.png

# image11.png

# image4.emf

# image1.png