Developing a Unified Sample Design for the Annual Integrated Economic Survey 

Katherine Jenny Thompson, Senior Mathematical Statistician

Associate Directorate for Economic Programs

Any views expressed are those of the author and not necessarily those of the Census Bureau. The Census Bureau has reviewed this data product for unauthorized disclosure of confidential information and has approved the disclosure avoidance practices applied. (Approval ID:  CBDRB-FY22-ESMD006-001).

Acknowledgements 

Sampling Team

Carol Caldwell (ret)

James Burton

James Hunt

Amy Newman-Smith

Laura Bechtel

Nicole Czaplicki

Stephen Kaputa

Adam Smeltz

Justin Z. Smith

Lucas Streng

Michael Christen

Marquita Reynolds

Reviewers

Andreana Able 

Lucas Streng

M3 Unfilled Orders

Covering Old Ground...

Annual Capital Expenditures

Survey

Annual Retail Trade Survey

Annual Wholesale Trade

Survey

Services Annual 

Survey

Annual Survey of Manufactures

Sampling Units:

Company

Employment Identification Number

Sampling Unit:  Establishment

Sampling Unit: Company

Annual Integrated Economic Survey

Sampling Unit: Company

More Old Ground...

ACES

Annual Integrated Economic Survey

Company as Sampling Unit

AWTS

ARTS

SAS

Same Company 

Estab. 1

Estab. 2

ASM 

Estab. 3

Sampling Unit = Company

Option 1: Sample multi-industry company in one industry

Collect roster data 

Company lists all industries

Company reports for all industries

Sampling issues:

Bias in industry-sampling stratum (overrepresentation)

Bias in other industries

    (underrepresentation)

Option 2:  Create more than one artificial sampling unit for each company, with each sampling unit representing expected proportion of activity in industry

Collect industry-specific data

Sampling issue:

Spurious allocation leads to bias!

Current Decision:  Sidestep Both Options

“Complexity” determines initial certainty status

* 4-digit NAICS industry 

**Sectors 44, 45, 62

The designated certainty units are excluded from sampling

 50,000 companies included with certainty (probability = 1)

 250,000 companies sampled (probability < 1)
| Company operates in | Number of Sectors | Number of Industries* | Number of States | Status         | Sampling Weight |
|---------------------|-------------------|-----------------------|------------------|----------------|-----------------|
|                     | 2 or more         | 2 or more             | (Not considered) | Certainty      | 1               |
|                     | 1                 | 3 or more             | (Not considered) | Certainty      | 1               |
|                     | 1                 | 2                     | 2+               | Certainty      | 1               |
|                     | 1                 | 2                     | 1                | Noncertainty** | > 1          |
|                     | 1                 | 1                     | 2+               | Noncertainty   | > 1          |

Sampling Design Requirements (Totals)

National

Revenue/receipts, Annual Payroll, Inventories, Capital Expenditures...

Disaggregated industry (NAICS) levels

Differs by sector

4-digit NAICS, 5-digit NAICS, 6-digit NAICS

Subnational (Geographic)

Revenue/receipts, Annual Payroll, 1st Quarter Employment

Aggregated NAICS levels (3-digit, 4-digit)

“Direct use” states (meet target reliability for Annual Payroll)

26 states

Allocated within region

“Balance” of region category for non-direct use states

Targeted Direct Use States
|           | Region        |           |                |            |
|-----------|---------------|-----------|----------------|------------|
|           | Northeast     | Midwest   | South          | West       |
| State     | Connecticut   | Illinois  | Florida        | Arizona    |
|           | Massachusetts | Indiana   | Georgia        | California |
|           | New York      | Michigan  | Maryland       | Colorado   |
|           | New Jersey    | Minnesota | North Carolina | Nevada     |
|           | Pennsylvania  | Missouri  | South Carolina | Oregon     |
|           |               | Ohio      | Tennessee      | Washington |
|           |               | Wisconsin | Texas          |            |
|           |               |           | Virginia       |            |
| Allocated | 5             | 7         | 8              | 6          |
| Total     | 9             | 12        | 16             | 13         |

Direct Use

Balance of Region

NE

MW

S

W

Stratification (Within Sector)Noncertainty Companies

Industry 1

Industry 2

Industry K

2 Industries

> 1 state

Region 1 

Direct Use State 1



Region 1 

Balance of States

31 Geographic Strata per Industry

Complete set repeated in each industry

Region 4 

Direct Use State 1



Region 4 

Balance of States



AIES strata defined by 

Industry

“Complexity” (company)

Geography

Historical strata defined by

Industry

Unit size

Measure of Size (MOS)

 Establishment value of Annual Payroll for establishment i in company c 

	    (i = 1, 2, ..., nc)

	     Note:  Annual Payroll is available for all units on the sampling frame

 = Company c value of Annual Payroll

Allocation (Within Sector)Noncertainty Companies

Industry 1

Industry 2

Industry K

2 Industries

> 1 state

Region 1 

Direct Use State 1



Region 1 

Balance of States

31 Geographic Strata per Industry

Complete set repeated in each industry

Region 4 

Direct Use State 1



Region 4 

Balance of States



Let

  Number of companies in sector

 Number of companies in stratum h

n  =    Total sample size in sector (predetermined)

For each 2+ company stratum or 2+ state stratum that contains  5 companies, 

Include all  companies in the stratum with certainty

Subtract count from N (get 

Subtract count from n (get 

Region 4 

Balance of States

Region 4 

Direct Use State 1

Region 1 

Balance of States

Region 1 

Direct Use State 1

Power Allocation (Within Sector)Noncertainty Companies

Industry 1

Industry 2

Industry K

2 Industries

> 1 state



31 Geographic Strata per Industry

Complete set repeated in each industry





For all remaining NONCERTAINTY strata

Sum up company Annual Payroll to obtain stratum-level total 

Take square-root of each total

Obtain initial Power Allocation for each stratum

4.  Round to nearest integer value

MOS2 industries

MOS>1 state

MOSDirect Use State 1

MOS Region 1 Balance

MOS Direct Use State 26

MOS Region 4 Balance

Probabilities of Selection (Company)

For each company, need to take into account

Contribution to National estimates

Contribution to Geographic estimates

Not all units are equally important

Small unit at the national level could be “large” at the subnational level

“Medium sized” company could operate in several states

Probability of selection needs to account for contributions to totals at different levels of aggregation

Important Slide

From here on, all examples use completely fictional data generated to prove my points.

Step 1:  Create Pseudo Units
| Company | Establish-ment | Industry | State | MOSic |
|---------|----------------|----------|-------|-------|
| 1       | 1              | AAAA     | MO    | 56    |
|         | 2              | AAAA     | MO    | 9     |
|         | 3              | AAAA     | TN    | 39    |
|         | 4              | AAAB     | MO    | 33    |
|         | 5              | AAAB     | MO    | 20    |
|         | 6              | AAAB     | TN    | 24    |
| 2       | 1              | AAAA     | MO    | 37    |
|         | 2              | AAAA     | TN    | 27    |
| 3       | 1              | AAAA     | MO    | 54    |
| 4       | 1              | AAAB     | MO    | 100   |
| Company | Sampling Stratum | MOSc |
|---------|------------------|------|
| 1       | 2 Industries     | 181  |
| 2       | > 1 State     | 64   |
| 3       | AAAA_MO          | 54   |
| 4       | AAAB_MO          | 100  |

Sampling & Allocation Units

FRAME

Step 1b:  Create National Estimate Pseudo Units
| Company | Establish-ment | Industry | State | MOSic |
|---------|----------------|----------|-------|-------|
| 1       | 1              | AAAA     | MO    | 56    |
|         | 2              | AAAA     | MO    | 9     |
|         | 3              | AAAA     | TN    | 39    |
|         | 4              | AAAB     | MO    | 33    |
|         | 5              | AAAB     | MO    | 20    |
|         | 6              | AAAB     | TN    | 24    |
| 2       | 1              | AAAA     | MO    | 37    |
|         | 2              | AAAA     | TN    | 27    |
| 3       | 1              | AAAA     | MO    | 54    |
| 4       | 1              | AAAB     | MO    | 100   |
| Unit   | Company | “Sampling Stratum”     | MOS |
|--------|---------|------------------------|-----|
| 1_AAAA | 1       | AAAA                   | 104 |
| 1_AAAB | 1       | AAAB                   | 77  |
| 2_AAAA | 2       | AAAA                   | 64  |
| 3_AAAA | 3       | AAAA                   | 54  |
| 4_AAAB | 4       | AAAB                   | 100 |

FRAME

National Industry Units

Step 1c:  Create National Estimate Initial Probabilities of Selection
| Unit   | Company | “Sampling Stratum”     | MOS |
|--------|---------|------------------------|-----|
| 1_AAAA | 1       | AAAA                   | 100 |
| 1_AAAB | 1       | AAAB                   | 77  |
| 2_AAAA | 2       | AAAA                   | 64  |
| 3_AAAA | 3       | AAAA                   | 54  |
| 4_AAAB | 4       | AAAB                   | 100 |

National Industry Units
| Unit   | Company | “Sampling Stratum”     | MOS |       |
|--------|---------|------------------------|-----|--------|
| 1_AAAA | 1       | AAAA                   | 104 | 0.4685 |
| 2_AAAA | 2       | AAAA                   | 64  | 0.2883 |
| 3_AAAA | 3       | AAAA                   | 54  | 0.2432 |
| 1_AAAB | 1       | AAAB                   | 77  | 0.4350 |
| 4_AAAB | 4       | AAAB                   | 100 | 0.5650 |
| Company |                                      |
|---------|---------------------------------------|
| 1       | 1 – ((1-0.4685)(1-0.4350)) = 0.6997   |
| 2       | 0.2936                                |
| 3       | 0.2477                                |
| 4       | 0.5650                                |

Company National 

Probability that Company 1 is NOT sampled in AAAA

Probability that Company 1 is NOT sampled in AAAB

Steps 2a&B : Repeat for Geographic Strata
| Company | Establish-ment | Industry | State | MOSic |
|---------|----------------|----------|-------|-------|
| 1       | 1              | AAAA     | MO    | 56    |
|         | 2              | AAAA     | MO    | 9     |
|         | 3              | AAAA     | TN    | 39    |
|         | 4              | AAAB     | MO    | 33    |
|         | 5              | AAAB     | MO    | 20    |
|         | 6              | AAAB     | TN    | 24    |
| 2       | 1              | AAAA     | MO    | 37    |
|         | 2              | AAAA     | TN    | 27    |
| 3       | 1              | AAAA     | MO    | 54    |
| 4       | 1              | AAAB     | MO    | 100   |

FRAME
| Unit     | Company | Ind | State | MOS |
|----------|---------|-----|-------|-----|
| 1_AAA_MO | 1       | AAA | MO    | 118 |
| 1_AAA_TN | 1       | AAA | TN    | 63  |
| 2_AAA_MO | 2       | AAA | MO    | 37  |
| 2_AAA_TN | 2       | AAA | TN    | 27  |
| 3_AAA_MO | 3       | AAA | MO    | 54  |
| 4_AAA_MO | 4       | AAA | MO    | 100 |

Geographic Industry Units

Steps 2a&B : Repeat for Geographic Strata
| Unit     | Company | Ind | State | MOS |
|----------|---------|-----|-------|-----|
| 1_AAA_MO | 1       | AAA | MO    | 118 |
| 2_AAA_MO | 2       | AAA | MO    | 37  |
| 3_AAA_MO | 3       | AAA | MO    | 54  |
| 4_AAA_MO | 4       | AAA | MO    | 100 |
| 1_AAA_TN | 1       | AAA | TN    | 63  |
| 2_AAA_TN | 2       | AAA | TN    | 27  |

Geographic Industry Units
| Unit     | Company | Ind | State | MOS |       |
|----------|---------|-----|-------|-----|--------|
| 1_AAA_MO | 1       | AAA | MO    | 118 | 0.3819 |
| 2_AAA_MO | 2       | AAA | MO    | 37  | 0.1197 |
| 3_AAA_MO | 3       | AAA | MO    | 54  | 0.1748 |
| 4_AAA_MO | 4       | AAA | MO    | 100 | 0.3236 |
| 1_AAA_TN | 1       | AAA | TN    | 63  | 0.7    |
| 2_AAA_TN | 2       | AAA | TN    | 27  | 0.3    |
| Company |                                    |
|---------|-------------------------------------|
| 1       | 1 – ((1-0.3819)(1-0.70)) = 0.8146   |
| 2       | 1 – ((1-0.1197)(1-0.30)) = 0.3838   |
| 3       | 0.1748                              |
| 4       | 0.3236                              |

Company Geographic 

Step 3: Sampling Unit Probabilities
| Strata                     | Index         | Original Company | National    | Geographic    | InitialCompany    | FinalCompany    |
|----------------------------|---------------|------------------|--------------|----------------|--------------------|------------------|
| 2 Industries               | 1             | 1                | 0.6942       | 0.8146         | 0.8146             |                  |
|                            |            |               |              |                |                    |                  |
|                            | N2 industries | N2 industries    | 0.0652       | 0.0322         | 0.0652             |                  |
| Industry AAAA > 1 State |            |               |           |             |                 |               |
| Industry AAAA MO           |            |                  |              |                |                    |                  |
| Industry AAAB TN           |               |                  |              |                |                    |                  |
| Industry AAAB> 1 State  |               |                  |              |                |                    |                  |
| Industry AAAB MO           |               |                  |              |                |                    |                  |
| Industry AAAB TN           |               |                  |              |                |                    |                  |
| Strata                     | Index         | Original Company | National    | Geographic    | InitialCompany    | FinalCompany    |
|----------------------------|---------------|------------------|--------------|----------------|--------------------|------------------|
| 2 Industries               | 1             | 1                | 0.6942       | 0.8146         | 0.8146             |                  |
|                            |            |               |              |                |                    |                  |
|                            | N2 industries | N2 industries    | 0.0652       | 0.0322         | 0.0652             |                  |
| Industry AAAA > 1 State |            |               |           |             |                 |               |
| Industry AAAA MO           |            |                  |              |                |                    |                  |
| Industry AAAB TN           |               |                  |              |                |                    |                  |
| Industry AAAB> 1 State  |               |                  |              |                |                    |                  |
| Industry AAAB MO           |               |                  |              |                |                    |                  |
| Industry AAAB TN           |               |                  |              |                |                    |                  |

Select MAX
| Strata                     | Index         | Original Company | National    | Geographic    | InitialCompany    | FinalCompany    |
|----------------------------|---------------|------------------|--------------|----------------|--------------------|------------------|
| 2 Industries               | 1             | 1                | 0.6997       | 0.8146         | 0.8146             | 0.7722           |
|                            |            |               |              |                |                    |                  |
|                            | N2 industries | N2 industries    | 0.0652       | 0.0322         | 0.0652             | 0.0524           |
| Industry AAAA > 1 State |            |               |           |             |                 |               |
| Industry AAAA MO           |            |                  |              |                |                    |                  |
| Industry AAAB TN           |               |                  |              |                |                    |                  |
| Industry AAAB> 1 State  |               |                  |              |                |                    |                  |
| Industry AAAB MO           |               |                  |              |                |                    |                  |
| Industry AAAB TN           |               |                  |              |                |                    |                  |

Sum initial probabilities within company sampling strata

Rake each value so that initial probabilities sum to  company sampling strata allocation

Current (Ongoing) Research Topics

Determine PPS-WOR selection method

Requirements

Variance estimation (3+ sampled units per stratum)

Need joint inclusion probabilities within stratum (variances, sampling weights)

Rotation groups and/or permanent random numbers

Determine weights for 

Industry pseudo units (goal: national industry estimates)

Industry-geographic pseudo units (goal: geographic industry estimates)

What about Weights?

Industry 1

Industry 2

Industry K

2 Industries

> 1 state

Region 1 

Direct Use State 1



Region 1 

Balance of States

Region 4 

Direct Use State 1



Region 4 

Balance of States



Direct Use State Estimate

Initial Certainty Companies

Remaining Decisions

Sampling frequency

Annual, quinquennial, other?

Economic indicator design(s)

Monthly and quarterly estimates

Estimation, imputation, variance estimation...

Questions, suggestions, comments?

katherine.j.thompson@census.gov

# image2.png

# image3.svg

# image20.png

# image3.png

# image4.png

# image5.png

# image6.png

# image7.png

# image8.png

# image8.jpeg

# image1.png