An Exploratory Research on 

Optimization of CFS Sampling Design

GASP 2019

Mehdi Hashemipour, PhD

Julie Parker
Commodity Flow Survey (CFS)

• Component of the Economic Census

• Partnership: Conducted by US Census Bureau with Funding from Bureau of 

Transportation Statistics (BTS) and 

• Every 5 years

• Data provide a comprehensive, multimodal picture of national freight flows for 

sampled industries

• Information collected:

• Commodity

• Value

• Weight

• Mode of Transportation

• Destination

• Temperature Control, HAZMAT, Exports
Goals of the Survey

• The goal of the Commodity Flow Survey (CFS) is to make reliable estimates of:

• Annual total value of shipments, 

• Annual total tonnage of shipments, 

• Annual total ton-miles of shipments, and other quantities. 

• We want to make these estimates for various areas of interest such as 

• Geographic origin of the shipment

• Geographic destination of the shipment

• Industry of the establishment making the shipment

• Commodity shipped

• Mode of transportation of the shipment 

• Other variables
Target Population

• The frame population (for the first stage of the sample) consists of 

establishments that are:

• likely to ship physical products, 

• likely to be in business in 2017, and

• for which we have data in the Business Register (the database from which 

we construct the frame).

• Shipper survey of ~100,000 establishments

• Mining

• Manufacturing

• Wholesale Trade

• Select Retail and Services
Questionnaire

Census 

receives 

data

Enter data 

into online 

questionnaire

Current CFS Process and Sample Design

Find 

commodity 

codes

Sample 

Shipments

US Census Bureau Industry Respondent

Sample

Establishments

Sample 

Weeks
Current CFS Process and Sample Design cont.

• The sample design of the first stage is stratified simple random sampling 

without replacement of establishments in the in-scope industries. 

• The strata are (establishment) Origin x Industry x Measure of Size (MOS) size 

classes

• Sample Size:

• To determine the sample size in each stratum, we assign a target coefficient of 

variation (CV) to each origin x industry cell. 

• Within each origin x industry cell, we determine the boundaries of the MOS 

size class cells that minimize the sample size needed to achieve the target 

CV, where the sample is allocated according to the Neyman Allocation 

among the noncertainty strata. 
Joint Stratification and Allocation Method 

• In this exploratory research, we applied the optimal stratification and 

allocation method based on Genetic Algorithm (GA) and Simulated 

Annealing (SA) in a CFS like scenario. 

• Objective:

• With this method, we aim at minimizing the total sample cost while 

satisfying the target Coefficient of Variation (CV) constraints. 

• Experimenting these methods by using domain variables such as different 

CFS areas and Industries may result in different CFS sampling design.
Search Techniques

Calculus Base 

Techniques

Guided random search 

techniques

Enumerative 

Techniques

DFS BFS Dynamic 

Programming

Tabu Search Hill Climbing Evolutionary 

Algorithms

Genetic 

Programming

Genetic 

Algorithms

Fibonacci Sort

Simulated 

Anealing

Classes of Search Techniques
• A class of probabilistic optimization algorithms

• Inspired by the biological evolution process

• Uses coミcepts of さNatural “electioミざ aミd さGeミetic 

Iミheritaミceざ ふDarwiミ 1Β5Γぶ

• known about the underlying search space

• Widely-used in business, science and engineering

• A genetic algorithm maintains a population of candidate 

solutions for the problem at hand, and makes it evolve by

iteratively applying a set of stochastic operators

Genetic Algorithms (GA) OVERVIEW
selection

population evaluation

crossover

discard

deleted 

members

parents

modified

offspring

evaluated offspring

initiate &

evaluate

The Evolutionary Cycle
Optimal stratification and allocation 

based on Genetic Algorithm for CFS

• The optimal stratification and allocation method aims at minimizing the total 

sample cost while satisfying the precision (CV) constraints. 

• This method explores the set of all possible stratifications (referred to as the 

universe of stratifications) based on atomic strata which is the most detailed 

stratification derived from the Cartesian product of all auxiliary variables as the 

solution space. 

• The objective function minimizes the total sampling cost. Cost of sampling per 

unit can be set according to the effort associated with collecting and processing 

each unit. 

• Barcaroli (2014) implemented this method in and R package titled 

さSamplingStrataざ. 

Barcaroli, G. (2014a). SamplingStrata: An R Package for the Optimization of Stratified Sampling. Journal of Statistical Software, 

61(1), 1–24. https://doi.org/10.18637/jss.v061.i04
Genetic Algorithm Procedure

• Iミ the first step, the iミput data is read aミd loaded iミ さCFSFrameDataざ マatrix.

CFSFrameData <- read.csv(file="./100K_Frame.csv", header=TRUE, sep=",")

• Then, the frame based on the loaded data is created as follows. 

CFSFrame <- buildFrameDF(df = CFSFrameData,

id = "estno",

X = c("state","county","naics"),

Y = c("value"),

domainvalue = "naics")

• Atomic strata which is the most detailed strata resulting from the cartesian

product of all auxiliary variables is then constructed and stored in 

さAtoマic“trataざ マatrix. 

AtomicStrata <- buildStrataDF(CFSFrame, progress = TRUE)
Atomic strata for the case study

• (X1, X2, X3, and X4): auxiliary variables 

• N: the number of units (establishments) in each stratum, 

• M1 and S1: mean and standard deviation of the value for each stratum, 

• Cost: the assigned sampling costs

• CENS: certainty

• DOM1: domain variable (industry)
Genetic Algorithm Procedure cont.

• Next, CV constraints for each domain is imported.

CVConst <- read.csv("./CV.csv", header=TRUE, sep=",")

• The last step is to call さoptimizeStrataざ fuミctioミ which perforマs the optiマal 

stratification and allocation based on Genetic Algorithm. 

solution <- optimizeStrata(errors = CVConst, 

strata = AtomicStrata,

parallel = TRUE,

iter = 100,

writeFiles = FALSE,

showPlot = FALSE)
• Applying Simulated Annealing (SA) Method

• Combining the GA and SA method and create a New Method

• Comparing New Methods with the Current CFS Sampling 

Design and Finding the Difference

• Future Update:

• The Sixth International Conference on Establishment Statistics (ICES 

VI) will be held in New Orleans, Louisiana, USA, June 15–18, 2020.

Future Work
Questions and Comments

M.Hashemipour@dot.gov

Julie.Parker@dot.gov

Special thanks to: 

Saeed Ghanbartehrani

Sara Akbar Ghanadian