Evaluating Imputation Models for 

Totals Considering Missing Patterns

2023 FCSM Research & Policy Conference

Session E-6: The Missing Data Puzzle: Exploring Imputation Methods

2023-10-25

Young-Jun Kweon

Bureau of Transportation Statistics
This study was performed under the sponsorship of the Department of Transportation in 

the interest of information exchange. The U.S. Government assumes no liability for its 

contents or use thereof.

2

Disclaimer
• INTRODUCTION

NCFO / Imputation Study

• DATA

Questionnaire / Released Tables

• METHODS 

Analysis Flow / Individual Models / Combined Models

• RESULTS

Individual Models / Combined Models / Overall Best Models

• CONCLUSIONS

Outline

3
INTRODUCTION

4
Add a footer

National Census of Ferry Operators

• The Safe, Accountable, Flexible, Efficient, Transportation Equity Act–A Legacy for 

Users (SAFETEA-LU) of 2005 (P.L. 114-94) requires BTS to maintain a national ferry 

database. 

• BTS conducts a biennial census of all ferry operators in the U.S. and its territories. 

• Who should be included?

• Ferry operators providing itinerant, fixed route, common carrier passenger/vehicle rollon, roll-off (RoRo) ferry service, and railroad car float operations

• Who should NOT be included?

• Non-itinerant operations (e.g., cruise-to-nowhere services)

• Excursion (e.g., whale watches, casino boats, dinner cruises, etc.)

• Passenger only water taxi services not operating on a fixed route

• LoLo (Lift-on/Lift-off) freight/auto carrier services

• Long distance passenger only cruise ship services

5
Add a footer

• Background:

 Passenger and vehicle boarding counts are key information

 34% of ferry segments have missing passenger boarding in 2020 NCFO

 Missing counts make it difficult to monitor changes in national totals

• Purpose:

 To impute annual boarding counts for estimating national totals (not for releasing 

imputed values to the public or for performing analysis on imputed data)

• 2020 NCFO

 246 operators invited (frame) 

 164 operators responded

2020 NCFO Imputation Study

6
DATA

7
Add a footer

• 5 Sections: (1) Operator Info, (2) Vessel Info, (3) Terminal Info, 

(4) Segment Info, and (5) Funding Info. 

2020 NCFO Questionnaire

8
Add a footer

2020 NCFO Data Release

5 Tables Operator Operator-Segment Segment Terminal Vessel

152 Variables 50 41 7 20 34

9
METHODS

10
Add a footer

Analysis Flow: Data Prep 

11
Add a footer

Analysis Flow: Modeling

12
Add a footer

• Total Error

• Model Selection

 Visual Examination: Boxplot and Density Plot

 Statistical Test: Paired Sample Wilcoxon Test

Individual Models 

13
Add a footer

• Model Combinations: A total of 4,083 Combinations

∙ 2-Model: CART-CRF, CART-MRF, ..., RF-AML = = 66

∙ 3-Model: CART-CRF-MRF, ..., KNN-RF-AML = = 220

∙ ...

∙ 12-Model: CART-CRF-MRF- ... - AML= = 1

∙ Averaged Total Error 

Combined Models 

14
RESULTS

15
Add a footer

Individual Models 

16

• Boxplot

Top 12 Individual 

Models

1 CART 

2 CRF 

3 MRF 

4 SHD 

5 ENG 

6 LM 

7 IRMI 

8 EM 

9 PMM 

10 KNN 

11 RF 

12 AML 
Add a footer

Combined Models ➊

17

• Connected Violin Plot

Top 12 Individual 

Models

1 CART 

2 CRF 

3 MRF 

4 SHD 

5 ENG 

6 LM 

7 IRMI 

8 EM 

9 PMM 

10 KNN 

11 RF 

12 AML 

4,083 Combinations 
Add a footer

Combined Models ➋

18

• Boxplot

Top 10 Models
Add a footer

Top Individual & Combined Models

19
CONCLUSIONS

20
Add a footer

• Some models show very poor imputation performance: ENP, RLM, RR. 

• Attributable to the evaluation metric

• AutoML does not live up to expectation

• Averaging results from individual models stabilizes the distribution of 

total errors. But, majority of variation stays

• Thus, the best model cannot be determined statistically

Conclusions

21

• Imputation for estimating totals

• Limited to this study
Add a footer

What’s Next?

22
Add a footer

• Clara Reschovsky, NCFO Program Manager/Survey Statistician, USDOT-BTS

• Cha-Chi Fan, Office Director, USDOT-BTS

Acknowledgement

23
Contact

Young-Jun Kweon

young-jun.kweon@dot.gov

NCFO

ferry@dot.gov
Questions?

25