Analyzing Nonresponse in 

National Census of Ferry Operators

2022 FCSM Research & Policy Conference

Session E-3: Modernizing Bias Reduction Techniques

2022-10-26

Young-Jun Kweon

Bureau of Transportation Statistics
This study was performed under the sponsorship of the Department of Transportation in 

the interest of information exchange. The U.S. Government assumes no liability for its 

contents or use thereof.

2

Disclaimer
• INTRODUCTION

NCFO / NRB Study

• DATA

Questionnaire / Examples / Released Tables

• METHODS 

Data Quality Metrics / Bias Estimation / Influential Variables

• RESULTS

• CONCLUSIONS

Outline

3
INTRODUCTION

4
Add a footer

National Census of Ferry Operators

• The Safe, Accountable, Flexible Efficient, Transportation Equity Act–A Legacy for 

Users (SAFETEA-LU) of 2005 (P.L. 114-94) requires BTS to maintain a national ferry 

database. 

• BTS conducts a biennial census of all ferry operators in the U.S. and its territories. 

• Who should be included?

• Ferry operators providing itinerant, fixed route, common carrier passenger/vehicle rollon, roll-off (RoRo) ferry service, and railroad car float operations

• Who should NOT be included?

• Non-itinerant operations (e.g., cruise-to-nowhere services)

• Excursion (e.g., whale watches, casino boats, dinner cruises, etc.)

• Passenger only water taxi services not operating on a fixed route

• LoLo (Lift-on/Lift-off) freight/auto carrier services

• Long distance passenger only cruise ship services

5

INTRODUCTION
Add a footer

• Motivations:

 Recommended by OMB Guidelines (2006)

 Improve data quality of NCFO

 Perform trend analysis of national total boarding counts

• Purposes:

 To improve data quality of NCFO

 To inform data users of potential bias in analyzing 2020 NCFO data

• 2020 NCFO

 247 operators invited (frame)

 164 operators participated

2020 Nonresponse Bias Study

6

INTRODUCTION
DATA

7
Add a footer

• 5 Sections: (1) Operator Info, (2) Vessel Info, (3) Terminal Info, 

(4) Segment Info, and (5) Funding Info. 

2020 NCFO Questionnaire

8

DATA
Add a footer

• An operator serves 2 segments connecting 2 terminals with 1 vessel

Ferry Operation: Example 1

T1 T2 V1

S1

S2

9

DATA
Add a footer

• An operator serves 9 segments connecting 4 terminals with 5 vessels

Ferry Operation: Example 2

T1 T2

T3

T4

S7

S3

V1

S1

S2

S4 V4

S5 S6

V2 V3

V1

V4

S8 V1

S9 V1

V5

10

DATA
Add a footer

2020 NCFO Data Release

5 Tables Operator Operator-Segment Segment Terminal Vessel

158 Variables 51 42 9 21 35

11

DATA
METHODS

12
Add a footer

Unit Response

Definition of Response

• A unit answers all key questions of NCFO

• For 2020 NCFO, 19 key questions were designated.

13

METHODS
Add a footer

• 6 Data Quality Rates

• 2 Unit-Level Rates

➀ Unit Participation Rate (UPR)

➁ Unit Response Rate (URR)

• 4 Item-Level Rates

➂ Item Response Rate (IRR)

➃ Total Item Response Rate (TIRR)

➄ Overall Item Response Rate (OIRR)

➅ Modified Quantity Response Rate (MQRR)

 Proportion of observed total value of item x

Data Quality Metrics

Lineback and Thompson (2010). Conducting Nonresponse Bias Analysis 

for Business Surveys. JSM 2010. 

14

METHODS
Add a footer 15

• Define Nonresponse Bias

• In sample mean of survey (OMB, 2006)

BBBBBBBB xx̅

RR = xx̅RR − xx̅SS =

nnNN

nnSS

xx̅

RR − xx̅NNRR

• In sample total of census (2020 NCFO)

BBBBBBBB TT�

xx,RR = TT�xx,RR − TT�xx,SS where TT�xx,SS = TT�xx,RR + TT�xx,NN

Bias Estimation ➊

15

METHODS
Add a footer

• Calculate TT�

xx,NN for Passenger Boarding 

• Use growth ratio of 2018-2020: GGGGGGGGGGG RR PPPPPPPP = TT�PASS,RR,2020

TT�PASS,RR,2018

• (e.g.) PPPPPPPP �ii,NN ,2020 = GGGGGGGGGGG RR PPPPPPPP × PPPPPPPPii,RR,2018

• Growth ratio by Measure of Size (MOS)

Bias Estimation ➋

Group Description Definition Num. of 

Operators

Total 

Boarding Proportion

1 Extra Large (XL) >= 10,000,000 3 60,942,777 40%

2 Large (L) 10,000,000 > & >=2,000,000 12 41,403,641 27%

3 Medium (M) 2,000,000 > & >= 500,000 33 34,583,713 23%

4 Small (S) 500,000 > & > 0 129 14,845,905 10%

16

METHODS
Add a footer

• Method: Conditional Tree Analysis

• Variables considered:

➀ Accept Public Fund (Yes, No)

➁ Serve National Park Service (Yes, No)

➂ Report on behalf of government (Yes, No)

➃ Segment Type (Interstate, Intrastate, International)

➄ Population (Based on American Community Survey’s 2019 estimate)

➅ State (Based on operator’s address) 

Influential Variables

17

METHODS
RESULTS

18

2020 NCFO

Data Collection Year

2022 NCFO

Data Year
Add a footer

• Unit Participation & Response Rates: UPR & URR

2 Unit-Level Rates 

Subgroup Category UPR URR

All None 66% 50%

Report On Behalf 

of Government

No 67% 46%

Yes 90% 73%

Ticket Revenue < 50% 87% 67%

≧50% 73% 56%

Number of 

Segments

2 77% 59%

3-6 72% 57%

6+ 87% 61%

Number of 

Vessels

1-2 71% 55%

3-6 82% 63%

6+ 82% 56%

Higher UPR & URR (Report On Behalf of 

Gov.)

Higher UPR & URR (Ticket Revenue < 50%)

Higher UPR (6+ Segments)

Higher UPR (3+ Vessels)

19

RESULTS
Add a footer

• Item Response Rate (IRR) and Total Item Response Rate (TIRR)

4 Item-Level Rates ➊

19 Key Response Variable (x) Table IRRx TIRRx

Accept Public Funding

Operator

100.0% 50.0%

Operator City 100.0% 50.0%

Operator State 100.0% 50.0%

Operator Name 100.0% 50.0%

Average Trip Time

Operator 

Segment

81.1% 40.5%

Most Used Vessel 81.1% 40.5%

Passenger Boarding 81.1% 40.5%

Segment Length 81.1% 40.5%

Season End Date 78.7% 39.3%

Season Start Date 78.7% 39.3%

Vehicle Boarding 81.1% 40.5%

Segment Name Segment 81.1% 40.5%

Terminal 1 City

Terminal

98.8% 49.4%

Terminal 2 City 89.0% 44.5%

Terminal 1 State 98.8% 49.4%

Terminal 2 State 89.0% 44.5%

Terminal 1 Name 98.8% 49.4%

Terminal 2 Name 89.0% 44.5%

Vessel Name (of Most User Vessel) Vessel 81.1% 40.5%

Lower IRR & TIRR

20

RESULTS
Add a footer

• Overall Item Response Rate (OIRR) & 

Modified Quantity Response Rate (MQRR)

4 Item-Level Rates ➋

Variable (x) OIRRx MQRRx

Passenger Boarding 54.3% 77.2%

Based on values in x

respondents provided

Based on the number 

of respondents in x

21

RESULTS
Add a footer

Group 1

33%

Group 2

27%

Group 3

27%

Group 4

13%

(3)

(9)

(43)

(1)

• Growth Ratios by MOS Group

• Estimated Bias in Total Boardings

Bias Estimate (Passenger Boardings) 

Group Growth Ratio

1 (Extra-Large Operators) 0.992

2 (Large Operators) 0.981

3 (Medium Operators) 1.105

4 (Small Operators) 1.053

Group Nobs Type 2020 NCFO

All 177

Total Observed 113,990,893

Estimated 153,852,554

Bias Number -39,861,661

Percentage -35%

• Contribution to Total Bias 

in 2020 NCFO

22

RESULTS
Add a footer

• Nonresponse to All 19 Key Variables

• Without State Variables

Influential Variables

• Nonresponse on All 19 Key Variables

• With State Variables

Not accept public fund

Not serve National Park, 

Not report on behalf of government

Serve intrastate segment

In Florida

Terminals in an area with population > ~300K 

23

RESULTS
CONCLUSIONS

24
Add a footer

• Rates vary across subgroups (e.g., reporting on behalf of government, 

ticket revenue, and fleet size)

 Data users should be cautious when focusing their analysis on a specific group

• Bias in total passenger boarding is affected largely by nonresponding big 

operators

 Estimating national total boarding count is challenging

• Operators in few States with certain characteristics have lower 

responses. 

Conclusions

25

CONCLUSIONS
Add a footer

• RECOMMENDATION: Develop a monitoring application keeping track 

of response on each of the key items

 Use: The application identifies nonresponding ferry operators as a census rolls out.

 Implication: This recommendation would increase the unit response rate (URR). 

• RECOMMENDATION: Develop a list of ferry operators with a group 

indication based on boarding counts. 

 Use: This list helps BTS identify which ferry operators would be critical in reducing bias in the 

national total boarding estimate. 

 Implication: This recommendation would increase the unit response rate (URR) and the itemlevel rates on the 2 key items (passenger and vehicle boardings).

Recommendations

26

CONCLUSIONS
Add a footer

• Aubrey Nguyen, IT Auditor, U.S. Government Accountability Office (GAO)

ORISE Fellow, USDOT (Formerly)

• Clara Reschovsky, NCFO Program Manager/Survey Statistician, USDOT

Acknowledgement

27
Contact

Young-Jun Kweon

young-jun.kweon@dot.gov

NCFO

ferry@dot.gov
Questions?

29