Respondent Device Analysis for the Internet 

Instrument of the American Community Survey

Gregory J. Mills

R. Chase Sawyer

American Community Survey Office

Lindsay Longsine

Decennial Statistical Studies Division

Federal Committee on Statistical Methodology

October 27, 2022

This presentation is released to inform interested parties of ongoing research and to encourage discussion of work in progress. Any views 

expressed on statistical, methodological, or technical issues are those of the authors and not necessarily those of the Census Bureau.

DRB Approval Number: CBDRB-FY23-ACSO003-B0043
American Community Survey

The Foundation

The American Community Survey is on the leading edge of survey design, continuous 

improvement, and data quality

• The nation’s most current, reliable, and accessible data source for local statistics on 

critical planning topics such as age, children, veterans, commuting, education, 

income, and employment

• Surveys 3.5 million addresses and informs over $675 billion of Federal government 

spending each year

• Covers 40+ topics, supports over 300 evidence-based Federal government uses, 

and produces 11 billion estimates each year

• Three key annual data releases:

• 1-year Estimates (for large populations, geographies of 65,000+ population)

• 1-year Supplemental Estimates (for small populations, geographies of 20,000+ 

population)

• 5-year Estimates (for very small populations, geographies down to Census 

Tracts and Block Groups)

2
ACS Data Collection Process

3
Background

• ACS internet instrument launched in production in 2013

• 2011 Internet Test showed 95% of logins came from PCs and laptops, very little 

mobile phone (0.9%) or tablet usage (3.6%) (Hortwitz et al., 2012)

• Purpose of this report is to provide an update to these figures, and explore other 

device usage trends

4
Research Goals

1. To understand how device usage trends on the ACS have changed over time

2. To understand how internet instrument usage patterns (multiple logins, changing 

device types, etc.) are associated with device type 

3. A preliminary look at how demographic characteristics are associated with device type 

usage

5
Data

• Useragent string information

• Device type, operating system, browser, timestamps

• ACS control file systems for case dispositions

• Unedited response data

• 2013-2021 ACS sample cases for historical analyses

• 2019 ACS sample cases for cross sectional analyses

6
Research Goal 1. To understand how device usage 

trends on the ACS have changed over time

7
0

10

20

30

40

50

60

70

American Community Survey Self Response Rates, 

2013-2019 and 2021 (chart excludes 2020 panels)

Total Self Response Internet Mail

8

<-2019 | 2021->
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 ... 2019 2019-2013 P-value

PC 90.3 (<0.1) ... 73.1 (0.1) -17.2 (0.1) <0.01

Mobile 2.2 (<0.1) ... 18.3 (<0.1) 16.1 (<0.1) <0.01

Tablet 7.4 (<0.1) ... 8.6 (<0.1) 1.2 (<0.1) <0.01

9

• PC usage fell from 90.3 to 73.1 percent

• Mobile phone usage increased from 2.2 to 18.3 percent

• Less clear trend for tablets going forward
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

10

• PC usage fell from 90.3 to 73.1 percent

• Mobile phone usage increased from 2.2 to 18.3 percent

• Less clear trend for tablets going forward

Categories 2013 2015 2017 2019

PC 90.3 (<0.1) 83.6 (<0.1) 78.2 (0.1) 73.1 (0.1)

Mobile 2.2 (<0.1) 5.9 (<0.1) 11.8 (<0.1) 18.3 (<0.1)

Tablet 7.4 (<0.1) 10.5 (<0.1) 10.0 (<0.1) 8.6 (<0.1)
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 2015 2017 2019 2021

PC 90.3 (<0.1) 83.6 (<0.1) 78.2 (0.1) 73.1 (0.1) 70.4 (0.1)

Mobile 2.2 (<0.1) 5.9 (<0.1) 11.8 (<0.1) 18.3 (<0.1) 25.9 (<0.1)

Tablet 7.4 (<0.1) 10.5 (<0.1) 10.0 (<0.1) 8.6 (<0.1) 3.6 (<0.1)

11
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 2019 2019-2013 P-value

PC 90.3 (<0.1) 73.1 (0.1) -17.2 (0.1) <0.01

Mobile 2.2 (<0.1) 18.3 (<0.1) 16.1 (<0.1) <0.01

Tablet 7.4 (<0.1) 8.6 (<0.1) 1.2 (<0.1) <0.01

12
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 2019 2019-2013 P-value

PC 90.3 (<0.1) 73.1 (0.1) -17.2 (0.1) <0.01

Windows 85.2 (<0.1) 75.3 (<0.1) -9.9 (0.1) <0.01

Mac OS X 14.3 (<0.1) 21.9 (<0.1) 7.7 (0.1) <0.01

Chrome OS 0.2 (<0.1) 2.3 (<0.1) 2.1 (<0.1) <0.01

Linux 0.2 (<0.1) 0.3 (<0.1) 0.1 (<0.1) <0.01

Other PC 0.2 (<0.1) 0.2 (<0.1) >-0.1 (<0.1) 0.02

Mobile 2.2 (<0.1) 18.3 (<0.1) 16.1 (<0.1) <0.01

Tablet 7.4 (<0.1) 8.6 (<0.1) 1.2 (<0.1) <0.01

13
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 2019 2019-2013 P-value

PC 90.3 (<0.1) 73.1 (0.1) -17.2 (0.1) <0.01

Windows 85.2 (<0.1) 75.3 (<0.1) -9.9 (0.1) <0.01

Mac OS X 14.3 (<0.1) 21.9 (<0.1) 7.7 (0.1) <0.01

Chrome OS 0.2 (<0.1) 2.3 (<0.1) 2.1 (<0.1) <0.01

Linux 0.2 (<0.1) 0.3 (<0.1) 0.1 (<0.1) <0.01

Other PC 0.2 (<0.1) 0.2 (<0.1) >-0.1 (<0.1) 0.02

Mobile 2.2 (<0.1) 18.3 (<0.1) 16.1 (<0.1) <0.01

iOS 48.8 (0.4) 54.3 (0.1) 5.5 (0.4) <0.01

Android 49.5 (0.4) 45.7 (0.1) -3.8 (0.4) <0.01

Other Mobile 1.7 (0.1) <0.1 (<0.1) -1.7 (0.1) <0.01

Tablet 7.4 (<0.1) 8.6 (<0.1) 1.2 (<0.1) <0.01

14
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 2019 2019-2013 P-value

PC 90.3 (<0.1) 73.1 (0.1) -17.2 (0.1) <0.01

Windows 85.2 (<0.1) 75.3 (<0.1) -9.9 (0.1) <0.01

Mac OS X 14.3 (<0.1) 21.9 (<0.1) 7.7 (0.1) <0.01

Chrome OS 0.2 (<0.1) 2.3 (<0.1) 2.1 (<0.1) <0.01

Linux 0.2 (<0.1) 0.3 (<0.1) 0.1 (<0.1) <0.01

Other PC 0.2 (<0.1) 0.2 (<0.1) >-0.1 (<0.1) 0.02

Mobile 2.2 (<0.1) 18.3 (<0.1) 16.1 (<0.1) <0.01

iOS 48.8 (0.4) 54.3 (0.1) 5.5 (0.4) <0.01

Android 49.5 (0.4) 45.7 (0.1) -3.8 (0.4) <0.01

Other Mobile 1.7 (0.1) <0.1 (<0.1) -1.7 (0.1) <0.01

Tablet 7.4 (<0.1) 8.6 (<0.1) 1.2 (<0.1) <0.01

iOS 86.5 (0.1) 72.6 (0.2) -13.9 (0.2) <0.01

Android 12.5 (0.1) 19.0 (0.1) 6.6 (0.2) <0.01

Windows 0.8 (<0.1) 8.4 (0.1) 7.6 (0.1) <0.01

Other Tablet 0.3 (<0.1) <0.1 (<0.1) -0.2 (<0.1) <0.01

15
Device Type and Operating System for all 

Internet Cases by Year, Using Final Login

Categories 2013 2019 2021

PC 90.3 (<0.1) 73.1 (0.1) 70.4 (0.1)

Windows 85.2 (<0.1) 75.3 (<0.1) 70.1 (0.1)

Mac OS X 14.3 (<0.1) 21.9 (<0.1) 26.3 (<0.1)

Chrome OS 0.2 (<0.1) 2.3 (<0.1) 3.0 (<0.1)

Linux 0.2 (<0.1) 0.3 (<0.1) 0.3 (<0.1)

Other PC 0.2 (<0.1) 0.2 (<0.1) 0.2 (<0.1)

Mobile 2.2 (<0.1) 18.3 (<0.1) 25.9 (<0.1)

iOS 48.8 (0.4) 54.3 (0.1) 59.6 (0.1)

Android 49.5 (0.4) 45.7 (0.1) 40.4 (0.1)

Other Mobile 1.7 (0.1) <0.1 (<0.1) <0.1 (<0.1)

Tablet 7.4 (<0.1) 8.6 (<0.1) 3.6 (<0.1)

iOS 86.5 (0.1) 72.6 (0.2) 60.0 (0.3)

Android 12.5 (0.1) 19.0 (0.1) 36.6 (0.3)

Windows 0.8 (<0.1) 8.4 (0.1) 3.5 (0.1)

Other Tablet 0.3 (<0.1) <0.1 (<0.1) <0.1 (<0.1)

16

Operating system trends 

found in 2019 appear to 

mostly continue into 2021
17

Final Outcome Code by Device Type

• Over 95% completion rate for all devices at panel closeout

• Final outcome type is related to device type

Outcome PC Mobile Tablet Chi-square P-value

Complete or Sufficient Partial 97.3 (<0.1) 95.9 (0.1) 97.4 (<0.1) 251.5 <0.01

Vacant 1.2 (<0.1) 1.9 (<0.1) 1.3 (<0.1)

Blank 1.5 (<0.1) 2.2 (<0.1) 1.4 (<0.1)
Research Goal 2. To understand how internet 

instrument usage patterns are associated with 

device type 

18
Number of Logins by Device Type

Categories 2013 2014 2015 2016 2017 2018 2019 2019-2013 p-value

1 login 78.2 (<0.1) 78.1 (0.1) 78.1 (<0.1) 79.8 (0.1) 80.0 (0.1) 79.9 (0.1) 78.1 (0.1) -0.1 (0.1) 0.08

2 logins 15.3 (<0.1) 15.3 (<0.1) 15.4 (<0.1) 14.6 (<0.1) 14.5 (<0.1) 14.5 (<0.1) 15.5 (<0.1) 0.3 (0.1) <0.01

3 logins 4.1 (<0.1) 4.2 (<0.1) 4.2 (<0.1) 3.8 (<0.1) 3.7 (<0.1) 3.7 (<0.1) 4.2 (<0.1) <0.1 (<0.1) 0.15

4+ logins 2.3 (<0.1) 2.4 (<0.1) 2.3 (<0.1) 1.9 (<0.1) 1.9 (<0.1) 1.9 (<0.1) 2.2 (<0.1) -0.1 (<0.1) <0.01

19

• Changes statistically significant, but no clear directional shift

• 78 - 80 percent had 1 login overall

• 14 - 16 percent had 2 logins overall

• 5 - 7 percent had 3 or more logins overall
Number of Logins by Device Type

Categories 2013 2014 2015 2016 2017 2018 2019 2019-2013 p-value

PC

1 login 78.4 (<0.1) 78.3 (0.1) 78.0 (<0.1) 79.3 (<0.1) 79.4 (<0.1) 79.4 (<0.1) 77.1 (<0.1) -1.2 (0.1) <0.01

2 logins 15.3 (<0.1) 15.3 (<0.1) 15.5 (<0.1) 14.9 (<0.1) 14.9 (<0.1) 14.8 (<0.1) 16.0 (<0.1) 0.8 (0.1) <0.01

3 logins 4.1 (<0.1) 4.1 (<0.1) 4.2 (<0.1) 3.9 (<0.1) 3.8 (<0.1) 3.8 (<0.1) 4.4 (<0.1) 0.3 (<0.1) <0.01

4+ logins 2.3 (<0.1) 2.3 (<0.1) 2.3 (<0.1) 1.9 (<0.1) 1.9 (<0.1) 2.0 (<0.1) 2.4 (<0.1) 0.1 (<0.1) <0.01

Mobile

1 login 73.5 (0.3) 76.4 (0.2) 80.2 (0.2) 83.6 (0.1) 83.3 (0.1) 82.5 (0.1) 81.6 (0.1) 8.1 (0.3) <0.01

2 logins 16.2 (0.2) 14.2 (0.2) 13.4 (0.1) 11.9 (0.1) 12.1 (0.1) 12.9 (0.1) 13.7 (0.1) -2.5 (0.2) <0.01

3 logins 5.6 (0.2) 5.0 (0.1) 3.9 (0.1) 2.9 (0.1) 2.9 (<0.1) 3.0 (<0.1) 3.2 (<0.1) -2.3 (0.2) <0.01

4+ logins 4.8 (0.2) 4.4 (0.1) 2.5 (0.1) 1.7 (<0.1) 1.6 (<0.1) 1.6 (<0.1) 1.5 (<0.1) -3.3 (0.2) <0.01

Tablet

1 login 78.3 (0.2) 77.1 (0.1) 77.7 (0.1) 79.7 (0.1) 79.9 (0.1) 79.8 (0.1) 79.2 (0.1) 0.9 (0.2) <0.01

2 logins 15.2 (0.1) 16.0 (0.1) 15.7 (0.1) 14.6 (0.1) 14.5 (0.1) 14.5 (0.1) 15.0 (0.1) -0.2 (0.2) 0.29

3 logins 4.1 (0.1) 4.3 (0.1) 4.3 (0.1) 3.7 (0.1) 3.7 (0.1) 3.7 (<0.1) 3.9 (0.1) -0.2 (0.1) 0.03

4+ logins 2.4 (0.1) 2.5 (0.1) 2.3 (0.1) 2.0 (<0.1) 1.9 (<0.1) 2.0 (<0.1) 2.0 (<0.1) -0.4 (0.1) <0.01

20
Number of Logins by Device Type

Categories 2013 2014 2015 2016 2017 2018 2019 2019-2013 p-value

PC

1 login 78.4 (<0.1) 78.3 (0.1) 78.0 (<0.1) 79.3 (<0.1) 79.4 (<0.1) 79.4 (<0.1) 77.1 (<0.1) -1.2 (0.1) <0.01

2 logins 15.3 (<0.1) 15.3 (<0.1) 15.5 (<0.1) 14.9 (<0.1) 14.9 (<0.1) 14.8 (<0.1) 16.0 (<0.1) 0.8 (0.1) <0.01

3 logins 4.1 (<0.1) 4.1 (<0.1) 4.2 (<0.1) 3.9 (<0.1) 3.8 (<0.1) 3.8 (<0.1) 4.4 (<0.1) 0.3 (<0.1) <0.01

4+ logins 2.3 (<0.1) 2.3 (<0.1) 2.3 (<0.1) 1.9 (<0.1) 1.9 (<0.1) 2.0 (<0.1) 2.4 (<0.1) 0.1 (<0.1) <0.01

Mobile

1 login 73.5 (0.3) 76.4 (0.2) 80.2 (0.2) 83.6 (0.1) 83.3 (0.1) 82.5 (0.1) 81.6 (0.1) 8.1 (0.3) <0.01

2 logins 16.2 (0.2) 14.2 (0.2) 13.4 (0.1) 11.9 (0.1) 12.1 (0.1) 12.9 (0.1) 13.7 (0.1) -2.5 (0.2) <0.01

3 logins 5.6 (0.2) 5.0 (0.1) 3.9 (0.1) 2.9 (0.1) 2.9 (<0.1) 3.0 (<0.1) 3.2 (<0.1) -2.3 (0.2) <0.01

4+ logins 4.8 (0.2) 4.4 (0.1) 2.5 (0.1) 1.7 (<0.1) 1.6 (<0.1) 1.6 (<0.1) 1.5 (<0.1) -3.3 (0.2) <0.01

Tablet

1 login 78.3 (0.2) 77.1 (0.1) 77.7 (0.1) 79.7 (0.1) 79.9 (0.1) 79.8 (0.1) 79.2 (0.1) 0.9 (0.2) <0.01

2 logins 15.2 (0.1) 16.0 (0.1) 15.7 (0.1) 14.6 (0.1) 14.5 (0.1) 14.5 (0.1) 15.0 (0.1) -0.2 (0.2) 0.29

3 logins 4.1 (0.1) 4.3 (0.1) 4.3 (0.1) 3.7 (0.1) 3.7 (0.1) 3.7 (<0.1) 3.9 (0.1) -0.2 (0.1) 0.03

4+ logins 2.4 (0.1) 2.5 (0.1) 2.3 (0.1) 2.0 (<0.1) 1.9 (<0.1) 2.0 (<0.1) 2.0 (<0.1) -0.4 (0.1) <0.01

21
Device Use Patterns for Users with Multiple 

Logins

• Vast majority of multiple login cases do not switch devices

• Mobile phone most likely to change device types

• Both mobile and tablet users most likely to switch to a PC+

• Very small number PC users change devices 

22

First Device 

Final Login 

with PC

Final Login 

with Mobile 

Final Login 

with Tablet

Device 

Changers 

Comparison p-value

Most Likely 

Changed to

PC 98.1 (<0.1) 1.1 (<0.1) 0.8 (<0.1) 0.3 (<0.1) <0.01 Mobile 

Mobile 14.2 (0.2) 84.0 (0.2) 1.8 (0.1) 12.4 (0.2) <0.01 PC

Tablet 10.6 (0.2) 1.8 (0.1) 87.7 (0.3) 8.8 (0.3) <0.01 PC
Research Goal 3. A preliminary look at how demographic 

characteristics are associated with device type usage

23
24

Demographics

Odds Ratios from Logistic Regression Models

Independent Variable PC Model Mobile Model Tablet Model

Age 1.025 0.952 1.021

Educational Attainment 1.145 0.846 0.968

Household Size 0.944 1.123 0.925

Number of Logins 1.111 0.880 0.944

Sex – Female vs Male 0.729 1.338 1.338

Hispanic Origin – Hispanic vs Not Hispanic 0.977 1.042 0.937

Tenure – Renter vs Owner 0.896 1.206 0.778

Internet Access – No vs Yes 0.733 1.684 0.802

Smartphone Ownership – No vs Yes 1.447 0.672 0.739

Race – Black vs White 0.694 1.601 1.101

Race – Asian vs White 1.543 0.628 0.821 (NS)

Race – AIAN vs White 0.790 1.365 0.970

Race – NHPI vs White 0.705 1.429 1.201 (NS)

Race – SOR vs White 0.977 (NS) 1.090 0.832

Race – Multiple Races vs White 1.038 0.998 (NS) 0.873

Marital Status – Never Married vs Married 0.994 (NS) 1.034 0.811

Marital Status – Divorced vs Married 0.833 1.525 0.846

Marital Status – Separated vs Married 0.725 1.698 0.843

Marital Status – Widowed vs Married 0.729 1.814 0.816
Conclusions

• ACS Internet instrument usage continues to increase

• PC usage still surpasses the other device types, but mobile phone 

usage has increased

• Could show parity with PC usage within a decade

• Windows operating systems still outpace other operating systems for 

PCs

• Mobile phones are split about 60/40 for iOS and Android devices

25
Conclusions, cont.

• Most cases complete the survey, regardless of device type

• About 79 percent of cases login once

• Most multiple login cases do not switch devices

• Mobile phone and tablet users show some propensity to change to a PC

• Based on the regression models, mobile phone response showed 

higher odds for younger respondents and minority groups, and also 

renters and nonmarried respondents

• PC and tablet on the other hand had higher odds for older, married 

respondents, and homeowners

26
Future Directions

• Instrument research and planning should account for mobile phone 

usage

• Current research underway looking at how device usage relates to 

data quality and functionality

• Data quality: Codability, breakoffs, item nonresponse

• Functionality: Error messages, help screens, completion time

27
Thank you!

ACS Research and Evaluation

https://www.census.gov/programs-surveys/acs/library/publications-and-working-papers/research-and- evaluation.html

Devices Paper

https://www.census.gov/library/working-papers/2021/acs/2022_Mills_04.html

Gregory J. Mills

Gregory.Mills@census.gov

Lindsay Longsine

Lindsay.Longsine@census.gov

R. Chase Sawyer

Robert.C.Sawyer@census.gov

28
• Purpose: 

• Improve understanding of the value and utility of ACS data. 

• Promote information sharing among data users about key ACS data issues and 

applications.

• Includes users group website and online community 

• ACS Data Users Conference

• Membership is free and open to all interested ACS data users

acsdatacommunity.prb.org

29
30

Website

census.gov/acs

Email Alerts 

https://public.govdelivery.com/accounts/USCENS

US/signup/12426

ACS User Support 

acso.users.support@census.gov

Social Media

@uscensusbureau

#ACSdata

Census Customer Service Center

800-923-8282

Source Us

U.S. Census Bureau, [YYYY – YYYY] American 

Community Survey

[1/5]-year [estimates/statistics/data release]

The American Community Survey

Continuing the Conversation