National Center for Health Statistics

Lessons Learned from the National Electronic Health 

Records Survey (NEHRS)

Kelly L. Myrick, PhD, CPH, National Center for Health Statistics, Mohsin Mahar, MPH, 

Reli Group Inc.

2023 Federal Committee on Statistical Methodology Research & Policy Conference

Innovations in Gathering Health Data: Pilots, Methods, and Lessons Session

October 26, 2023
2

Disclosure

 None of the authors have any conflicts of interest to disclose.

 The findings and conclusions in this presentation are those of the 

authors and do not necessarily represent the official position of 

the National Center for Health Statistics, Centers for Disease 

Control and Prevention, Department of Health and Human 

Services, or Reli Group, Inc. 
3

Overview  Background  Research Objective  Methods  Results  Lessons Learned  Data Availability 
4

Background
5

Background

 Information gained from surveys on physician experiences are important 

for health policy.

 Clinician and physician survey response rates have declined.1,2

 Understanding preferred response can provide valuable information for 

future survey design.2,3

1. McLeod CC, Klabunde CN, Willis GB, Stark D. Health care provider surveys in the United States, 2000-2010: a review. Eval Health Prof. 2013 Mar;36(1):106-26. 

https://doi.org/10.1177/0163278712474001

2. Cho YI, Johnson TP, Vangeest JB. Enhancing surveys of health care professionals: a meta-analysis of techniques to improve response. Eval Health Prof. 2013 

Sep;36(3):382-407. https://doi.org/10.1177/0163278713496425

3. Brtnikova M, Crane LA, Allison MA, Hurley LP, Beaty BL, Kempe A. A method for achieving high response rates in national surveys of U.S. primary care 

physicians. PLoS ONE 2018 Aug;13(8):e0202755. https://doi.org/10.1371/journal.pone.0202755
6

History of the National Electronic Health Record Survey 

(NEHRS)

 National Ambulatory Medical Care Survey Electronic Medical Records 

Supplement ran 2008—2011. 

 In 2012, the survey became independent and was renamed NEHRS.

 NEHRS is conducted by the National Center for Health Statistics and 

sponsored by the Office of the National Coordinator for Health 

Information Technology (ONC).
7

Research Objective
8

Research Objective

 To discuss the lessons we have learned about changes 

in response by mode, who responded, and how they 

responded over time. 
9

Methods
10

Methods

 Nationally representative, mixed-mode 2018, 2019 and 2021 NEHRS

 Office-based physicians are the primary target

– NEHRS allowed proxy respondents

– Office manager, knowledgeable staff member

 Mixed modes

– Self administered paper questionnaire (mail)

– Self administered online questionnaire (online)

– Interviewer administered computer assisted telephone

interview (phone)
11

Methods (continued)

 2018 (n=2,000), 2019 (n=10,302) email invitation to online* survey, 

online, mail, phone

– 8-page survey

– Email tracing

 2021 (n=10,302) email invitation to online* survey, online, mail

– 4-page survey

– Email tracing

*Emails were only sent to physicians for whom working email addresses were found.
12

Results
13

Table 1. Weighted physician characteristics - National Electronic Health Records 

Survey, 2018, 2019, and 2021

2018

Estimate SE

2019

Estimate SE

2021

Estimate SE

Physician Specialty

Primary care 47.1 1.9 47.6 2.1 45.7 1.9

Surgical 16.2 1.4 22.5 1.7 21.1 1.5

Medical 36.7 2.5 29.9 1.9 33.2 1.8

Wave of completion

1st mailing 32.8 3.4 38.4 2.0 41.1 1.9

2nd mailing 19.3 3.0 18.3 1.5 25.0 1.6

3rd mailing 10.1 1.9 9.2 1.1 11.9 1.3

Phone 20.7 3.1 7.5 1.1 0.0 0.0

Online 17.1 2.5 26.5 2.0 22.0 1.5

NOTES: SE is standard error of the estimate

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
14

Table 2. Weighted practice characteristics - National Electronic Health Records 

Survey, 2018, 2019, 2021

2018

Estimate SE

2019

Estimate SE

2021

Estimate SE

Practice setting

Private solo or group practice 75.2 3.0 77.0 1.7 74.5 1.6

Other setting 24.8 3.0 23.0 1.7 25.5 1.6

Practice size

1 physician 25.1 3.0 24.9 1.8 27.0 1.7

2-3 physicians 19.9 2.8 19.5 1.8 19.7 1.5

4-10 physicians 32.1 3.2 30.6 1.9 28.0 1.7

11-50 physicians 9.5 1.6 13.9 1.3 12.8 1.1

More than 50 physicians 13.5 2.6 11.1 1.3 12.5 1.3

NOTES: SE is standard error of the estimate

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
15

Figure 1. Unweighted percentage of response, by survey mode - National Electronic 

Health Records Survey, 2008—2015, 2017—2019, 2021

81.2

71.4

78.0

74.2

71.1

72.5

64.2

69.3

68.9

62.8

68.9

77.3

18.8

28.6

22.0

25.8

28.9

27.5

35.8

29.1

11.8

18.6

7.9

1.6

19.3

18.6

23.2

22.7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2008

2009

2010

2011

2012

2013

2014

2015

2017

2018

2019

2021

Mail Phone Online

SOURCE: NCHS, National Electronic Health Records Survey, 2008—2015, 2017—2019, 2021
16

Figure 2. Weighted percentage of response, by survey mode -

National Electronic Health Records Survey, 2018, 2019, 2021

62.2

65.9

78.0

20.7

7.5

17.1

26.5

22.0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2018

2019

2021

Mail Phone Online

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
17

Figure 3. Weighted percentage of response, by respondent -

National Electronic Health Records Survey, 2018, 2019, 2021

68.8

78.2

82.1

29.3

21.3

15.8

1.9

0.5

2.2

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2018

2019

2021

Physician Office Staff Other & Unknown

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
18

Figure 4. Weighted percentage of physician response, by mode 

- National Electronic Health Records Survey, 2018, 2019, 2021

74.1

68.0

74.3

3.4

0.7

22.5

31.3

25.7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2018

2019

2021

Mail Phone Online

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
19

Figure 5. Weighted percentage of office staff response, by mode -

National Electronic Health Records Survey, 2018, 2019, 2021

32.5

58.3

94.3

62.7

32.8

4.8

8.9

5.7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2018

2019

2021

Mail Phone Online

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
20

Figure 6. Weighted percentage of other & unknown response, by 

mode - National Electronic Health Records Survey, 2018, 2019, 2021

88.5

54.8

100.0

11.5

45.2

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2018

2019

2021

Mail Phone Online

SOURCE: NCHS, National Electronic Health Records Survey, 2018, 2019, 2021
21

Lessons Learned
22

Lessons Learned

 Overall, the most used mode was self-administered mail 

questionnaire

– Physician primary mode

 CATI or phone diminishing yield

– Physicians used this mode least

 Physician percentage of response by self-administered online

questionnaire increased from 2018 to 2019

– Led to decision to eliminate phone mode in 2021 NEHRS

 Percentage of physician respondents increased over time

– Tracing for email addresses, contact information

 Percentage of office staff respondents decreased over time
23

Data Availability
24

Data Availability: National Electronic Health Records Survey 

(NEHRS)

 2018, 2019, and 2021 NEHRS have Public Use data files

– SAS statements 

– SAS files

– Stata files 

 National Electronic Health Records Survey instruments, data sets, and 

documentation 

– https://www.cdc.gov/nchs/nehrs/questionnaires.htm
25

Research Data Centers (RDCs) have restricted data

National Center for Health Statistics 

RDCs

https://www.cdc.gov/rdc/index.htm

rdca@cdc.gov

The RDCs will accept and review 

proposals and amendments via the 

Standard Application Process 

https://www.researchdatagov.org/

Federal Statistical RDCs

https://www.cdc.gov/rdc/b2accessmo

d/acs220.htm

There are over 30 Federal Statistical 

RDCs. The RDCs partner with over 50 

research organizations including 

universities, non-profit research 

institutions, and government 

agencies.
26

Thank You 

Ambulatory Health Care Data Website

https://www.cdc.gov/nchs/ahcd.htm

Call the Ambulatory and Hospital Care Statistics Branch at 

301-458-4600

Kelly Myrick 

kmyrick@cdc.gov