A Methodological Comparison of Ambulatory Health Care Data Collected in Two National Surveys Steven R. Machlin and Marc W. Zodet

Agency for Healthcare Research and Quality Working Paper No. 07001 October 2007

Suggested citation: Machlin SR, Zodet MW. A Methodological Comparison of Ambulatory Health

Care Data Collected in Two National Surveys. Agency for Healthcare Research and Quality

Working Paper No. 07001, October 2007, http://gold.ahrq.gov.

AHRQ Working Papers provide preliminary analysis of substantive, technical, and methodological

issues. The papers have not undergone external peer review. They are distributed to share

valuable experience and research. Comments are welcome and should be directed to the

authors. The views expressed are those of the authors and no official endorsement by the

Agency for Healthcare Research and Quality or the Department of Health and Human Services is

intended or should be inferred.
A Methodological Comparison of Ambulatory Health Care Data Collected in Two National 

Surveys 

ABSTRACT 

Access and expenditures for health care are critical areas of interest for health services 

researchers and policy makers. Surveys, administrative data, and other sources that contain 

health care utilization data used to analyze these issues have varying objectives and data 

collection methodologies. Consequently, it is important for analysts to understand the strengths 

and limitations of the particular data source(s) they are using, and not presume that similar 

estimates or conclusions would result from the use of an alternative data source. The purpose of 

this paper is to illustrate the types of complexities and differences that arise when comparing 

estimates of health care utilization from different sources. In particular, we compare 2004 data 

on ambulatory health care utilization (excluding dental care) collected in the Medical 

Expenditure Panel Survey (MEPS) and the National Health Interview Survey (NHIS), two 

nationally representative surveys sponsored by the U.S. Department of Health and Human 

Services. 

Steven R. Machlin, MS 

Agency for Healthcare Research and Quality 

Division of Statistical Research and Methods 

Center for Financing, Access, and Cost Trends 

540 Gaither Road 

Rockville, MD 20850 

steven.machlin@ahrq.hhs.gov 

Phone: 301-427-1480 

and 

Marc W. Zodet, MS 

Agency for Healthcare Research and Quality 

Division of Statistical Research and Methods 

Center for Financing, Access, and Cost Trends 

540 Gaither Road 

Rockville, MD 20850 

marc.zodet@ahrq.hhs.gov 

Phone: 301-427-1563 

2
 
A Methodological Comparison of Ambulatory Health Care Data Collected in Two National 

Surveys 

1. Introduction 

Access and expenditures for health care are critical areas of interest for health services 

researchers and policy makers. Surveys, administrative data, and other sources that contain 

health care utilization data used to analyze these issues have varying objectives and data 

collection methodologies. Consequently, it is important for analysts to understand the strengths 

and limitations of the particular data source(s) they are using, and not presume that similar 

estimates or conclusions would result from the use of an alternative data source. 

The purpose of this paper is to illustrate the types of complexities and differences that 

arise when comparing estimates of health care utilization from different sources. In particular, 

we compare 2004 data on ambulatory health care utilization (excluding dental care) collected in 

the Medical Expenditure Panel Survey (MEPS) and the National Health Interview Survey 

(NHIS), two nationally representative surveys sponsored by the U.S. Department of Health and 

Human Services. The NHIS is a cross-sectional household interview survey that collects annual 

data that are used to monitor the nation’s health on a broad range of health topics 

(http://www.cdc.gov/nchs/nhis.htm ). For each family in the NHIS, one sample adult and one 

sample child under 18 years of age (if any children are in the household) are randomly selected 

for the Adult and Child Core questionnaires, respectively. The MEPS, which is drawn as a 

subsample of households that participated in the prior year’s NHIS, collects detailed data on 

health care use, expenditures, sources of payment, insurance coverage, and quality of care ( 

http://www.meps.ahrq.gov/ ). The panel design of the survey includes 5 rounds of interviews 

3
 
that cumulatively cover two consecutive calendar years (consecutive panels that overlap are used 

to produce annual estimates). At each interview, one adult respondent typically provides 

information about all persons in the MEPS reporting unit (defined as a person or group of 

persons who are related by blood, marriage, adoption, foster care or other family association). 

Both NHIS and MEPS collect data through personal household interviews and cover the U.S. 

civilian noninstitutionalized population. 

2. Ambulatory Care Data Collection Methodologies 

The approach to collecting health care utilization data differs substantially between the 

two surveys. In the NHIS respondents are asked a series of questions about the frequency of 

health care visits in the past 12 months, and the following 9 response categories are provided: 

None, 1, 2-3, 4-5, 6-7, 8-9, 10-12, 13-15, and 16 or more. Ambulatory care utilization is 

captured through two separate questions; one on frequency of emergency room use in the past 12 

months and another on frequency of visits to health care professionals in a doctor’s office, clinic, 

or some other place in the past 12 months (see Appendix I for survey questions). In contrast, 

MEPS health care utilization data are collected as a precursor to the collection of associated 

expenditure data. Annual data are based on three interviews at different times in which a 

household respondent is asked to enumerate all health care visits for household members in the 

interview reference period (typically a retrospective period of 3-6 months). Respondents are 

asked to classify ambulatory visits they report into the following three categories based on 

setting of care: office-based visits, hospital outpatient department visits, and hospital emergency 

room visits. Consequently, detailed data on number of visits are available from MEPS because 

4
 
 

 

 

 

an annual count of visits for each MEPS sample person is constructed as compared to NHIS data 

that are based on categorical ranges. 

3. Comparison of Selected Estimates for 2004 

This section compares NHIS and MEPS distributions and summary statistics (average 

and total visits) on ambulatory care use in 2004. Comparisons are made separately for 

emergency room and other ambulatory visits. The NHIS sample size is 43,9801 while the MEPS 

sample size is 32,737. Estimates shown in the tables are weighted to account for the complex 

survey design and adjust for nonresponse (Botman, Cohen). 

To compare estimates of the distribution of visits, we classified MEPS annual data into 

the 9 NHIS question categories listed in the methodology section above. In addition, we 

combined office-based and hospital outpatient department visits reported in MEPS into one 

category to make MEPS data for non-emergent ambulatory care as comparable as possible to that 

collected in the NHIS. 

To produce estimates of means and totals from NHIS it is necessary to make an 

assumption about the average value for each question category. We computed estimates using 2 

different “average” values --the lower bound and midpoint of the NHIS question response range 

categories (except for 0 and 1 which constitute their own categories and the upper bound 

category of 16 or more--see Appendix II for details).2

 To facilitate comparisons we also 

1

 A small percentage of the NHIS sample was excluded from the estimates due to item nonresponse (see footnotes to 

tables). Because MEPS data are based on counts of reported medical care events, by definition there are no missing 

utilization data in the survey (i.e. no reported events means no use). 

2 We did not compute estimates using the upper bound of the NHIS categories because MEPS data indicate that 

visits are generally skewed toward the lower end of the NHIS categories (see Tables 1a and 1b). We used the same 

value for the top category (16 or more visits) for both the lower bound and midpoint estimates, but used different 

values depending on the type of visit (i.e., 16 for emergency room visits and 31.7 for other ambulatory visits). For 

emergency room visits we used the lower bound for the top category (16) because the MEPS sample had no persons 

5
 
 

 

produced adjusted MEPS estimates of means and totals by applying the lower bound and 

midpoint of the NHIS question response range categories to the MEPS data. Unadjusted “direct” 

estimates based on the actual MEPS count data are also included in summary data tables (2a and 

2b). 

3.1 Distributions of Number of Visits 

MEPS estimates of the frequency of non-emergent ambulatory visits are higher at the 

extremes of the distribution compared to NHIS (Table 1a). In particular, while the estimated 

percent of persons with no ambulatory visits is substantially higher in MEPS (27.2 percent) than 

NHIS (17.5 percent), the percent with a large number of visits was also higher (MEPS estimate 

of 12.5 percent with 13 or more visits versus only 7.2 percent for NHIS). Conversely, the 

estimated percent of persons with visits between these extremes (1-12 visits) was lower in MEPS 

(60.3 percent) than NHIS (75.4 percent). 

The MEPS distribution of number of emergency room visits is somewhat lower than the 

NHIS distribution (Table 1b). According to MEPS, 85.8 percent of the population had no 

emergency room visit during 2004 while NHIS estimates that 79.3 percent of persons had no ER 

visit in the 12 months preceding the 2004 interview. The estimated percent of persons with 2 or 

more ER visits was about twice as high in NHIS as MEPS (7.6 percent versus 3.7 percent). 

3.2 Aggregate Total Number of Visits 

MEPS estimates of the total number of non-emergent ambulatory visits are higher than 

NHIS estimates (Table 2a). More specifically, the MEPS estimate is 28 percent higher than 

with more than 16 visits. For other ambulatory visits we used the average number of visits for MEPS sample 

persons with 16 or more visits (31.7). 

6
 
NHIS for the lower bound comparison (1.61 versus 1.25 billion) and 25 percent higher for the 

midpoint comparison (1.69 versus 1.34 billion). While the estimated percent of persons with one 

or more ambulatory visits is lower in MEPS (72.9 percent) than NHIS (82.5 percent), the MEPS 

average for those with at least one visit exceeds the corresponding NHIS estimate by 2.2 visits 

per person for both the lower bound (7.5 versus 5.3) and midpoint approaches (7.9 versus 5.7). 

These higher averages are explained by a larger proportion of cases reporting 13 or more visits 

(12.5 versus 7.2 percent), which ultimately drives the larger MEPS estimates of total nonemergent ambulatory visits. 

In contrast to non-emergent ambulatory visits, NHIS estimates of the total number of 

emergency room visits are nearly twice as large as the corresponding MEPS estimates for both 

the lower bound (103 versus 55.1 million) and midpoint (114.1 versus 60 million) approaches 

(Table 2b). These differences are driven by larger NHIS estimates than MEPS of both: a) the 

proportion of the population with 1 or more emergency room visits (20.7 versus 14.2 percent), 

and b) the average number of visits per person with at least one emergency room visit (1.73 

versus 1.3 per person for lower bound estimates and 1.92 versus 1.4 per person for midpoint 

estimates). 

4. Discussion/ Summary 

Using data from the MEPS and NHIS, this paper demonstrates that the types of health 

care utilization estimates that can be made and the magnitude of those estimates can be quite 

different based on household surveys with different methodologies. Following is a summary of 

the major differences in data collection methodologies across the two surveys and the potential 

impact of these differences on survey estimates. 

7
 
Ambulatory setting categories: NHIS ambulatory care utilization data are captured through two 

separate questions; one on frequency of emergency room use and another on frequency of visits 

to health care professionals in a doctor’s office, clinic, or some other place. MEPS respondents 

are asked to classify ambulatory visits they report into three categories based on setting of care: 

hospital emergency room visits, office-based visits, and hospital outpatient department visits. 

These differences in setting categories could contribute toward the divergent survey estimates. 

For example, it is possible that a visit reported in MEPS as having occurred in a hospital 

outpatient department would have been reported by a comparable respondent in NHIS as an 

emergency room visit since there is not a separate question for hospital outpatient department 

visits. 

Question style: NHIS asks respondents to pick a range (10 categories) in which the number of 

visits for the person falls while MEPS asks respondents to enumerate all visits for the sample 

person, a process that results in a specific number of visits for each sample person. While both 

surveys can be used to estimate the percentage of the population with at least one visit during the 

year, NHIS data are less suitable for estimating a total or mean because it requires assigning a 

value to the different categorical ranges. This is most problematic for the highest category 

because it has an open ended range (16 or more visits), so it is not possible, for example, to apply 

a midpoint value. Moreover, because it is unlikely that a person would have more than 16 visits 

to an emergency room in a 12 month period but much more likely that an individual would have 

substantially more than 16 visits to other ambulatory settings, it is more difficult to assign a 

reasonable value for the top category of the question on ambulatory visits to “doctor’s offices, 

clinics, and other places.” To produce the NHIS estimates in Table 2a, we assumed that the 

8
 
 

average number of office-based and hospital outpatient department visits combined from MEPS 

is a reasonable average value for the top category. It would not have been possible to produce 

estimates of means and totals from NHIS without making assumptions about the average value 

or distribution for each category. 

Question reference period & number of interviews: The NHIS question refers to the past 12 

months while MEPS data are cumulated across 3 interviews with an average recall period of 

about 5-6 months. These differences likely have an impact on resultant estimates. For example, 

the longer NHIS recall period may make the responses more prone to overestimation biases 

associated with telescoping (Bradburn). While one would generally expect the shorter reference 

period in MEPS to facilitate more accurate reports of utilization from respondents, respondent 

fatigue across multiple interviews coupled with the increased burden of additional questions on 

expenditures and sources of payment that are associated with reported health care visits may 

produce incentives for MEPS respondents to underreport visits. The direction of differences 

between estimates from the two surveys are not completely consistent with these explanations, 

which highlights the difficulty in ascribing exact reasons for discrepancies across surveys with 

different methodologies. 

It is difficult to ascribe the extent to which the differences between MEPS and NHIS 

discussed in this paper explain varying estimates of health care utilization. For example, the 

NHIS estimates of total 2004 emergency department visits based on the methodology used in this 

paper are substantially closer than MEPS to the total estimate from the National Hospital 

Ambulatory Medical Care Survey (NHAMCS)3

, a national survey based on a sample of hospitals 

rather than households. While the MEPS aggregate population estimate of emergency 

3

 NAMCS estimate of emergency room visits in 2004 is 110.2 million. 

9
 
department use may be low due to underreporting of emergency room visits and/or 

misclassification of some visits as occurring in other settings, there may be overestimation in 

NHIS due to telescoping. Moreover, estimates from NHAMCS may be higher than MEPS due 

to methodological differences. In particular, what a household respondent in MEPS would 

consider to be an emergency room visit would not necessarily be consistent with how such visits 

are counted in the NHAMCS from the provider perspective. For example, a hospital visit that 

was initiated in the emergency room but then immediately referred to another department for 

tests may be reported as an outpatient visit in MEPS but counted as an emergency room visit in 

NHAMCS. In summary, whether working with one or multiple data sources, it is important for 

researchers to assess the strengths and limitations of the particular source(s) being used, and to 

use caution when interpreting and comparing estimates. 

10
 
References 

Botman SL, Moore TF, Moriarity CL, and Parsons VL. Design and estimation for the National 

Health Interview Survey, 1995–2004. National Center for Health Statistics. Vital Health Stat 

2(130). 2000. 

Bradburn, N. M., J. Huttenlocher, and L. V. Hedges. 1994. ‘‘Telescoping and Temporal 

Memory.’’ In Autobiographical Memory and the Validity of Retrospective Reports, edited by N. 

Schwartz, S. Sudman. New York: Springer-Verlag. 

Cohen SB, DiGaetano R, Goksel H. Estimation procedures in the 1996 Medical Expenditure 

Panel Survey Household Component. Rockville (MD):Agency for Health Care Policy and 

Research; 1999. MEPS Methodology Report No. 5. AHCPR Pub.No. 99-0027. 

Hing E, Cherry DK, Woodwell Da. National Ambulatory Medical Care Survey: 2004 summary. 

Advance data from vital and health statistics; no 374. Hyattsville, MD: National Center for 

Health Statistics, 2006. 

11
 
 

Table 1a: Distribution of Non-Emergency Room Ambulatory Visits, 2004 

Sample Sizes 

Number of Visits MEPS NHIS4 

0 10010 7665 

1 5332 7679 

2-3 6154 11806 

2 3608 --

3 2546 --

4-5 3179 6195 

4 1797 --

5 1382 --

6-7 1957 2984

6 1121 --

7 836 --

8-9 1251 1530 

8 677 --

9 574 --

10-12 1327 2237

10 528 --

11 433 --

12 366 --

13-15 926 840 

13 357 --

14 320 --

15 249 --

16+ 2601 2239 

MEPS NHIS
 

27.2 (.45) 17.5 (.26) 

15.8 (.32) 18.3 (.26) 

19.3 (.29) 27.6 (.26) 

11.1 --

8.1 --

10.3 (.22) 14.3 (.20) 

5.8 --

4.5 --

6.4 (.16) 6.8 (.13) 

3.7 --

2.7 --

4.2 (.15) 3.3 (.11) 

2.2 --

1.9 --

4.5 (.15) 5.0 (.12) 

1.7 --

1.5 --

1.3 --

3.3 (.12) 1.9 (.08) 

1.3 --

1.1 --

0.9 --

9.2 (.26) 5.3 (.13) 

Percent (SE)
 

4

 Excludes 805 observations with missing data on survey question 

12
 
 

 

Table 1b: Distribution of Emergency Room Visits, 2004 

Sample Sizes Percent (SE) 

Number of Visits MEPS NHIS5 MEPS NHIS 

0 28057 34363 85.8 (.25) 79.3 (.28) 

1 3547 5752 10.9 (.22) 13.1 (.21) 

2-3 970 2661 2.8 (.11) 5.8 (.15) 

2 768 -- 2.2 --

3 202 -- 0.6 --

4-5 114 478 0.3 (.03) 1.0 (.06) 

4 83 -- 0.2 --

5 31 -- 0.1 --

6-7 28 166 0.1 (.02) 0.4 (.03) 

6 15 -- 0.1 --

7 13 -- 0.0 --

8-9 12 67 0.0 (.01) 0.2 (.02) 

8 6 -- 0.0 --

9 6 -- 0.0 --

10-12 7 72 0.0 (.01) 0.1 (.02) 

10 2 -- 0 --

11 1 -- 0 --

12 4 -- 0.0 --

13-15 2 30 0 0.1 (.01) 

13 1 -- 0 --

14 1 -- 0 --

15 0 -- 0 --

16+ 0 49 0 0.1 (.02) 

Table 2a: Non-Emergency Room Ambulatory Visits: Comparison of Selected Summary Statistics, 2004 

Direct (SE) Lower Bound (SE) Midpoint (SE) 

Estimate MEPS NHIS MEPS NHIS MEPS NHIS 

Total Visits (millions) 1675.6 (44.2) -- 1605.6 (38.1) 1249.7 (17.7) 1687.1 (39.5) 1343.3 (18.5) 

Percent with 1+ visits 72.9 (0.5) 82.5 (0.3) -- -- -- --

Mean per person with visit 7.8 (0.1) -- 7.5 (0.1) 5.3 (5.2) 7.9 (0.1) 5.7 (5.2) 

Table 2b: Emergency Room Visits: Comparison of Selected Summary Statistics, 2004 

Direct (SE) Lower Bound (SE) Midpoint (SE) 

Estimate MEPS NHIS MEPS NHIS MEPS NHIS 

Total Visits (millions) 57.5 (1.7) -- 55.1 (1.6) 103 (2.2) 60.0 (1.8) 114.1 (2.4) 

Percent with 1+ visits 14.2 (0.3) 20.7 (0.3) -- -- -- --

Mean per person with visit 1.4 (0.0) -- 1.3 (0.0) 1.73 (2.2) 1.4 (0.0) 1.92 (2.3) 

5

 Excludes 342 observations with missing data on survey question. 

13
 
 

 

Appendix I: 2004 NHIS Survey Questions on Ambulatory Care 

DURING THE PAST 12 MONTHS, HOW MANY TIMES have [you/sample child] gone to a HOSPITAL EMERGENCY 

ROOM about [your/sample child’s] health? (This includes emergency room visits that resulted in a hospital admission.) 

00 None 

01 1 

02 2-3 

03 4-5 

04 6-7 

05 8-9 

06 10-12 

07 13-15 

08 16 or more 

97 Refused 

99 Don't know 

DURING THE PAST 12 MONTHS, HOW MANY TIMES have [you/sample child] seen a doctor or other health care 

professional about [your/sample child’s] health at a DOCTOR'S OFFICE, A CLINIC, OR SOME OTHER PLACE? Do 

not include times [you/sample child] were hospitalized overnight, visits to hospital emergency rooms, home visits, dental 

visits, or telephone calls. 

00 None 

01 1 

02 2-3 

03 4-5 

04 6-7 

05 8-9 

06 10-12 

07 13-15 

08 16 or more 

97 Refused 

99 Don't know
 

14
 
Appendix II. NHIS vs. MEPS: Values Used for Estimating Means and Totals for Comparison 1 

NHIS Category Lower Bound 

0 0 

1 1 

2-3 2 

4-5 4 

6-7 6 

8-9 8 

10-12 10 

13-15 13 

16 or more 16 (ER), 31.7 (Other Ambulatory) 

Midpoint 

0 

1 

2.5 

4.5 

6.5 

8.5 

11 

14 

16 (ER), 31.7 (Other Ambulatory) 

15