PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

The Effects of Recall Length and Reporting Aids on 

 Household Reporting of Health Care Events 

in the Medical Expenditure Panel Survey 

Samuel H. Zuvekas, Ph.D. 

Senior Economist 

Center for Financing, Access and Cost Trends 

Agency for Healthcare Research and Quality 

540 Gaither Road 

Rockville, MD 20850 

Phone: (301) 427-1673 

Fax: (301) 427-1276 

Email: samuel.zuvekas@ahrq.hhs.gov 

July 2009 
 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

The Effects of Recall Length and Reporting Aids on 

 Household Reporting of Health Care Events 

in the Medical Expenditure Panel Survey 

ACKNOWLEDGEMENTS/DISCLAIMERS 

The views expressed in this paper are those of the author, and no official endorsement by the Agency for Healthcare 

Research and Quality, or the Department of Health and Human Services is intended or should be inferred. The 

author is indebted to Gary Olin, retired, for guidance in preparing the Medicare claims and MEPS analytic files and 

to Bob Baskin for his assistance in deriving BRR-based statistical tests. I also wish to thank Virender Kumar, Brian 

Taiffe, Kitty Williams, Diana Wobus, and Pat Ward of Westat for the preparation files and Doris Lefkowitz 

(AHRQ) for arranging the Data Use Agreement (15816) with the Centers for Medicare and Medicaid Services. 

Approval to conduct this study was granted by the Westat Institutional Review Board (IRB, FWA 5551) on October 

11, 2005 and Chesapeake IRB on February 15, 2008. 
 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

The Effects of Recall Length and Reporting Aids on 

 Household Reporting of Health Care Events 

in the Medical Expenditure Panel Survey 

ABSTRACT 

The Medical Expenditure Panel Survey (MEPS) is a widely used nationally representative survey of the levels and 

determinants of health care use and spending by US households. The MEPS uses a number of procedures to 

enhance household recall of health care use but concerns remain about the completeness of reporting, particularly 

given recall periods that average 5 months. We used a sample of Medicare beneficiaries in the MEPS who were 

matched to their Medicare claims records for health care use to examine the effects that length of recall period and 

the use of reporting aids on recall. We found in bivariate comparisons and multivariate logistic regressions that 

concordance between household-reported use and Medicare claims records declined with length of recall period. 

Furthermore, we found that the diaries and bills, insurance statements, and other records of health care events that 

MEPS respondents are asked to keep reduce the likelihood of underreporting, the most significant recall problem in 

the MEPS. We discuss the implications of these findings for the MEPS. 
Introduction 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

The Effects of Recall Length and Reporting Aids on 

 Household Reporting of Health Care Events 

in the Medical Expenditure Panel Survey 

The Medical Expenditure Panel Survey (MEPS) is a widely used nationally representative survey of the levels and 

determinants of health care use and spending by US households. Household respondent reports of each family 

member’s use of doctors, hospitals, home health agencies, prescription drugs, and other health care services form the 

cornerstone of not only the health care use estimates, but also the health care spending estimates in the MEPS 

(Cohen 1997; Cohen et al. 1997/1997). This is because household reported data on health care use are combined 

with per unit expenditure data collected from a series of follow-back surveys of the doctors, hospitals, and 

pharmacies used by households to produce the spending estimates. However, an extensive literature suggests that 

people imperfectly recall their health care use and, on average, tend to underreport use, thus raising concerns about 

the accuracy of MEPS estimates (for a review of this literature, see Bhandari and Wagner 2006). For example, 

Zuvekas and Olin (forthcoming) found that inpatient hospital stays for Medicare beneficiaries in the MEPS were 

accurately reported but that office-based and emergency department visits were systematically underreported across 

all socioeconomic groups. 

 Early methodological studies identified length of recall as an important factor affecting the quality of respondent 

reporting of health care use. In particular, these studies found that reporting of less salient events such as officebased visits declined substantially after a few weeks time (Madow 1967, Cohen and Burt 1985). These studies were 

important in the decision to use two-week recall periods in the National Health Interview Survey (NHIS) for officebased visits and other services (NCHS, 2008). However, many other large-scale household-based health surveys 

such as the Survey on Assets and Health Dynamics among the Oldest Old (AHEAD)and Health and Retirement 

Survey, National Comorbidy Survey-Replication (NCS-R), and the National Survey on Drug Use and Health 

(NSDUH) use 12-month reporting periods (in some cases, longer) to meet analytical objectives requiring the 

assessment of health care use over an entire year, while at the same time minimizing survey costs and respondent 

burden. The MEPS faces similar tradeoffs between its analytical objectives and resource constraints imposed by its 

budget and respondents’ willingness to sit through multiple lengthy interviews. Under its current design, based in 

part on a methodological experiment that assessed the implications of adding an additional interview round and thus 

shortening recall periods in the 1977 predecessor to MEPS (Cohen and Burt 1985), recall periods average 

approximately 5 months in the MEPS. While shorter than the 12-month recall periods used in many of the largescale health surveys, this 5-month recall period on average (with substantial variation across persons and interview 

rounds) raises concerns about the accuracy of reporting of health care events. 

The MEPS seeks to minimize the effects of 5-month on average recall periods by using extensive probes and by 

asking households to use calendars and keep diaries of all their health care use between interviews and to retrieve 

medical bills, explanation of benefits forms, and other documents during the actual interviews (Cohen et al. 

1996/1997). Perhaps as a result of these efforts, a comparison of MEPS and NHIS found similar levels of reporting 

of ambulatory care services (Machlin et al. 2001). The combination of shorter recall periods and the efforts 

expended to enhance recall also probably explain why Zuvekas and Olin (forthcoming) found substantially higher 

rates in household reporting accuracy of both hospital stays and office visits for Medicare beneficiaries in MEPS 

compared to those reported by Wolinsky et al. (1997) for Medicare beneficiaries in the AHEAD survey. 

We build on the previous study by Zuvekas and Olin (forthcoming) to examine the effect that length of recall period 

and the use of reporting aids have on the accuracy of household reporting of health care use in the MEPS. Like the 

previous study, our data were drawn from a sample of Medicare fee-for-service beneficiaries in the 2001-2003 

MEPS who were matched to their Medicare claims and enrollment files obtained from the Centers for Medicare and 

Medicaid Services (CMS). We compared the number of Medicare covered office visits and emergency department 

visits reported by respondents to the number of visits recorded in the Medicare claims for each of the five rounds of 

MEPS interviews (n=6875 person-rounds or 1375 persons). We exploited the variation across persons and across 
 

 

 

 

 

 

 

 

Analytic Sample 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

rounds in the length of recall periods and the use of reporting aids to examine their impact on household respondent 

reporting of health care use in the MEPS for our sample of matched Medicare beneficiaries. 

Methods 

MEPS 

The MEPS is a nationally representative survey of health care utilization and expenditures for the U.S. noninstitutionalized civilian population, sponsored by the Agency for Healthcare Research and Quality (SB Cohen 

1997; JW Cohen et al. 1997; JW Cohen 1997). It has a rotating panel design with two overlapping cohorts. A new 

cohort is initiated each year and interviewed 5 times to collect 2 calendar years of data. We pooled data for calendar 

years 2001 through 2003, and initially subset the sample to persons covered by Medicare at any point during a year. 

The full sample of Medicare beneficiaries in MEPS for 2001-2003 included 9,015 persons, or 13,680 person-year 

observations since some of the beneficiaries were in the survey for two years. 

Under a Data Use Agreement with the CMS, beneficiaries in our full sample were matched to their Medicare 

enrollment and claims data using survey-reported Medicare health insurance claim numbers (HICNs) or social 

security numbers (SSNs). Valid HICNs or SSNs were reported for 3,788 out of the 9,015 Medicare beneficiaries in 

the 2001-2003 MEPS (the majority did not provide either a HICN or SSN). Approximately 91 percent of those 

providing valid numbers (that is, the correct number of digits) matched exactly to Medicare administrative records 

(or 38 percent of the full sample of 9,015 Medicare beneficiaries in the 2001-2003 MEPS). A logistic regression 

found that the exact matches were more likely to be the household informant (self-respondent), live in the Midwest 

or South compared to the West and East regions, reside in a non-MSA, report their race as white compared to nonwhite, and be at least 65 compared to the Medicare beneficiaries who did not match exactly or provide their HICN 

or SSN for the matching (Zuvekas and Olin, forthcoming). We used propensity-score adjusted weights derived from 

this logistic regression to account for differential matching to Medicare administrative records. Once these 

propensity-score adjusted weights were applied there were no differences in total annual Medicare expenditures 

reported in MEPS for those Medicare beneficiaries who matched and those who did not, no differences in MEPS 

reported ED visits, and a slight difference of 6 percent in MEPS reported ambulatory visits. 

We restricted the matched sample to Medicare beneficiaries in our matched sample who were in the MEPS for all 

five interview rounds covering two calendar years. Thus, we only used the two MEPS panels that began in 2001 

(Panel 6 covering 2001-2002) and in 2002 (Panel 7 covering 2002-2003). We further restricted this group to 

beneficiaries with Part A and Part B Medicare fee-for-service coverage for the entire two years of each panel. 

Finally, we dropped Medicare beneficiaries whose utilization was reported by a non-resident proxy. These 

restrictions allowed us to compare utilization for survey respondents who were asked about their health care for the 

entire two years and also had Medicare claims for covered services regardless of what was reported in MEPS. The 

final analytic sample contained 1,375 persons with 5 full rounds of MEPS and Medicare claims data, or 6,875 

person-round observations. Our results were not sensitive to this set of exclusions. We obtained qualitatively and 

quantitatively similar results when also including Medicare beneficiaries in Panel 5 (ending in 2001) and Panel 8 

(starting in 2003) and those in Panels 5 through 8 where MEPS and Medicare claims data were available for less 

than five rounds (2,649 persons or 9,406 person-round observations), but met the restrictions used in our previous 

studies with this matched sample (Zuvekas and Olin, forthcoming; Zuvekas and Olin, 2009). 

Utilization Measures 

We compared household-reported use of emergency department (ED) visits and office-based visits from MEPS with 

Medicare claims for the matched sample of Medicare beneficiaries. We created dichotomous measures for any ED 

visit during the interview round because few beneficiaries had ED visits, and most had only one visit. In contrast, 

most Medicare beneficiaries had at least one office-based visit and often more (averaging close to one per month) 

and we therefore used continuous measures to compare reporting of office-based visits. We limit our analyses to ED 

and office-based visits where Medicare was one of the payers because the Medicare standard analytic files (SAFs) 

only include final action (non-rejected) claims for which a Medicare payment was made and all disputes and 

adjustments had been made. Services provided in Veteran’s Affairs (VA) facilities, for example, are excluded from 

our comparisons. Similarly, services paid entirely out of a beneficiary’s own pocket (for example, where the 

beneficiary had not met his or her annual deductible or the services were not covered by Medicare) were omitted 

from both the MEPS and Medicare claims utilization measures. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Our comparisons of ED visits were restricted to stand-alone events, with ED visits resulting in an inpatient 

hospitalization omitted. MEPS household-reported ED visits were derived primarily from the MEPS public use files 

(PUF) for emergency room visits (MEPS PUFs HC-059E for 2001, HC-067E for 2002, and HC-077E for 2003). 

We made a few adjustments to the MEPS reported ED events because the household-reported type of event can be 

different from the corresponding record in the claim files (see Zuvekas and Olin, forthcoming for additional details). 

For example, a household may have reported an ED visit that appears in a claim as an outpatient department visit to 

the same hospital. Results are not sensitive to these adjustments for household reporting of event type. Algorithms 

from the CMS-funded Research Data Assistance Center were used to construct ED visits from claims in the 

Medicare inpatient and outpatient standard analytical files (SAFs) files (Merriman 2003). We used dates of service 

to assign ED visits recorded in the CMS claims to the interview round during which they occurred. 

MEPS household-reported office-based visits were derived from the MEPS public use files (PUF) for office-based 

provider visits (MEPS PUFs HC-059G for 2001, HC-067G for 2002, and HC-077G for 2003). These included 

visits to physicians and non-physician medical providers, offices and non-hospital clinics. Following standard 

practice, office visits to the same provider occurring on the same day were combined into one visit. The 

corresponding Medicare claims-based measure of office-based visits was created from the Medicare Carrier SAFs 

claims file (Part B physician and supplier). In parallel to the MEPS, office-visits occurring on the same day to the 

same provider were combined into a single visit. Additional details are available in Zuvekas and Olin (forthcoming). 

Because the average length of the five interview rounds varies and the length of rounds varies both person-to-person 

and between rounds for the same person, we standardized both the MEPS and Medicare-claims measures to number 

of office-based visits per 90 days. 

Main Explanatory Variables 

Our primary explanatory variables of interest are length of recall periods and the use of reporting aids by the 

respondent in each round, but we also examine the impacts of respondent type, interview round, and interview 

language on recall. These are described below with sample means provided in Table 1. 

We calculated the length of the recall period as the start of the reference period for each round to the date of the 

MEPS interview for that round for each person and round. A series of 5 mutually exclusive dichotomous indicators 

was created then from this continuous measure of recall period length as follows: 0-60 days (6% of the sample, see 

Table 1), 61-120 days (17%), 121-180 days (43%), 180-240 days (31%), and 241 or more days (3% of the sample). 

Results were not sensitive to alternative specifications for length of recall period. 

MEPS interviewers record the reporting aids respondents used during the interview to help enhance recall of health 

care use. The categories of reporting aids recorded by interviewers were: a monthly planner with health care use 

recorded (for example, dates of doctor visits), a monthly planner without health care use recorded, health events 

worksheets or diaries, bills or statements from providers, health insurance statements, checkbooks, record file, 

medicine bottles or receipts, and other. From these we created a series of four mutually exclusive indicators for 

reporting aids used during each interview: 1) respondent used a monthly planner, calendar, worksheet and diary and 

the respondent used other records in the form of bills and statements from providers, insurance statements, 

checkbook, or some other written record; 2) respondent used monthly planner, calendar, worksheet or diary only; 3) 

respondent used bills and statements from providers, insurance statements, checkbooks, or other records but did not 

use a calendar or diary of some sort; and 4) none of the above written records were used. 

Previous analyses of MEPS suggest that reporting of health care use may drop off in later rounds of the MEPS 

(Ezzati-Rice, Rhode, Baskin 2009). We tested this formally by including a series of five dichotomous indicators for 

each of the five interview rounds. We also tested whether there are differences in reporting quality for health care 

use data reported when the Medicare beneficiary is the household respondent (that is, it is self-reported) and when 

health care use data was reported by their spouse or other household member (household proxy). Finally, prior to 

2007 the MEPS household instrument was only available in English. Spanish-language interviewers were used to 

conduct interviews in Spanish for some respondents and translators were used for non-English speaking respondents 

who spoke other languages. The quality of reporting for these non-English language interviews may be lower due to 

translational difficulties or increased length of the interviews needed to accommodate translation. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Other Covariates 

We included additional socio-demographic variables from the MEPS as controls in our multivariate analyses. Age 

was categorized as under 65, 65-74, 75-84, and 85 and older. Binary indicators represent the following categories: 

female, non-white including Hispanics, married, region (North, South, Midwest, and West), and living in an MSA. 

Family income was coded as below 100, 100-199 percent, and 200 percent or more of the federal poverty line (FPL). 

Education was categorized as less than 12 years, 12 years, and more than 12 years. Binary indicators represented 

the 5 categories of perceived health: excellent, very good, good, fair or poor. A cognitive limitation indicator was 

coded “1” for persons who experienced confusion or memory loss, had problems making decisions, or required 

supervision for their own safety. An activity limitation indicator was coded “1”, respectively, if the person had 

limited ability to work in a job, do housework, or go to school “because of impairment or physical or mental health 

problem.” Private insurance and Medicaid were coded “1” if the person had private coverage or Medicaid coverage 

at the first interview of the year. Finally, we created dichotomous indicators for whether the Medicare beneficiary 

was in MEPS Panel 6 (beginning in 2001) or MEPS Panel 7 (beginning in 2002). 

We also included a series of dichotomous indicators indicating quintiles of total number of office visits in the twoyear period for each person (0-6, 7-14, 15-22, 23-34 and 35+ office visits in the Medicare claims) as controls in our 

multivariate analyses because previous research suggests that reporting quality declines with intensity of treatment 

use. 

Analytical Approach 

We compared household reporting of any ED use and number of office-based visits per 90 days with Medicare 

claims both in aggregate for our analytic matched sample and with respect to concordance at the individual level. 

For the aggregate comparisons, we report sample means and results of z-score tests for differences in means. We 

also report the ratio of the sample mean of MEPS household reported use to the sample mean of Medicare-claims 

reported use and the results of adjusted Wald tests for group differences in this ratio for our key explanatory 

variables described above. 

We determined the overall level of concordance between the MEPS household reports of any ED use in the round 

and Medicare claims at the individual level using the mean agreement rate (”1” if one or more ED visits in the round 

in both MEPS and Medicare claims or both MEPS and Medicare show no ED visits, “0” otherwise) and standard 

Kappa statistics. We also report bivariate comparisons of mean agreement rates and Kappa statistics by our key 

explanatory variables and results of adjusted Wald tests for group differences. For the continuous measure of 

number of office-based visits per 90 days, we similarly used Lin’s concordance correlation coefficient (Lin 1989, 

2000) to estimate concordance between MEPS and Medicare claims both for the full sample and by our key 

explanatory variables. 

We further estimated multivariate logistic regressions of the degree of agreement between MEPS and Medicare 

claims to assess the independent contributions of recall period, reporting aids and the other key explanatory 

variables on reporting accuracy, controlling for socio-demographic and other covariates described above. For ED 

visits, we estimated a logistic regression model with perfect agreement (”1” if one or more ED visits in the round in 

both MEPS and Medicare claims or both MEPS and Medicare show no ED visits, “0” otherwise) as the dependent 

variable. For office-based visits, we estimated a series of logistic regression models with the following dichotomous 

dependent variables describing agreement between MEPS and Medicare claims: 1) less than perfect agreement 

(perfect agreement is defined as both MEPS and Medicare reporting the same number of office-based visits in the 

round to within 5 percent, or 19 out of 20 visits); 2) less than 80 percent reporting accuracy (number of MEPS 

reported visits outside 20 percent of the number of Medicare claims reported visits); 3) MEPS underreported by 1 or 

more visits per 90 days; and 4) MEPS overreported compared with Medicare claims by 1 or more visits per 90 days. 

We observed the same pattern of results with alternative dichotomous indicators of reporting accuracy (for example, 

50 percent reporting accuracy or better) as well as using ordered logistic regressions with ordinal dependent 

variables defined by tiers of reporting accuracy, but do not report them here (they are available from the author upon 

request). 

All standard errors and confidences intervals were constructed, and statistical tests performed, using the propensityscore adjusted MEPS survey weights and the method of balanced repeated replication (BRR) to adjust for the 

complex and stratified sampling design of the MEPS. The BRR method also corrects for repeated observations of 

individuals (Williams 2000). All analyses were conducted with Stata MP 10.1. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Results 

Any ED Visit in Round 

Basic descriptive statistics of the reporting of any ED visit in a MEPS interview round compared with Medicare 

claims are presented in Table 2. Overall, 7 percent of the sample of matched Medicare beneficiaries had an ED visit 

reported in MEPS during each round, while 9 percent of the matched sample had an ED visit recorded in the 

Medicare claims corresponding to the same time period as the MEPS interview round (z=6.55, p<.001 for z-score 

test for difference in means). The overall agreement rate (equals 1 if both sources reported either no ED visit or both 

sources reported an ED visit, 0 otherwise) was very high (97 percent), reflecting the fact that both sources reported 

only a small percentage with ED visits in each round. The overall Kappa statistic for the reporting of any ED visit

was 0.75 (0.71-0.80 95% C.I.) suggesting substantial agreement between MEPS and the claims data (Landis and 

Koch 1977). 

The concordance between MEPS and Medicare claims varied with recall period, recall aids, and interview round 

(Table 2). The agreement rate decreased as the length of the recall period increased (adjusted Wald test statistic 

F[4,60]=18.74, p<.001). The point estimates for the ratio of MEPS to CMS reported means and Kappa statistics 

also suggest a similar decline in reporting as the length of recall period increases, but the differences were not jointly 

statistically significant (F[4,60]=1.26, p=.29 and F[4,60]=1.85, p=.13, respectively, for the adjusted Wald tests). 

Recall aids used during the interview were associated with higher reported mean use in MEPS relative to CMS 

claims (adjusted Wald statistic F[3,61]= 9.65 , p<.001), higher agreement rates (adjusted Wald statistic 

F[3,61]=8.83, p<.001), and higher Kappa statistics (adjusted Wald statistic F[3,61]=33.44, p<.001). Similarly, 

agreement rates were 98 percent in Round 1 compared to 96 percent in each successive round (adjusted Wald 

statistic F[4,60]=4.64, p=.0025), but the Kappa statistic for agreement showed no strong pattern with respect to 

round (adjusted Wald statistic F[4,60]=1.62, p=.18). There were no statistically significant differences in the 

concordance between MEPS and the Medicare claims by whether ED utilization was self-reported or proxy reported 

and whether the interview was conducted in English or another language. 

A logistic regression on the agreement between MEPS and Medicare claims (equals 1 if both sources reported either 

no ED visit or both sources reported an ED visit, 0 otherwise) controlling for socio-demographic covariates similarly 

suggest the effects of interview characteristics on recall (Table 3). In particular, recall periods of less than 60 days 

were associated with better recall compared to the modal recall period of 121 to 180 days (odds ratio=9.32, 95% C.I. 

2.50-34.83). Similarly, the agreement rates were higher for those using both calendars/diaries and bills, insurance 

statements, or other records (odds ratio=2.09, 95% C.I. 1.18-3.68). In contrast to the bivariate analyses (Table 2), 

no statistically significant differences in agreement rates were found by interview round in this multivariate 

regression. Our results also suggest that poorer recall is associated with higher likelihood of healthcare use as 

measured by quintiles of ambulatory use intensity and poor health status. Among the other covariates included, age 

less than 65 (odds ratio=.502, 95% C.I. 0.26-0.98) and non-white (odds ratio=.52, 95% C.I. 0.36-0.77) were 

associated with lower agreement rates. 

Office-Based Visits per 90 days 

Basic descriptive statistics of the reporting of number of office-based visits per 90 days in a MEPS interview round 

compared with Medicare claims are presented in Table 4. Overall, the mean number of office visits per 90 days 

reported in MEPS was 2.3 compared to 2.7 in the Medicare claims (z-statistic=6.61, p<.001), or a 16 percent 

difference on average. The concordance correlation coefficient, which measures concordance at the level of each 

individual (in this case, each person-round), was 0.64 (0.56-0.72 95%. C.I.). 

Like reporting of ED visits, the quality of reporting of office-based visits varied in important ways with interview 

characteristics. For shorter recall periods of 0-60 days (2.3 vs. 2.2 per 90 days, z=1.24, p=0.22) and 61-120 days 

(2.0 vs. 2.2 per 90 days, z=1.58, p=0.12) there were no statistically significant differences in the number of reported 

office-visits in the MEPS compared with the Medicare claims. Reporting appeared to decline as the length of the 

recall period increased as measured both by the ratio of mean office-based visits in MEPS to Medicare Claims 

(adjusted Wald statistic F[4,60]= 5.91, p=<.001) and by Lin’s concordance correlation coefficient (adjusted Wald 

statistic F[4,60]= 3.30, p=.016). For example, there were 31 percent fewer office visits reported in the MEPS when 

the recall period was greater than 240 days (2.0 in MEPS compared with 2.9 in the Medicare Claims, z=3.06, 

p=.003). The Lin’s concordance correlation coefficient of 0.51 (0.28-0.74 95% C.I.) for recall periods greater than 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

240 days was also numerically lower compared to that for 0-60 days, but the difference was not statistically 

significant (z=1.63, p=.10). The ratio of mean reported office-visits in MEPS was highest in the first interview 

round (0.95, 0.87-1.04 95% C.I.), but no clear pattern emerged in subsequent interview rounds although the 

differences by round were statistically significant (adjusted Wald statistic for difference across all interview rounds 

F[4,60]=10.67, p<.001). Similarly, there was no clear pattern with respect to the association of interview round with 

Lin’s concordance correlation coefficient, although the joint null hypothesis that the coefficients were all the same 

across interview rounds also was rejected (adjusted Wald statistic F[4,60]=8.92, p<.001). 

The use of recall aids such as calendars/diaries or documentation in the form of bills, statements and other records 

was associated with better recall. Concordance was highest when both calendars/diaries and other records were 

used: the Lin’s concordance correlation coefficient was 0.70 (0.60-0.80 95% C.I.) and there was only a 7 percent 

gap in mean reported use in the MEPS compared to Medicare claims (2.9 vs. 3.1, z=2.69, p=.009). In contrast, when 

no recall aids were used office-visits were underreported by 45 percent on average in the MEPS (1.0 vs. 1.8 visits 

per 90 days, z=6.39, p<.001) and the concordance correlation coefficient was only 0.40 (0.28-0.53 95% C.I.). Like 

ED visits, there were no statistically significant differences in the concordance between MEPS and the Medicare 

claims by whether office-based visits were self-reported or proxy reported (adjusted Wald statistic~F[1,63]=.60, 

p=.44) and whether the interview was conducted in English or another language (adjusted Wald 

statistic~F[1,63]=.29, p=.59). 

Table 5 reports the results of a series of four logistic regressions simultaneously assessing the effect of these 

interview characteristics on the concordance between MEPS and the Medicare Claims, while controlling for sociodemographic covariates. Moving left to right on the table these regressions are: 1) the probability that MEPS visits 

were reported with less than perfect concordance (less than 95%) compared with the CMS claims; 2) the probability 

that MEPS visits were reported with less than 80% concordance ; 3) the probability that office-visits were 

underreported by more than 1 visit per 90 days in the MEPS; and 4) the probability that office-based visits were 

over-reported by more than 1 visit per 90 days in the MEPS. Each regression captures different aspects of reporting 

in the MEPS for Medicare beneficiaries compared to the Medicare claims. For example, both underreporting and 

overreporting contribute to a lack of concordance at either the 95% (first set of columns) or 80% levels (second set 

of columns)--underreporting by one or more visits (28.7%) occurred twice as often as over-reporting by one or more 

visits (14.1%). 

Short recall periods of 60 days or less were associated with the highest concordance at either the 95% or 80% 

concordance levels. For example, the odds ratio of reporting with less than 80 percent accuracy was 0.51 (0.36-0.71 

95% C.I.) compared with the modal recall period of 121-180 days (omitted category). In contrast, recall periods of 

181-240 days and more than 240 days were both associated with lower concordance compared to the omitted recall 

period of 121-180 days (not statistically significant in the 80% concordance regression). Similarly, the odds of 

underreporting visits in MEPS by 1 or more visits per 90 days were lower for shorter recall periods. However, 

shorter recall periods were also associated with greater odds of over-reporting office-based visits in MEPS. 

The effect of recall aids on concordance shows a different pattern in the multivariate analyses (Table 5) compared 

with the bivariate results reported in Table 4. The bivariate analyses suggest that the use of diaries, calendars and 

other records were associated with better concordance. In contrast, recall aids were associated with a lower 

probability of perfect concordance (indicated by odds ratios greater than one) controlling for other interview 

characteristics and socio-demographic characteristics. This relationship was weaker for concordance at the 80 

percent level (second set of columns). However, when looking at concordance at the 50 percent level, recall aids 

were associated with higher concordance (not shown). 

Recall aids were strongly associated with reduced odds of underreporting by one or more visits. For example, the 

use of both calendars/diaries and bills, insurance statements or other records compared with no use had an odds ratio 

of 0.39 (0.30-0.51 95% C.I.). However, the use of recall aids was also strongly associated with increased odds of 

overreporting office-based use by one or more visits. In sum, recall aids were simultaneously associated with 

reduced likelihood of underreporting but increased likelihood of over-reporting. The balance of these two offsetting 

effects differed depending on the particular measure of concordance used. 

The multivariate regressions were consistent with the bivariate analyses in showing that the first interview was 

associated with higher concordance in the reporting of office-based visits, but no clear pattern emerges for 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

subsequent rounds of interviews. There were also no statistically significant differences in reporting quality whether 

the interview was conducted in English or another language. However, self-reports were associated with a higher 

odds of underreporting by 1 one or more visits compared with household proxy reports (odds ratio=1.208, 95% C.I. 

1.004-1.454). Again consistent with the bivariate results (Table 4), there were no statistically significantly 

differences in reporting quality whether use was self or proxy reporting. Similar to ED visits, high use Medicare 

beneficiaries had increased odds of lower reporting quality and poorer health status was also associated with lower 

concordance. In general, other covariates included in the models were not strongly associated with concordance. 

Discussion 

Limitations 

We note several potential limitations with the analyses presented here. Most importantly, our findings regarding the 

effects of recall period and reporting aids on concordance must be interpreted strictly as associations and not 

causally. For example, MEPS respondents who diligently keep diaries of their health care use or have their 

insurance statements, bills and other records organized and ready for their interviews, might have been better 

reporters of their health care use even in the absence of these records when compared to MEPS respondents who 

used no records. Similarly, the longer recall periods result from difficulty in contacting and scheduling respondents 

for interviews, and these persons may also have poorer recall. Second, although we matched a large sample of 

Medicare beneficiaries in MEPS to claims data, our matched sample itself is not nationally representative of 

Medicare beneficiaries even with the propensity-score adjusted weights. Third, we examine household reporting 

for Medicare beneficiaries only and our findings may not generalize to the rest of the U.S. population. Finally, 

Medicare may be either incorrectly omitted or incorrectly identified as a source of payment for household-reported 

ED and office-based visits, potentially affecting comparisons with Medicare claims which generally contain only 

records for Medicare-covered care (Zuvekas and Olin, forthcoming). 

Implications 

Consistent with our previous study of this matched sample of Medicare beneficiaries in the MEPS (Zuvekas and 

Olin, forthcoming), we found that household respondents imperfectly recall ED and office-based visits. 

Furthermore, our findings suggest that the length of recall period is an important factor in determining the level of 

recall. Concordance between MEPS respondents’ reports of use and Medicare claims declined as the recall period 

increased. We did find that short recall periods were associated with a higher probability of overreporting officebased use, perhaps because of telescoping effects (that is, recalling events that occurred just outside of the true 

reference period) but we could not test this directly. However, underreporting was a bigger problem by a factor of 2 

to 1. As a result, we found that as the length of the recall period increased, the mean number of visits reported in the 

MEPS declined as a percentage of the number of visits found in the Medicare claims. 

Adding interview rounds to the MEPS to shorten recall periods on average might improve recall but would 

significantly increase both budgetary costs and respondent burden. Even if the budget would permit it, the increased 

respondent burden would likely further increase survey non-response and attrition, an issue of growing concern 

across all Federal statistical surveys. Moreover, as noted above, the Cohen and Burt (1985) study suggests that 

adding an additional interview round would only increase recall slightly at the margin. However, our study suggest 

that there may be some scope for improving recall in the MEPS by reducing variation in recall periods, particularly 

in reducing the occurrence of interviews that span periods of 8 months or more where recall is particularly poor. 

Our findings further suggest that the use of reporting aids such as calendars, diaries, insurance statements, bills, and 

other records may reduce underreporting of health care use and thus partially offset the effects of recall periods that 

average 5 months. Reporting aids were also associated with overreporting, but underreporting appears to be the 

larger problem on balance. Future methodological research might focus on the development of new Internet or 

other IT-based tools to allow households to more easily track their health care use between interviews and to 

integrate electronic medical records increasingly in use and available to patients directly (for example, many 

insurers allow patients to directly access all their claims via the internet). 

We also found that respondent recall was at its highest in the first MEPS interview round. This is consistent with 

other current methodological research with the MEPS (Ezzati-Rice, Rhode, and Baskin 2009). This is perhaps due 

to a conditioning effect whereby respondents learn during the first MEPS interview that reporting a health care event 

(for example, a visit to the doctor) leads to a series of additional questions, although we cannot test this hypothesis 
 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

directly with our data. Future methodological research might focus on whether reducing the detail that is collected 

about each health care event in MEPS would reduce conditioning effects, if indeed they exist at all. 

Finally, we did not find important differences in recall by whether health care use was reported by the Medicare 

beneficiaries themselves or by a household proxy. In other words, we found no evidence that suggests that using a 

single respondent to report for the entire household affects the quality of recall in MEPS. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

References 

Bhandari A., and T. Wagner. 2006. “Self-Reported Utilization of Health Care Services: Improving Measurement 

and Accuracy.” Medical Care Research and Review 63 (2):217-35. 

Cohen, J. 1997. "Design and Methods of the Medical Expenditure Panel Survey." MEPS Methodology Report 1 

[accessed June 22, 2009]. Available at Agency for Healthcare Research and Quality website: 

http://www.meps.ahrq.gov/mepsweb/data_files/publications/mr1/mr1.pdf. 

Cohen JW, Monheit AC, Beauregard KM et al. 1996/1997. The Medical Expenditure Panel Survey: A National 

Health Information Resource. Inquiry 33(4): 373-389. 

Cohen, S. 1997. "Sample Design of the 1996 Medical Expenditure Panel Survey Household Component." MEPS 

Methodology Report 2 [accessed June 22, 2009]. Available at Agency for Healthcare Research and Quality website: 

http://www.meps.ahrq.gov/mepsweb/data_files/publications/mr2/mr2.pdf. 

Cohen S.B. and V.L. Burt. 1985. “Data Collection Frequency Effect in the National Medical Care Expenditure 

Survey.” Journal of Economic and Social Measurement 13:125-151. 

Ezzati-Rice TM, Rohde F., Baskin R. “Evaluation of Respondents’ Reporting of Medical Events: Relationship to 

Response Propensities and Other Multiple Measures in the 

Medical Expenditure Panel Survey (MEPS).” Presented at the International Total Survey Error Workshop, Tallberg, 

Sweden, June 14-17, 2009. 

Landis JR and Koch GG. 1977. "The measurement of observer agreement for categorical data" Biometrics 33: 

159–174. 

Lin LI. 2000. "A Note on the Concordance Correlation Coefficient". Biometrics 56: 324–325. 

Lin LI. 1989. "A Concordance Correlation Coefficient to Evaluate Reproducibility". Biometrics 45 (1): 255–268. 

Machlin S, Valluzzi J, Chevarley F, Thorpe J. 2001. “Measuring ambulatory health care use in the United States: a 

comparison of 1996 estimates across four federal surveys.” Journal of Social and Economic Measurement 27(1-

2):57-69. 

Madow, W.G. 1967. Interview Data on Chronic Conditions Compared with Information Derived from Medical 

Records. National Center for Health Statistics: Vital and Health Statistics, Series 2. No. 23. Washington D.C.: U.S. 

Government Printing Office. 

National Center for Health Statistics. 2008. 2008 NHIS Questionnaire – Family. [Accessed July 6, 2009.] 

Available at: 

http://www.cdc.gov/nchs/nhis/quest_data_related_1997_forward.htm. 

Merriman K. 2003. “How to Identify Emergency Room Services in the Medicare Claims Data.” [Research Data 

Assistance Center web site accessed June 22, 2009.] Available at: http://www.resdac.umn.edu/Tools/TBs/TN003_EmergencyRoominClaims_508.pdf. 

Olin GL, Zuvekas SH, Kumar V, et al. 2008. “Medicare-MEPS Validation Study: A Comparison of Hospital and 

Physician Expenditures.” Agency for Healthcare Research and Quality Working Paper No. 08003, March 2008. 

Williams, R.L. 2000. A Note on Robust Variance Estimation for Cluster-Correlated Data. Biometrics 56(2): 645-

646. 

Wolinsky F.D., T.R. Miller, H. An et al. 2007. “Hospital Episodes and Physician Visits: The Concordance 

Between Self-Reports and Medicare Claims.” Medical Care 45 (4):300-307. 
 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Zuvekas SH and Olin GL. In press. “Validating Household Reports of Health Care Use in the Medical Expenditure 

Panel Survey,” Health Services Research. 

Zuvekas SH and Olin GL. 2009. “An Examination of the Accuracy of Medicare Expenditures in the Medical 

Expenditure Panel Survey,” Inquiry 46(1): 92-108. 
 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Table 1. Weighted Sample Means of Covariates, 2001-2003 pooled sample 

(n=6,875 person-rounds) 

Mean Standard Error 

Respondent Type

 Self-respondent (0,1) 0.68 0.01 

 Household member (0,1) 0.32 0.01 

Interview Language 

 English (omitted) 0.97 0.01 

 non-English interview (0,1) 0.03 0.01 

Length of Interview Round

 60 days or less (0,1) 0.06 0.004 

 61-120 days (0,1) 0.17 0.01

 121-180 days (0,1) 0.43 0.01 

 181-240 days (0,1) 0.31 0.01 

 >241 days (0,1) 0.03 0.002 

Recall Aids

 Diary + records (0,1) 0.45 0.01 

 Diary only (0,1) 0.32 0.01

 Records only (0,1) 0.07 0.01 

 None used (0,1) 0.15 0.01 

MEPS Panel 

 Panel 6 (0,1) 0.43 0.01 

 Panel 7 (0,1) 0.57 0.01 

Quintile of Ambulatory Use 

 1st

 quintile (0,1) 

 2nd quintile (0,1) 

 3rd quintile (0,1) 

 4th quintile (0,1) 

 5th quintile (0,1) 

0.22 

0.19 

0.22 

0.17 

0.20 

0.01 

0.01 

0.01 

0.01 

0.01 

Age 

 Age <65 (0,1) 0.14 0.01 

 Age 65-74 (0,1) 0.42 0.02

 Age 75-85 (0,1) 0.36 0.02 

 Age 85+ (0,1) 0.09 0.01 

Race/ethnicity 

 Non-white (0,1) 0.16 0.01 

 Non-Hispanic white (0,1) 

Sex 

 Female (0,1) 0.56 0.02 

 Male (0,1) 0.44 0.02 

Marital Status 

 Married (0,1) 0.55 0.02 

 Not married (0,1) 0.45 0.02 

11 
PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Census Region 

 Northeast (0,1) 0.20 0.02 

 Midwest (0,1) 0.25 0.02 

 South (0,1) 0.41 0.02 

 West (0,1) 0.15 0.02 

MSA Status 

 MSA (0,1) 0.72 0.02

 Non-MSA (0,1) 0.28 0.02 

Family Income 

 <100% FPL (0,1) 0.13 0.01 

 100-199% FPL (0,1) 0.28 0.01 

 >=200% FPL (0,1) 0.59 0.02 

Education 

 <12 years (0,1) 0.34 0.02 

 12 years (0,1) 0.34 0.02 

 >12 years (0,1) 0.32 0.02 

Perceived Health Status 

 Excellent (0,1) 0.18 0.01 

 Very good health (0,1) 0.24 0.01 

 Good health (0,1) 0.30 0.01 

 Fair health (0,1) 0.18 0.01 

 Poor health (0,1) 0.10 0.01 

Health Limitations

 Cognitive limitation (0,1) 0.11 0.01 

 Activity limitation (0,1) 0.27 0.01 

Additional Health Insurance 

 Private insurance (0,1) 0.55 0.02 

 Medicaid (0,1) 0.11 0.01 

Source: 2001-2003 MEPS, Center for Financing Access and Cost Trends, Agency for 

Healthcare Research and Quality. 

12 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Table 2. Concordance of Emergency Department Visits, 2001-2003 pooled sample (n=6,875 person-rounds) 

Any Emergency Department Visit in Round 

Mean Ratio of Agreement Kappa 

 MEPS Claims Means Rate Statistic 

Overall 0.07 0.09*** 0.76 0.97 0.75 

Respondent Type 

 Self-respondent 0.07 0.09*** 0.75 0.96 0.76

 Household member 0.05 0.07*** 0.79 0.97 0.72 

Interview Language 

 English 0.06 0.08*** 0.77 0.97 0.76

 non-English 0.11 0.15** 0.72 0.93 0.68 

Interview Round 

 Round 1 0.04 0.04* 0.84 0.98^^^ 0.73

 Round 2 0.08 0.11*** 0.78 0.96 0.77

 Round 3 0.07 0.09*** 0.74 0.96 0.73

 Round 4 0.08 0.11*** 0.76 0.96 0.78

 Round 5 0.06 0.08*** 0.73 0.96 0.72 

Length of Interview Round 

 0-60 days 0.02 0.02 0.98 0.99^^^ 0.84

 61-120 days 0.05 0.06** 0.85 0.98 0.79

 121-180 days 0.07 0.09*** 0.74 0.96 0.75

 180-240 days 0.08 0.11*** 0.78 0.96 0.75

 >241 days 0.07 0.13*** 0.56 0.93 0.63 

Recall Aids

 Diary + records 0.07 0.08*** 0.87### 0.98^^^ 0.84+++ 

 Diary only 0.07 0.09*** 0.79 0.96 0.73

 Records only 0.07 0.09*** 0.70 0.97 0.77

 none 0.04 0.09*** 0.48 0.94 0.53 

Sources: 2001-2003 MEPS, Center for Financing, Access and Cost Trends, Agency for Healthcare Research and Quality; 

CMS Medicare Standard Analytical Files 

* p<.10, ** p<.05, *** p<.01 for difference in means between CMS and MEPS. 

# p<.10, ## p<.05, ### p<.01 for difference in ratio of means by group characteristic 

^ p<.10,^^ p<.05, ^^^ p<.01 for difference in agreement rate by group characteristic 

+ p<.10,++ p<.05, +++ p<.01 for difference in Kappa Statistic by group characteristic 

13 
 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Table 3. Logistic Regression on MEPS-Medicare Claims ED Agreement (overall agreement rate 

=95.5%), pooled sample 2001-2003 (n=6,875 person-rounds) 

 Odds Ratio Standard Error 

Respondent Type

 Self-respondent (0,1) 0.802 0.174 

 Household member (omitted) 

Interview Language 

 English (omitted) 

 non-English interview (0,1) 1.463 0.625 

Interview Round 

 Round 1 (omitted) 

 Round 2 (0,1) 1.100 0.507

 Round 3 (0,1) 1.069 0.464 

 Round 4 (0,1) 1.315 0.574

 Round 5 (0,1) 1.501 0.650 

Length of Interview Round

 60 days or less (0,1) 9.323 6.132*** 

 61-120 days (0,1) 1.802 0.776

 121-180 days (omitted) 

 181-240 days (0,1) 0.752 0.147 

 >241 days (0,1) 0.519 0.209 

Recall Aids

 Diary + records (0,1) 2.086 0.589** 

 Diary only (0,1) 1.454 0.378

 Records only (0,1) 1.405 0.555 

 None used (omitted) 

MEPS Panel 

 Panel 6 (omitted) 

 Panel 7 (0,1) 1.040 0.243 

Quintile of Ambulatory Use 

 1st

 quintile (omitted) 

 2nd quintile (0,1) 

 3rd quintile (0,1) 

 4th quintile (0,1) 

 5th quintile (0,1) 

0.454 

0.269 

0.412 

0.278 

0.146** 

0.090*** 

0.129*** 

0.090*** 

Age 

 Age <65 (0,1) 0.502 0.167** 

 Age 65-74 (omitted) 

 Age 75-85 (0,1) 0.991 0.194 

 Age 85+ (0,1) 0.700 0.223 

Race/ethnicity 

 Non-white (0,1) 0.523 0.100*** 

 Non-Hispanic white (omitted) 

14 
 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Sex 

 Female (0,1) 

 Male (omitted) 

Marital Status 

0.996 0.197 

 Married (0,1) 

 Not married (omitted) 

Census Region 

 Northeast (omitted) 

 Midwest (0,1) 

 South (0,1) 

 West (0,1) 

MSA Status 

1.227 

0.956 

0.902 

0.824 

0.290 

0.264 

0.251 

0.311 

 MSA (0,1) 

 Non-MSA (omitted) 

Family Income 

 <100% FPL (omitted) 

 100-199% FPL (0,1) 

 >=200% FPL (0,1) 

Education 

0.959 

0.818 

1.578 

0.182 

0.175 

0.469 

 <12 years (omitted) 

 12 years (0,1) 

 >12 years (0,1) 

Perceived Health Status 

0.846 

1.323 

0.171 

0.428 

 Excellent (omitted) 

 Very good health (0,1) 

 Good health (0,1) 

 Fair health (0,1) 

 Poor health (0,1) 

Health Limitations

0.899 

0.631 

0.542 

0.423 

0.306 

0.239 

0.225 

0.186* 

 Cognitive limitation (0,1) 

 Activity limitation (0,1) 

Additional Health Insurance 

0.746 

1.320 

0.212 

0.278 

 Private insurance (0,1) 

 Medicaid (0,1) 

1.280 

0.752 

0.391 

0.199 

Sources: 2001-2003 MEPS, Center for Financing, Access and Cost Trends, Agency for Healthcare 

Research and Quality; CMS Medicare Standard Analytical Files. 

* p<.10, ** p<.05, *** p<.01 coefficient estimate. 

Notes: Standard errors of the coefficients were estimated using the BRR method and account for the 

complex and stratified design of the MEPS survey. 

15 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Table 4. Concordance of Ambulatory Use, 2001-2003 pooled sample (n=6,875 person-rounds) 

Number of Office Visits per 90 Days 

Mean Ratio of Concordance 

MEPS Claims Means Correlation Coefficient 

Overall 2.3 2.7*** 0.84 0.64 

Respondent Type 

 Self-respondent 2.4 2.8*** 0.85 0.66

 Household member 2.2 2.6*** 0.82 0.59 

Interview Language 

 English 2.3 2.7*** 0.85 0.65 

 non-English 2.9 3.9* 0.75 0.53 

Interview Round 

 Round 1 2.1 2.2*** 0.95 ### 0.62+++ 

 Round 2 2.1 2.7*** 0.78 0.64

 Round 3 2.3 2.7*** 0.85 0.64

 Round 4 2.6 2.9*** 0.88 0.70

 Round 5 2.5 3.2*** 0.77 0.58 

Length of Interview Round 

 0-60 days 2.3 2.2 1.08 ### 0.74++ 

 61-120 days 2.0 2.2 0.91 0.57

 121-180 days 2.3 2.8*** 0.81 0.65 

 180-240 days 2.5 2.9*** 0.84 0.65 

 >241 days 2.0 2.9*** 0.69 0.51 

Recall Aids

 Diary + records 2.9 3.1*** 0.93 ### 0.70+++ 

 Diary only 2.2 2.7*** 0.82 0.58 

 Records only 1.7 2.5*** 0.69 0.59

 none 1.0 1.8*** 0.55 0.40 

Sources: 2001-2003 MEPS, Center for Financing, Access and Cost Trends, Agency for Healthcare Research and Quality; 

CMS Medicare Standard Analytical Files 

* p<.10, ** p<.05, *** p<.01 for difference in means between CMS and MEPS. 

# p<.10, ## p<.05, ### p<.01 for difference in ratio by group characteristic. 

^ p<.10,^^ p<.05, ^^^ p<.01 for difference in agreement mean by group characteristic. 

+ p<.10,++ p<.05, +++ p<.01 for difference in Lin's concordance correlation coefficient by group characteristic. 

16 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

Table 5. Logistic Regressions on Reporting Accuracy of Number of Ambulatory Visits per 90 Days, pooled sample (n=6,875 person-rounds) 

Odds of Reporting with Odds of Reporting with Odds of Underreporting by Odds of Overreporting by

<95% Accuracy <80% Accuracy 1 or more visits 1 or more visits 

(66.6%) (58.5%) (28.7%) (14.1%) 

Odds Odds Odds Odds 

Ratio Standard Error Ratio Standard Error Ratio Standard Error Ratio Standard Error 

Respondent Type 

 Self-respondent (0,1) 

 Household member (omitted) 

Interview Language

 English (omitted) 

 non-English interview (0,1) 

Interview Round 

1.149 

1.284 

0.100 

0.229 

1.050 

0.865 

0.088

0.241 

 1.208 

0.832 

0.112** 

0.219 

0.903 

1.022 

0.102

0.389 

 Round 1 (omitted) 

 Round 2 (0,1) 

 Round 3 (0,1) 

 Round 4 (0,1) 

 Round 5 (0,1) 

Length of Interview Round

 60 days or less (0,1) 

 61-120 days (0,1) 

 121-180 days (omitted) 

 181-240 days 

 >241 days 

Recall Aids 

1.356 

1.741 

1.330 

1.382 

0.393 

0.675 

1.277 

1.429 

0.170** 

0.243*** 

0.201* 

0.226* 

0.071*** 

0.124** 

0.098*** 

0.282* 

1.176 

1.360 

1.108 

1.354 

0.508 

0.818 

1.025 

1.162 

0.143 

0.177** 

0.149 

0.181** 

0.086*** 

0.132 

0.073 

0.220 

0.998 

1.082 

0.957 

1.593 

0.520 

0.618 

0.787 

1.431 

0.180 

0.200 

0.162 

0.311** 

0.130** 

0.111*** 

0.068*** 

0.294* 

0.564 

0.750 

0.757 

0.705 

1.458 

1.404 

0.896 

0.633 

0.095 ***

0.113 * 

0.134

0.112 ** 

0.287 * 

0.205 **

0.078

0.274 

 Diary + records (0,1) 

 Diary only (0,1) 

 Records only (0,1) 

 None used (omitted) 

MEPS Panel 

1.210 

1.288 

1.355 

0.122* 

0.154** 

0.159** 

0.980 

1.139 

1.218 

0.090 

0.131 

0.142* 

0.388 

0.537 

0.681 

0.053*** 

0.069*** 

0.121** 

3.219 

2.425 

1.461 

0.581 ***

0.411 ***

0.382

 Panel 6 (omitted) 

 Panel 7 (0,1) 

Quintile of Ambulatory Use 

 1st quintile (omitted) 

 2nd quintile (0,1) 

 3rd quintile (0,1) 

0.990 

2.669 

4.582 

0.078 

0.301*** 

0.549*** 

0.944 

2.446 

3.199 

0.077 

0.264*** 

0.371*** 

0.941 

7.134 

15.718 

0.085 

1.495*** 

2.997*** 

0.998 

0.924 

1.044 

0.124 

0.168

0.192 

17 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

 4th quintile (0,1) 

 5th quintile (0,1) 

6.351 

8.961 

0.839*** 

1.406*** 

3.614 

3.147 

0.461*** 

0.408*** 

34.643 

47.439 

7.329*** 

9.612*** 

0.772 

1.026 

0.171

0.227 

Age 

 Age <65 (0,1) 1.088 0.151 1.060 0.116 0.711 0.094** 1.833 0.285 ***

 Age 65-74 (omitted) 

 Age 75-85 (0,1) 0.949 0.086 0.916 0.082 1.073 0.098 1.043 0.117

 Age 85+ (0,1) 1.031 0.177 1.048 0.184 1.000 0.174 1.044 0.251 

Race/ethnicity 

 Non-white (0,1) 1.072 0.127 1.161 0.133 1.145 0.143 0.951 0.179

 Non-Hispanic white (omitted) 

Sex 

 Female (0,1) 1.077 0.080 1.023 0.070 0.878 0.069 1.325 0.106 ***

 Male (omitted) 

Marital Status 

 Married (0,1) 0.898 0.078 0.910 0.072 0.929 0.084 0.958 0.113

 Not married (omitted) 

Census Region 

 Northeast (omitted) 

 Midwest (0,1) 1.109 0.141 1.062 0.132 1.119 0.115 1.129 0.191

 South (0,1) 1.008 0.118 0.978 0.119 1.004 0.114 1.003 0.132

 West (0,1) 0.943 0.151 0.959 0.160 0.972 0.170 1.024 0.260 

MSA Status

 MSA (0,1) 1.010 0.083 1.035 0.076 1.179 0.112* 0.937 0.134

 Non-MSA (omitted) 

Family Income 

 <100% FPL (omitted) 

 100-199% FPL (0,1) 0.947 0.103 0.946 0.092 1.128 0.113 0.748 0.098 **

 >=200% FPL (0,1) 0.938 0.107 0.874 0.089 0.874 0.083 0.977 0.157 

Education

 <12 years (omitted) 

 12 years (0,1) 1.051 0.140 0.946 0.122 0.948 0.103 1.185 0.163

 >12 years (0,1) 1.050 0.143 0.967 0.122 0.933 0.136 1.342 0.166 ** 

Perceived Health Status

 Excellent (omitted) 

 Very good health (0,1) 1.101 0.128 1.133 0.125 0.852 0.102 1.667 0.308 ***

 Good health (0,1) 1.253 0.142* 1.212 0.121* 0.847 0.094 2.020 0.381 *** 

18 
 

 

 

 

 

 

 

 

PLEASE DO NOT CITE, QUOTE OR DISRIBUTE WITHOUT PERMISSION 

 Fair health (0,1) 1.261 0.167* 1.078 0.128 0.861 0.116 1.775 0.402 **

 Poor health (0,1) 1.349 0.309 1.238 0.243 0.972 0.145 2.145 0.530 *** 

Health Limitations

 Cognitive limitation (0,1) 0.936 0.110 1.017 0.089 0.788 0.089** 1.071 0.177

 Activity limitation (0,1) 0.986 0.098 0.940 0.086 0.919 0.095 1.085 0.172 

Additional Health Insurance 

 Private insurance (0,1) 1.008 0.107 0.986 0.093 0.972 0.088 0.807 0.123

 Medicaid (0,1) 1.290 0.213 1.294 0.191* 1.303 0.161** 1.189 0.275 

Sources: 2001-2003 MEPS, Center for Financing, Access and Cost Trends, Agency for Healthcare Research and Quality; CMS Medicare Standard 

Analytical Files. 

* p<.10, ** p<.05, *** p<.01 for difference in means between CMS and MEPS. 

Notes: Standard errors of the coefficients were estimated using the BRR method and account for the complex and stratified design of the MEPS 

survey. 

19