1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

A CARI Analysis of Interviewer and 

Respondent Behavior Within the Consumer 

Expenditures (CE) Interview Survey 

Ariana Welsh1, Victoria Narine2, Erica Yu2, Brett McBride1, & Wendy Carlton1

Bureau of Labor Statistics

1Office of Prices and Living Conditions, Branch of Research and Program Development

2 Office of Survey Methods Research, Behavioral Science Research Center

FCSM 2023 
2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

 Recording uses the interviewer’s laptop microphone

 Consent obtained from respondent at the start of interview

 Pre-selected items are recorded

Not full interview

 Recording starts when interviewer navigates to screen 

containing item

Recording stops when interviewer leaves screen

Computer Audio Recorded Interviewing (CARI)
3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Traditional research focus:

Interviewers report difficult 

parts of the survey

Analysts identify unusual 

response patterns

Processing triggers frequent 

edit flags

Data show longer-thanexpected question duration 

CARI research focus:

Clarity of scripted survey 

question wording

Effect of question wording

Effect of question order 

Effect of interviewer 

deviations from protocol

CARI allows us to investigate more nuanced 

research questions:
4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

What can we do with CARI?

SOURCES AND CAUSES OF INTERVIEWER/RESPONDENT MISUNDERSTANDING CAN BE 

UNCOVERED USING CARI

&

Questionnaire Design Interviewer Behavior

Ask leading probes

Deviate from scripted wording

Incomplete set of response 

options

Unclear measurement 

concepts
5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Current Research

 Used CARI to investigate interviewer and respondent behaviors 

and their effects on data quality 

 Research guided by item-specific questions 

Health Insurance (HIKIND)  Are there instances of initial 

underreporting? 

Private Pension (PRIVPENS)  Is there evidence of respondent 

confusion surrounding the reporting of private pension payroll 

deductions? 

New Vehicles (various items)  Are interviewers able to work with 

respondents to derive accurate vehicle purchase data?
6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

• n=270 cases

• November 2022 – March 2023

Health Insurance 

(HIKIND)
7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Question Wording

What kind of health insurance or health care coverage does your household have or pay for? 

INCLUDE those that pay for only one type of service (nursing home care, vision care, or dental care). 

EXCLUDE private plans that only provide extra cash while hospitalized.

1. Private health insurance, including employer and 

healthcare exchange plans

2. Medicare

3. Medicare Supplemental Insurance (Medi-Gap)

4. Medicaid

 5. SCHIP

6. Military

7. HIS

8. Single service plan (e.g., dental, vision, longterm care)

9. No coverage of any type
8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Question Wording

What kind of health insurance or health care coverage does your household have or pay for? 

INCLUDE those that pay for only one type of service (nursing home care, vision care, or dental care). 

EXCLUDE private plans that only provide extra cash while hospitalized.

Follow-up if 2.Medicare not selected AND Age > 65:

Are you covered by Medicare?

1. Yes

2. No

1. 

2. Medicare
9 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Evidence of Underreporting

 36 cases of confirmed initial 

underreporting

Respondents who did not report 

Medicare initially later reported it in the 

follow-up question 

No. Cases Percent

Medicare 

reported at first 

question 217 80.4%

Medicare 

reported at 

second question 36 13.3%

Medicare not 

reported 17 6.3%
10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Causes of Initial Underreporting

 In the subset of cases in which Medicare was not initially 

reported, the interviewer did not read Medicare as a response 

item in 17 (47%) of the 36 cases

Largest predictor of initial underreporting

Alright, so for employers, I gotta find it. Oh, employer.

No one’s covered by Medicare?

I am.

Oh, you are.

[inaudible]...through my employer.
11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Causes of Initial Underreporting

 In 30% of all evaluated cases, the interviewer was not heard 

reading Medicare as an item in the audio recording

In the majority of these cases, Medicare was still reported by the 

respondent
12 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

• n=64 cases

• July 2022 – September 2022

New Vehicle 

Purchases
13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Question Wording

Was a trade-in allowance received? (yes/no)

How much? 

What was the amount paid for the vehicle after trade-in allowance, rebate, and 

discount? Please include any sales tax in the amount paid.

What was the amount of the cash down payment?

Was the source of credit a Home Equity Loan? (yes/no)

How much was borrowed, excluding any interest?
14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Evidence of Interviewer Assistance

 Are interviewers able to work with respondents to derive 

accurate responses to the series of questions about vehicle 

purchase? 

Interviewer Calculated Amount

Yes No

Respondent answered on own 0 4

Respondent did not answer on own 10 1

Total 10 5
15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Evidence of Interviewer Assistance

 Are interviewers able to work with respondents to derive 

accurate responses to the series of questions about vehicle 

purchase? 

Short answer: It depends...!

Interviewer Calculated Amount

Yes No

Respondent answered on own 0 4

Respondent did not answer on own 10 1

Total 10 5
16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Alright. Hold one second. I can do this. How much 

was borrowed excluding interest? Uh. I have no idea.

OK. So we know that it was at least $35,831. It was 

more than that actually, but you don’t know how much 

more. Alright. Was the $18,000...it would usually 

have been applied to the purchase price. Yes. 

Um...alright. Just give me one second here. So the 

$18,000 is actually probably part of the car right now. 

Alright.

Interviewer Assistance: An Example
17 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

• n=266 cases

• January 2023 – April 2023

Private Pensions

(PRIVPENS)
18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Question wording

Was there any money deducted from your pay for - private pension fund?

1. Yes 

2. No
19 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Effects of Interviewer Comprehension

Major Change No. Cases Percent

Yes 57 26%

No 160 74%

Total 217 100%

Request for clarification No. Cases Percent

Yes 43 28%

No 108 72%

Total (codeable cases) 151 100%

Do they take out anything for pension 

or retirement for you [name?]

Respondents expressed uncertainty 

about whether to report

-401(k) plans

-457(b) plans

-other retirement plans

Interviewer clarification No. Cases Percent

Accurate 23 53%

Inaccurate 13 30%

Uncodeable 7 16%

Total (cases with request) 43 100%

...Other than 401(k), this is not 401(k), 

this is a private pension that the, I guess 

the Sheriff's department would have for 

all its officers, or whatever...
20 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

 CARI offers insight into the interviewer-respondent interaction and 

questionnaire design

 Can illuminate potential sources of error

 E.g., CARI shows that questions aren’t always read verbatim

– Meaning of key concepts may be changed (private pension, Medicare coverage)

 CARI highlights role of interviewer in improving data quality 

 CARI shows that interviewers coach the respondents to decrease respondent 

burden and increase data quality (new vehicle purchases, private pension)

 This research demonstrates the need for interviewer to be thoroughly 

trained on the concepts 

 Showcards can help with this

Conclusions
21 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

For questions, contact

Victoria Narine

Narine.Victoria@bls.gov

Ariana Welsh

Welsh.Ariana@bls.gov

Thank you!