GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Is it worth the effort? 

Respondent record use on the British Household Panel 

Study

Heather Laurie, ISER, University of Essex

Nick Moon, GfK NOP Social Research, UK
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Overview

• The study and the task

• The data sources

• The data

• The conclusions
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

The Study

• British Household Panel Study

• Longitudinal Survey, interviewing every adult in household

• Started in 1990

• Merged into Understanding Society in 2008

• Approx 12,000 individuals in 7,000 households
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

The records

• Specifically requested

 Payslip

—Recorded on data as used to get tax code

 NB Respondent is asked to consult it, not to show it to interviewer

• Not specifically requested but often volunteered

 Electricity bills

 Gas bills

 Rent statements

 Benefit statements

 Mortgage statements

 Building Society books
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

The Survey

• All Wave 18 interviewers still on the GfK NOP books were sent 

a CAPI questionnaire 

• About 100 questionnaires despatched

• 34 completed

 GfK NOP not worked on BHPS for 2 years

 Failure to win new contract upset many interviewers

• Results need to be treated with considerable caution
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Extent of use of payslips
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Use of other documents

• Interviewer survey suggests payslips, utility bills and benefits 

statements were used fairly evenly

• Bank statements used less often

• Reported use of payslips is broadly consistent with actual data
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Older respondents more likely to consult 

payslip

0

5

10

15

20

25

30

35

40

45

Used payslip

16-25

26-35

36-50

51-65
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

More educated respondents more likely 

to consult payslip

0

5

10

15

20

25

30

35

40

45

50

Used payslip

Degree or higher

Some post-secondary

A, O level

CSE lower

none
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Larger households, especially those with 

children, less likely to consult payslip

0

5

10

15

20

25

30

35

40

45

50

Used payslip

single, non-elderly

single, elderly

couple, dep children

couple, non-dep children

couple, no children

lone parent, dep children

lone parent, non-dep

children

2+ unrelated adults

Larger households, esp with children may have more pressure 

on time
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Not surprisingly, those rated as more cooperative by 

the interviewers were more likely to consult payslip

0

5

10

15

20

25

30

35

40

Used payslip

very good

good

fair

poor

But use of payslip may be part of what makes the interviewer 

think they were cooperative
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Members of organisations were more likely to 

consult payslip

0

5

10

15

20

25

30

35

40

Used payslip

member

not member

Membership = good citizens?
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Higher income households more likely to 

consult payslip

0

5

10

15

20

25

30

35

40

Used payslip

Bottom quartile

3rd

2nd

Top quartile 

Getting accurate data is probably more important for the lower 

income households
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Higher earning respondents more likely to 

consult payslip

0

5

10

15

20

25

30

35

40

45

50

Used payslip

Bottom quartile

3rd

2nd

Top quartile 

Any error is likely to be a greater proportion of the total 
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

What is the impact on data quality?

• Problems with financial information

 Heaping/rounding

 Particular problem of leaping on longitudinal surveys

—Going from one “heap” on one wave to a different one at the next one 

can give false picture of change

• Some rounding on gross pay but not net pay

 People more aware of net pay

• Little impact on rounding of use of payslips

• But data more erratic when payslips not used
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Distribution of net pay 

0 5.0e-04 .001 .0015 .002

0 1000 2000 3000 0 1000 2000 3000

Yes No

Density

net rate of pay per month: last payment 

Graphs by Payslip seen
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Consistency of use of payslips

Seen Wave 1 Not seen Wave 1

Seen Wave 2 70 33

Not seen Wave 2 30 67

Seen Wave 2 Not seen Wave 2

Seen Wave 2 68 25

Not seen Wave 3 32 75
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Similar figures over long term

Seen Wave 1 Not seen Wave 1

Seen Wave 17 56 33

Not seen Wave 17 43 67
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Respondents took it seriously

• 62% of interviewers said respondents always or usually had 

documents ready when they arrived

• Usually one member of the household took the lead in ensuring 

that the relevant documents for the whole household were 

available
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Interviewers agree about consistency

Was it usually the same people each year who did this, or did it 

vary a lot from year to year? 

Usually the same 91%

Varied a lot 9%
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Strong correlation between use of payslips and 

long-term survey participation
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Attrition

• No evidence that asking respondents to look at payslips causes 

increased attrition in later waves

• But this may just be because it is the more cooperative 

respondents who look at payslips

 32% said it was only the easiest to interview who had documents 

ready

 6”% said it was mainly the easiest to interview

• No significant impact on length of interview:

 38% of interviewers said it made interview quicker, 24% slower 

and 38% about the same
GfK NOP Social Research The Times They Are A‘Changin – AAPOR 2010 13th May 2010

Conclusions

• Encouraging respondents to refer to documents is likely to lead 

to the collection of better quality data at little risk to respondent 

cooperation