Copyright ©2012 The Nielsen Company. Confidential and proprietary.

1

Robin Gentry

Nielsen

July 12, 2016

HOW A PAPER DIARY OF RADIO 

LISTENING BECAME AN ONLINE 

MEASUREMENT INSTRUMENT
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

2

AGENDA

• Overview of the Radio Diary Service

• 2007 eDiary Implementation – Lessons Learned

• New Radio eDiary

• Overview 

• Methodology

• Results

• Next Steps

• Where are we now?
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

3

RADIO DIARY SERVICE
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

4

NIELSEN RADIO DIARY SERVICE

• Conducted in ~230 markets across the country

• Four quarterly surveys—each 12 weeks long

• Fourty-eight weekly samples across the entire year

• Landline RDD sample plus an ABS supplement (to include cell-phone only and nophone households)

• Two-stage methodology

• Landline RDD: Call to receive permission to send diaries and send radio diaries to 

household

• ABS Supplement: Mail screener to determine phone status and demographics and mail 

radio diaries to eligible households

• Household flooding approach

• Each person 12+ in household receives a diary

• Diary is kept for one week (7 days)
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

5

5

DIARY SURVEY CYCLE

Pre-Conversion 

Note Card w/ $

PostPlacement 

Letters w/ $

Diary Box w/$ 

and $10 PI 

(treatment)

Follow-up 

Letter w/$

Thank You 

Postcard

Pre-Placement 

Letters w/ $

Note: recruitment materials and incentives 

vary by demo and frame.

Placement 

Calling

Conversion Calling

Follow-up Calling 

(Before, During and After 

Diary Week)
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

6

DIARY MATERIALS

Diary Box Package

All household 

members 

(aged 12+) 

are invited to 

Participate 
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

7

2007 RADIO E-DIARY IMPLEMENTATION

(MODE CHOICE APPROACH)
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

8

Copyright ©2012 The Nielsen Company. Confidential and proprietary.

2007 E-DIARY MATERIALS

NEW : eDiary Insert in Diary Box

Current : Promised Incentive 

Insert in Diary Box

“....or if you 

prefer, you can 

submit your 

eDiary online”
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

9

Copyright ©2012 The Nielsen Company. Confidential and proprietary.

2007 E-DIARY APPLICATION
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

10

Copyright ©2012 The Nielsen Company. Confidential and proprietary.

FINDINGS FROM THE ORIGINAL E-DIARY

Lessons Learned

Respondents were offered a 

choice between completing a 

paper diary and the eDiary. 

Significant declines in response and return rates.

Methodology Results

eDiary application

Many of the respondents thought that they had

successfully submitted an eDiary when, in fact, they 

hadn’t.

Communication: reminder 

emails, calls, and letters

Registrants and non-registrations received the same 

communication. Registrants felt harassed by 

reminders.

Majority of respondents were P25-54!
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

11

THE NEW RADIO E-DIARY
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

12

MULTI-YEAR PILOT STUDY

• Past Research Tests

• Pilot 1 – Proof of Concept test (2010)

• Used previous eDiary website with a few modifications

• Households recruited by mail for web survey only (no choice) 

• Pilot 2 – Complete end-to-end test (2011)

• Complete redesign of the eDiary website using outside design company

• Survey available on PC or mobile smartphone browser (works on Apple or Android)

• Testing recruitment materials: package design, pre-alert methods, and personalization.

• Pilot 3 – Fielded (2012-2013)

• Incorporates learnings from first 2 pilots and qualitative testing (focus groups, usability)

• First pilot to include Hispanic markets and bilingual materials

• Allows for paper diary requests to maximize response

• Testing recruitment materials: pre-alert materials and personalization.

• Demonstration Test – Fielded (2014)

• Run for a full 12 week period alongside our production service to demonstrate to clients how 

the new methodology would work in combination with the traditional methodology

• Only about 5% of total completes came from eDiary
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

13

METHODOLOGY

Sample ABS sample with unmatched phone number (though pilot testing include both ABS 

sample with matched and unmatched phone numbers)

Household flooding approach (One household member registers themselves and 

everyone else in the household)

All household members (aged 13+) are invited to participate 

Recruitment

Direct mail to households with a pre-placement, initial invitation, and three reminder 

mailings

Direct mail to non-respondents in registered households

Invitations to complete an online diary with last two reminder packages providing a 

paper option

Incentives are up front ($1/pre-alert) and ($2/box) with ($10/20 promised incentives 

for diary completion)

Application Two steps: (1) registration (2) 7-day online radio diary

Mobile application optimized for iPhone and Android users

Automated emails (registration confirmation, reminders , and thank you)

Promised incentive choices during registration (Amazon or Visa Gift Cards)

Toggle for Spanish
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

14

METHODOLOGY

Recruitment Materials

Box Mailer

(initial 

Invitation)

6x9 Reminder 

#1

(Still Time)

#10 Reminder 

#2

(Last Chance)

Extended 

deadline

Reminder #3

Pre-alert
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

15

METHODOLOGY

Application

• Name, email, user name, password, security 

Q&A, address confirmation, and incentive 

selection.

• Age, gender, race/ethnicity, language, and diary 

dates selection.

• Household size and race/ethnicity of 

household.

• List of all household members (13+) with name, 

email (optional), age, and gender.

• Confirmation, terms of use, and submit

Registration

Online Diary • Home page with instructions, help link, and 

status updates of everyone registered.

•7-day online diary with tabs for each day

• Quick questions with education, employment, 

and income questions.

• Thank you page with either an Amazon code or 

Visa gift card instructions.
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

16

RESPONSE RATES AND REPRESENTATION

•Response rate of 8.6% vs. ~11% with traditional 

methodology (unmatched sample only)

•Completion rate for web diary nearly 90% (postregistration)

•Within household registration improved from the previous 

pilot (59.2% vs. 46.7%)—sent follow-up mailings

Response Rates

• Women 18-34 and Men 25-34 had very strong 

representation compared to the currency service.

• 65+ households indexed at very low levels in all 

markets.

Young Adult 

Representation

• In past tests, reported listening was much higher (~30%) 

for eDiary respondents than for the traditional survey.

• In final test, listening was only 10% higher for eDiary

• Changes likely due to redesign of diary screens 

Radio Listening
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

17

QUALITATIVE TESTING 
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

18

Copyright ©2012 The Nielsen Company. Confidential and proprietary.

USER EXPERIENCE (UX)

Our Qualitative Research Partners

Location: Nielsen office in 

Columbia, MD. The testing 

room included a laptop 

computer, video camera, and 

refreshments for 

participants.

Recruitment: Twenty-one 

participants were recruited 

for

the usability study.

Scenarios: The respondents 

were provided radio listening 

scenarios in order to 

measure the accuracy of 

filling out a listening entry.

Company: MARKETVISION & 

FOCUS LATINO

Objective: An in-home 

ethnography study was 

conducted in three markets 

where participants used 

scenarios and completed 

tasks to enter radio listening 

entries in a diary page.

Details: Usability testing held 

in respondents’ homes 

included three in-home visits 

(60 minutes)

per market. All in-home 

usability tests were 

conducted in

English, Spanish, or both.

Company: FORS MARSH 

GROUP LLC (FMG)

Objective: The goal of the 

expert reviewers was to 

assess the user experience of 

the new web-based 

application and to provide 

recommendations to 

improve it.

Details: FMG was asked to 

conduct an expert review of 

the current web-based diary 

application and to propose 

modifications that will 

increase usability of the 

application.

USABILITY STUDY (2011) EXPERT REVIEWS (2011) ETHNOGRAPHY (2012)
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

19

ORIGINAL DIARY HOMEPAGE • To include a 

homepage where 

respondents could 

see instructions, a 

tutorial, and 

progress updates for 

the rest of the 

household.
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

20

REDESIGNED HOMEPAGE • Added a 

homepage to 

include: • A help page link • Instructions • Household 

member status

20
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

21

ORIGINAL DIARY DAY PAGE • Major usability 

issues • Difficulty with 

slider

• Confusion 

about “finish 

for the day”

• Didn’t notice 

entries (Below 

the fold)

21
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

22

REDESIGNED DIARY DAY PAGE » Re-designed the 

online diary page 

to include: » Manual time entry » Easy view of the 

listening entries

» Moved “Add” and 

“Done with This 

Day”
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

23

WHERE ARE WE NOW?
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

24

CURRENT STATUS OF E-DIARY

• eDiary is still not in use for our production survey

• Why wasn’t it implemented? The testing looked promising...

• Cost of implementation was very high

• Particularly challenging to combine paper and electronic diaries together

• Cost per eDiary complete was comparable to the traditional methodology, but 

response rate was considerably lower

• Issues getting participation from the whole household

• Respondent feedback and behavior suggests that paper diary is just easier to complete

• However, the rise in Smart Phone usage may make electronic diaries more viable

• Where do we go from here?

• We are still considering eDiary as a potential methodology

• Might be feasible when it is used as a follow-up to another electronic survey

• Can recruit directly via email with a personalized link to the survey
Copyright ©2012 The Nielsen Company. Confidential and proprietary.

25