Putting Disclosures to the Test September 15, 2016

Lorrie Cranor

Chief Technologist, Federal Trade Commission
Putting Disclosures to the Test September 15, 2016

Edith Ramirez

Chairwoman, Federal Trade Commission
Putting Disclosures to the Test September 15, 2016

Michael S. Wogalter

Professor Emeritus of Psychology

North Carolina State University
A COGNITIVE FRAMEWORK FOR DISCLOSURE 

EFFECTIVENESS: COMMUNICATION-HUMAN 

INFORMATION PROCESSING (C-HIP) MODEL 

Michael S. Wogalter, Ph.D.

Human Factors Consultant – Fort Lauderdale, FL

Professor Emeritus of Psychology (retired)

North Carolina State University

FTC Workshop 

Putting Disclosures to the Test

September 15, 2016 
Background

• Area: Human Factors, Cognitive Ergonomics

• Discipline that is concerned with the design of things (products, 

equipment, tasks, and built environments) based on people’s 

abilities and limitations with an aim at increasing productivity, 

comfort/satisfaction and safety

• Person–Thing Interface (& Interactions)
Background

• Broad research on warnings: Communications to prevent 

injury or loss in a very general sense

• Risk/hazard/safety information

• Disclosures (informed consent forms, credit card terms, online 

acceptance of terms)

• Telling the negatives, not just the positives, and doing it 

effectively

• Factors that influence effectiveness, both negative and 

positive
Background

• Warning research:

• Consumer product warnings including labels, accompanying 

inserts/sheets, tags, product manuals

• Posted signs for environmental hazards, directions, & information

• DTC advertising of prescription drugs, OTC labels

• Symbols/icons/pictograms/pictorials

• Visual and auditory presentation, and other modalities

• Print, video, internet
Coverage

• Communication-Human Information 

Processing (C-HIP) Model 

• General cognitive processing framework

• Combines basic communication 

theory and human information 

processing stage theory 

• Utility: Organizes a lot of seemingly 

haphazard constructs & research

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
C-HIP Model

• Breaks it up into stages

• Linear process - “Bottlenecks” that could 

disrupt 

• Sequential, but there are feedback loops

• Describes what is needed for a warning 

or disclosure to work

• Helpful in investigating why a warning or 

disclosure is not working & suggests 

ways to improve it

• Will start from top and work down

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
• Entity that transmits disclosure/warning

• Determines its necessity (by law, by 

standards, by hazard/risk analysis, etc.)

• Some research on source effects (Beliefs)

• Surgeon General & government added to 

warning - greater credibility

• Social persuasion literature 

• Characteristics of source

• Expert • Likeable

• Trustworthy • Similarity

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

C-HIP Model: Source
C-HIP Model: Channel

• via Modalities

• Visual

• Auditory

• via Media

• Print & video (TV, radio, Internet), labels, 

signs, posters/placards, tags, brochures, 

manuals, inserts, billboards, voice 

warnings, etc.

• Different characteristics

• Generally, redundancy (more than one 

method) is better

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
• Third stage of communications theory 

part, most complicated

• “Recipient” member of target audience

• Demographic, person variables

• Different message for general public vs. 

sophisticated (trained, expert) group

• Future: tailor disclosures to individuals

C-HIP Model: Receiver Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

Receiver
• Delivery – Did the disclosure 

actually get to the “receiver”?

• Different methods reach 

different groups/percentages 

of persons

• Assessment method: Check 

whether sample received it

C-HIP Model: Receiver Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

Receiver
• Noticing (catching attention) in a cluttered 

(noisy) context

• Salient, conspicuous, prominent

• Assessment methods:

• Looking behavior, eye movement 

(saccades), response time (faster), postexposure (if remember it, must have seen 

it), subjective evaluation

• Features that benefit

• Large, high contrast, color 

• Location, placed in visual field, reduction of 

competing stimuli

• Symbols

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

C-HIP Model: Attention Switch
Attention Holding to Read or Examine

• Adequate time given to acquire info

• Features that benefit:

• Legible (distinguish attributes of print)

• Large

• High contrast (print to background 

brightness difference)

• Brief, low density

• Structured format

• Most relevant, priority information first

C-HIP Model: Attention Maintenance

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
• Assessment methods:

• Dwell time, eye fixations

• Legibility: 

distance/obscuration 

techniques

• Participant evaluation

• Prioritization evaluation

C-HIP Model: Attention Maintenance

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
Issue: Habituation

• Seeing same or similar thing over & 

over

• Novel visual things are more 

salient

• Problem with standardization

• Material all looks alike—not a 

good thing for Attention stage

• Need some change

C-HIP Model: Attention

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
• Purpose is to give 

appreciation/understanding of risk & 

enable informed judgment

• Assessment methods: 

• Convenience: readability formulae 

• Better: Show participants material & then 

test

• Open-ended questions & cognitive 

interview

C-HIP Model: Comprehension

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
• Features that benefit

• Simpler terms, high freq in language

• Has content reflecting intended concepts

• Message components: 

(1) Nature of Risk/Hazard

(2) Instructions

(3) Consequences

• Direct, active, organized/structured

• Sufficiently explicit/specific

• Avoid ambiguity & misinterpretation

C-HIP Model: Comprehension

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
• Considerations: 

• Target audience factors

• Skills & their levels, cognitive impairment

• Development 

• Check content for necessary content

• Usability testing: Iterative design (changes) & 

test cycles

• Symbols/pictorials/pictograms

• ANSI Z535.3 standard comprehension test

• Acceptable to use when 85% (of sample) 

correctly understand what symbol means 

with no more than 5% critical confusions

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

C-HIP Model: Comprehension
Beliefs: Knowledge structures based on 

experiences, accepted as true

• Attitudes - more emotion/affect

• Easier/quicker to process if message concurs 

with existing beliefs

• Problem: if beliefs are discrepant with 

message, e.g.,

• Perceive lower risk than it is

• Government would not allow substantial risk to 

exist

• Could lead to not looking/attending

• Need salient, persuasive message to 

overcome erroneous beliefs 

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

C-HIP Model: Attitudes & Beliefs
Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

C-HIP Model: Attitudes & Beliefs

• Perceptions of hazard-risk

• Consumers primarily consider severity and to 

lesser extent probability / likelihood

• Match message characteristics to risk level

• e.g., Use different signal words: Danger vs. 

Caution vs. Note

• Familiarity 

• If believe already familiar or adequately 

knowledgeable then less likely to read

• Perceived relevance 

• Relevant to me or is it for someone else?

• Assessment Methods: self report, participant 

evaluations using rating scales
Energy/motivation to carry out task

• Cost of compliance and noncompliance

• Effort, time, money 

• Explicitness 

• Severity of loss

• Other factors

•Social influence / modeling – doing what 

others do

•Time stress, mental workload, busyness

• Interferes at all stages

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior

C-HIP Model: Motivation
• Complying, adhering, doing or not doing 

something appropriate or safe

• Good measure of effectiveness if disclosure 

appropriately changed behavior 

• Assessment methods:

• Empirical behavioral compliance

• Did they do what message directs?

• Indirect assessment (measure related 

outcome)

• A lot of things have to come together to 

change behavior!

C-HIP Model: Behavior

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
C-HIP Model - Summary

• Try to design disclosure/warning system so 

information passes through stages

• Described linear processing of stages but 

there are feedback loops: later stages can 

affect earlier stages

• e.g., familiarity (Attitudes-Beliefs stage) affects 

earlier Attention stage

• Helps to organize diverse research

• Helps to track down reason for 

warning/disclosure not doing its job

• Enables more directed/specific fixes

Receiver

Source

Attention

Switch & Maintenance

Comprehension

Attitudes 

Beliefs

Motivation

Channel

Behavior
C-HIP Model

• Current Version 

• Separates Attention Switch & 

Maintenance

• Environmental stimuli (competing 

for attention)

• Delivery (did it actually get to 

receiver)

• Memory (with comprehension)
Contact Information

• Michael S. Wogalter, Ph.D.

• WogalterM@gmail.com

• www.safetyhumanfactors.org (pre-2013 publications)

Suggested resources:

Wogalter, M. S. (Ed.) (2006). Handbook of 

Warnings. Mahwah, NJ: LEA (Boca Raton, 

FL: CRC Press).

Wogalter, M. S., DeJoy, D. M., & Laughery, K. 

R. (Eds.) (1999). Warnings and Risk 

Communication. London: Taylor & Francis. 

Putting Disclosures to the Test September 15, 2016

Evaluation procedures and methods

Ilana Westerman

CEO and Co-founder of Create with Context, Inc.

Craig Andrews

Department of Marketing

Marquette University
Putting Disclosures to the Test September 15, 2016

Ilana Westerman

CEO and Co-founder of Create with Context, Inc.
32

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
Putting Disclosures to the Test September 15, 2016

Craig Andrews

Department of Marketing

Marquette University
Tradeoffs and 

Traps in Testing 

Disclosures 

 

Craig Andrews

Marquette University

September 15, 2016
Privacy Disclosure Example: 

Understanding and/or behavior? 
Before one begins testing... 

 Clearly identify purpose/objectives and expected outcomes 

 Is an evaluation plan in place before the testing is launched? If 

not, don’t get involved. 

 With teams/panels in evaluation – watch out! KISS principle and 

the difference between “interesting” and “important” (e.g., do you 

really need 30 different treatments or 100 coding categories?) 

 Are the IVs and DVs clearly named and tied to key information 

processing objectives? 

 Target market’s priors, motivation, ability (e.g., sample knowledge, 

literacy), opportunity to process information/disclosures? 

 Try to understand objectives and focus of different disciplines. 
Information Processing

(Outcome) Variables 

Possible Testing Methods

(1) Exposure Impressions (page views), ratings

(2) Attention Recognition, recall, eye tracking

(3) Affect Emotions evoked, sentiment 

analysis, facial/brain imaging

(4) Comprehension Message beliefs/ knowledge and 

accuracy tests

(5) Yielding/Persuasion Attitude change

(6) Decision-Making Choice scenarios

(7) Behavior Click through to action/choice re: 

privacy settings (location, 

public/friends, 3rd parties)

(8) Post-Behavior Longitudinal change

*Adapted from McGuire’s (1980) Communication-Persuasion Model and Wogalter’s (2006) C-HIP Model. See Shimp and Andrews (2013) 

Advertising, Promotion, and other aspects of Integrated Marketing Communications, 9th ed., for ad testing methods.
Tradeoffs with testing methods 

Awareness 

 Recall – cognitively based; memory issues with age progression. 

 Recognition – more contextual, better with emotional/affect ads (Bruzzone tests; FCB 

Recognition); more brand focused. 

 Eye tracking – good with controversial issues/topics (warnings); doesn’t assess 

sentiment/persuasion/cognition/understanding. Is ad received, understood? accepted? 

Comprehension 

 Beliefs, accuracy, knowledge tests – can assess understanding of disclosures/stimuli 

objectively. Open to validity issues given selection of exact beliefs/information tested. 

Decision-making and behavior 

 The ultimate tests, yet many things can affect behavior beyond the disclosure/stimuli 

tested. Field studies (package disclosures) – pricing, promotion, place, prior attitudes.... 

 Experiments – Behavior (learning) without understanding? null effects? – poor stimuli, 

exposure issues, wrong sample ... Control groups are very important to infer causality. 

Shimp and Andrews (2013) Advertising, Promotion, and other aspects of Integrated Marketing Communications, 9th ed., Ch. 17. 
Type of Study Design? 

 Type of data? Primary versus secondary? Experimental (randomized 

control; causality; does x -> y?); Quasi-experimental; Focus groups; Survey; 

Eye-Tracking; Content analyses; Meta-analyses; Reviews, ... 

 Internal versus external validity (Cook and Campbell 1979) 

 

 Cross-sectional versus longitudinal (change over time)? 

 

 Study designs (after-only, pre-post/no control, after-only/control, prepost/control, Solomon four-group) 

 

 Different types of control groups (cf. Andrews and Maronick 1995 JPPM) 
Common Designs in Testing Ads: 

The Importance of Control Groups 

1) One-shot case study:

x O any problems?

2) Pre-post with no control:

O1 x O2any problems?

3) After-only with a control group:

EG (R): x O1

CG (R): O2any problems?

4) After-only with a control group:

EG (R): O1 x O2

CG (R): O3 O4any problems?

Key: x = ad treatment, O = observation; Burns and Bush (2010), Churchill (1979), Cook and Campbell (1979)
Sampling issues 

 Consumers in your target market? 

Knowledge/ literacy issues (e.g., average U.S. adult readability scores 

between 7th-9th grade; Neuhauser 2011)? 

Senior citizens? English as second language? 

 Collecting data online: 

e.g., “Who are these people?” – address-based versus opt-in sampling; 

need for cognitive interviews; mTurk and rewards; panel data: “clickthroughs” and checks; mobile device viewing 

 Probability (simple random, cluster, stratified); Non-probability 

(convenience, quota, expert) 

 Panel company “partners” and different recruitment/ incidence levels 

 Weighting/propensity scores; size per cell/ power tables (Cohen 1969) 
Disclosure Stimuli!

 Color? large enough? type size/contrast/other distractors? 

 Same testing context as viewed normally 

 FTC Clear and Conspicuous Std. (1970; 2013) 

 If text – use readability indices 

 Pretesting with control groups 
Questions and measures 

 Screeners (target market: demographics, product usage? consent; quotas) 

 Specificity: TACT (target, action, context, time; Ajzen and Fishbein 1980) and 

product experience/ sufficient knowledge/literacy? credence claims- EOI – 

(e.g., prescription drugs) 

 Funnel (open-ended successively narrow/closed-ended); question 

order/priming? 

 When/ how long to show stimuli? Prominent? Mobile access? 

 Manipulation/confound checks? (Perdue & Summers 1986 JMR) 

 Avoid negatives, biasing, hypotheticals, lack of options 

 Multiple measures? hierarchy of effects? (Vakratsas and Ambler 1999 JM) 

 Creativity (e.g., comprehension tests: if ate four servings – greater than, equal, 

or less than rec. daily amount? WTP, auctions, choices in mock stores) 

 Behavior versus communication/comprehension (understanding) 
Analysis Issues 

Match with objectives/data/measures 

 

Read outside of discipline (e.g., mediation) 

 

Comparisons with control group(s) 

 

Significance levels, comparison adjustments 
Common Problems Experienced in the 

Testing Process 

 Poor planning: no objectives or evaluation 

 Treating exposure as awareness 

 No control groups, bad stimuli, wrong sample 

 “Sample of one, ” “We already know that” 

 Incidence/qualification rates, panel “partners,” re-bidding 

 Too many “cooks in kitchen,” 30 different test conditions, 

100 coding categories,.... 

 Conclusion based on six decades of disclosure research: “...when accounting 

for audience characteristics ... and proper delivery modes ... disclosures can ... 

be effective communication tools and remedies for consumer and public 

health policy” (Andrews, 2011, Communicating Risks & Benefits, FDA, p. 156). 
Some helpful research on testing/disclosures: 

 Study Design Issues: 

Pechmann, Cornelia and J. Craig Andrews (2010), “Methodological Issues and Challenges in Conducting Social 

Impact Evaluations” in Scaling Social Impact (Chapter 12), Paul N. Bloom and Edward Skloot, eds., New York: 

Palgrave Macmillan, pp. 217-234.

 Copy Testing Issues: 

Pechmann, Cornelia and J. Craig Andrews (2011), “Copy Test Methods to Pretest Advertisements,” in Wiley 

International Encyclopedia of Marketing, Jagdish Sheth and Naresh K. Maholtra, Editors-in-Chief, v. 4 (Advertising 

and Integrated Marketing Communication),West Sussex, UK: John Wiley & Sons, Ltd., pp. 54-62.

Andrews, J. Craig and Thomas J. Maronick (1995), “Advertising Research Issues from FTC Versus Stouffer Foods 

Corp.," Journal of Public Policy & Marketing, 14 (Fall), pp. 301-309.

 Clear and Conspicuous Standard: 

Mariea Hoy and J. Craig Andrews (2004), “Adherence of Prime-Time Television Advertising Disclosures to the 

“Clear and Conspicuous Standard”: 1990 vs. 2002,” Journal of Public Policy & Marketing, 23 (Fall), pp. 170-182.

 Warnings and Disclosures: 

Andrews, J. Craig (2011), “Warnings and Disclosures” (Chapter 15) in Communicating Risk and Benefits: An 

Evidence-Based Users Guide, Baruch Fischhoff, Noel T. Brewer, and Julie S. Downs, eds., Silver Spring, MD: U.S. 

Food & Drug Administration, pp. 149-161.
Putting Disclosures to the Test September 15, 2016

Morning break

The next session begins at 11 am