WORKING 

 PAPERS

CONSUMER PERCEPTIONS OF QUALIFIED 

HEALTH CLAIMS IN ADVERTISING

R. Dennis Murphy

WORKING PAPER NO. 277

July 2005

___________________________________________________________________________

FTC Bureau of Economics working papers are preliminary materials circulated to stimulate discussion

and critical comment. The analyses and conclusions set forth are those of the authors and do not

necessarily reflect the views of other members of the Bureau of Economics, other Commission staff, or

the Commission itself. Upon request, single copies of the paper will be provided. References in

publications to FTC Bureau of Economics working papers by FTC economists (other than

acknowledgment by a writer that he has access to such unpublished materials) should be cleared with the

author to protect the tentative character of these papers.

______________________________________________________________________________

BUREAU OF ECONOMICS

FEDERAL TRADE COMMISSION

WASHINGTON, DC 20580
Consumer Perceptions of Qualified Health Claims in Advertising

R. Dennis Murphy

Division of Consumer Protection, Bureau of Economics

Federal Trade Commission

Washington, DC 20580

 dmurphy@ftc.gov

(202) 326-3524

Abstract: This working paper presents the findings of copy test research on consumer

perceptions of food and dietary supplement print advertisements containing qualified health

claims for diet-disease relationships that lack a high level of scientific support. The research was

motivated by court decisions that struck down the Food and Drug Administration’s outright ban

on such claims in labeling. The decisions effectively have placed the burden on the government

to allow qualified claims for these diet-disease relationships unless it can demonstrate that

consumers are unable to understand qualifications characterizing the true level of certainty in the

relevant scientific evidence. My findings indicate that qualified language can have a significant

impact on consumer evaluation of scientific certainty. My research also suggests, however, that

it will be a difficult task to craft qualifications in advertising that communicate a low level of

scientific certainty. None of the tested disclaimers, whether appearing in real advertisements for

real products or in fictitious advertisements, communicated serious limitations in scientific

evidence (i.e. science that FDA would rate at a “D” level). In addition, consumers interpreted all

of the tested advertisements in widely disparate fashion. For example, although consumers

seeing an ad for a fictitious antioxidant vitamin supplement on average rated the degree of

scientific evidence correctly at a “C” level of support, approximately two-thirds of the

consumers either overestimated or underestimated the certainty of the science. 

JEL Classification: D18, K20, K23, L15

The views expressed herein are those of the author and do not represent the views of the Federal

Trade Commission or any individual Commissioner.

I thank Pauline Ippolito, Thomas Pahl, and Lee Peeler for their valuable comments. I am

particularly indebted to Christopher Kelley for his assistance in data analysis and preparation of

the graphical displays. 
1 FDA describes this standard as follows:

The significant scientific agreement standard is intended to be a strong standard

that provides a high level of confidence in the validity of a substance/disease

relationship. Significant scientific agreement means that the validity of the

relationship is not likely to be reversed by new and evolving science....Significant

scientific agreement does not require a consensus or agreement based on

unanimous and incontrovertible scientific opinion. However, on the continuum of

scientific discovery that extends from emerging evidence to consensus, it

represents an area on the continuum that lies closer to the latter than to the former.

See Guidance for Industry: Significant Scientific Agreement in the Review of Health

Claims for Conventional Foods and Dietary Supplements, U. S. Food and Drug Administration,

Center for Food Safety and Applied Nutrition, December 22, 1999.

1

I. INTRODUCTION

This working paper presents the results of copy test research concerning consumer

interpretation of qualified health claims in advertising for food and dietary supplement products. 

This research is part of a series of related copy tests, the first of which was presented in the

Generic Copy Test of Food Health Claims in Advertising, published as an FTC staff report in

November 1998. Three follow-up tests have been performed since that date. The latest and

most extensive of these was completed in November 2003. 

 The discussion that follows focuses on the last of the copy tests, which remedied certain

limitations and methodological weaknesses in the earlier studies, and addressed issues raised by

recent court decisions that modified related FDA regulations of health claims in labeling. 

The core issue in all of these projects has been whether advertising disclosures can

communicate clearly and nondeceptively limitations in the level of scientific support for dietdisease relationships that have not met the significant scientific agreement standard that FDA

requires for unqualified health claims in labeling.1

 At the time the original research was

conducted in 1998, FDA regulations implementing the Nutrition Labeling and Education Act of

1990 (NLEA) banned any health claim in labeling unless FDA determined that the claim was

supported by significant scientific agreement. The Federal Trade Commission’s Enforcement

Policy Statement on Food Advertising (“Statement”) adopted a more flexible approach toward

advertising. The Statement advised that, under well established deception law, advertisements

could discuss unapproved diet-disease relationships, provided that such ads were carefully

qualified to convey the actual level of scientific support for the relationship, and there was no
2 Federal Trade Commission, Enforcement Policy Statement on Food Advertising,

(May 1994), Washington, D.C., p. 20. 

3 Virginia State Bd of Pharmacy v Virginia Citizens Consumer Council, 425 U.S.

748, 765-70 (1976).

4 Pearson v. Shalala, 164 F.3rd 650 (D.C. Cir. 1999); Whitaker v. Thompson, 248

F. Supp. 2d 1 (D.D.C. 2002).

5 The approved claims are for: Selenium & Cancer, Antioxidant Vitamins &

Cancer, Nuts & Heart Disease, Walnuts & Heart Disease, Omega-3 Fatty Acids & Heart

Disease, Olive Oil and Heart Disease, B Vitamins and Vascular disease, Phosphatidylserine &

Cognitive Dysfunction, and 0.8 mg Folic Acid & Neural Tube Defects. See Summary of

Qualified Health Claims Permitted, CFSAN/Office of Nutritional Products, Labeling, and

Dietary Supplements, U.S. Food and Drug Administration, September 2003.

6 See Interim Procedures for Qualified Health Claims in the Labeling of

Conventional Food and Human Dietary Supplements, CFSAN, U.S. Food and Drug

Administration, July 2003.

2

larger body of scientific evidence that contradicted the claim.2

 In essence, the Statement made

clear that the FTC Act allowed adequately qualified claims in areas of emerging science where

the claimed diet-disease relationship was supported by the weight of scientific evidence, but not

by the higher standard of significant scientific agreement. 

The First Amendment commercial speech doctrine has long protected consumers’ right to

receive truthful information.3

 A series of recent court decisions applying this doctrine

overturned on First Amendment grounds FDA’s blanket ban on health claims that do not meet

the significant scientific agreement standard.4

 These decisions prohibit FDA from banning a

health claim unless the agency can demonstrate that disclosures would not be effective in

conveying to consumers the correct level of scientific support for the claim. In Whitaker v.

Thompson, the United States Court of Appeals for the D.C. Circuit determined that qualifying

disclosures, rather than outright proscriptions, may be appropriate when as little as one-third of

the scientific studies support a health claim.

To date, FDA, as a matter of enforcement discretion, has tentatively allowed specific

qualified claims for 9 diet-disease relationships.5 Further, as part of a broader effort to establish

a science-based framework for evaluating requests for new qualified claims, the agency issued a

guidance document in 2003 that suggested a four-tier framework for classifying the strength of

evidence supporting a given diet-disease relationship.6 The highest ranking (hereafter cited as

the “A” level of support) corresponds to significant scientific agreement, and health claims for

such relationships would not need to be qualified. The next ranking (level “B”) corresponds to a

weight-of-the-evidence standard, and health claims would require some degree of qualification. 

The third level (level “C”) represents relatively weak science and would require strong
7 In contrast, the scientific evidence concerning trans fatty acids and heart disease

has solidified, and FDA now regards this relationship as being supported by significant scientific

agreement, i.e. an “A” level claim. 

3

qualification. The last level (level “D”) applies to very weak evidence requiring severe

qualification. 

II. FTC FINAL COPY TEST OF QUALIFIED HEALTH CLAIMS 

A. Overview

The most recent FTC consumer research built on lessons learned from the earlier copy

tests and explored new issues raised by the court decisions on the FDA labeling rules. As

discussed above, the FTC Enforcement Policy Statement on Food Advertising advised marketers

to use qualified health claims only when the weight of scientific evidence supported the dietdisease relationship in question. Our initial consumer research centered on diet-disease

relationships that, at the time of the tests, met this standard of scientific support. We tested

qualifying language that was intended to convey a level of certainty consistent with the weightof-the-evidence standard. The diet-disease relationships chosen for the 1998 Generic Copy Test

were trans fatty acids and an increased risk of heart disease, and antioxidant vitamin supplements

and a reduction in the risk of certain kinds of cancer. This research found that most of the tested

qualifiers communicated significantly lower levels of scientific certainty than did unqualified

claims.

Following the Whitaker decision, it was apparent that both the FDA and FTC needed to

test the effectiveness of qualified claims in cases where the relevant science does not meet a

weight-of-the-evidence standard. Accordingly, I designed new consumer research to determine

whether strong qualifiers could communicate an accurate impression of this weaker scientific

support. Due to shifts in the underlying scientific evidence, it was not necessary to jettison

antioxidant vitamin supplements and a reduced cancer risk as one of the subject areas for study. 

Since 1998, the evidence for the cancer prevention benefits of antioxidant vitamin supplements

has weakened to the point where the FDA chose a “C” level of qualification when it tentatively

approved a qualified health claim for antioxidant vitamin supplements following Whitaker.7

Consequently, this diet-disease relationship was an appropriate mechanism for testing new and

stronger qualifiers, and its inclusion in the new research also allowed useful comparisons with

the results of prior research incorporating milder disclaimers.

Given FDA’s tentative decision to tailor approved health claim language to fit one of four

levels of scientific support (“A” through “D”), additional diet-disease relationships were chosen

to test appropriate qualifiers for as many of the remaining three levels as possible. Testing was

done using eight test cells with 60 subjects in each test condition. Four of these cells were

reserved for antioxidant vitamin supplements in order to test alternative disclaimer formats that 

FDA had proposed for this diet-disease relationship, and to provide a control treatment that
8 One of these ads made an unqualified calcium-osteoporosis claim for a sliced

cheese product that was high in calcium. FDA has approved unqualified health claims in

labeling for this familiar diet-disease relationship. In the 1998 Generic Copy Test, unqualified

claims were tested for both the trans-fatty acid-heart disease and the antioxidant vitamin-cancer

prevention relationships, although at the time of the test FDA had not approved unqualified

claims for either relationship. 

9 These were Atlanta, Chicago, Colorado Springs, New York, Los Angeles,

Schenectady, Toledo, and Wichita. 

4

would measure respondents’ prior beliefs about the efficacy of antioxidant vitamin supplements

in lowering cancer risk. 

Prior FTC copy tests have included advertisements that made unqualified “A” level

health claims.8

 The remaining four cells therefore were devoted to “D” and “B” level dietdisease relationships. Fortuitously, two large scale advertising campaigns current at the time of

the test featured health claims for two such relationships. One of these, for Planters Peanuts,

promoted the possible heart health benefits of peanuts. FDA considers this relationship to be

supported by the weight of scientific evidence, and has tentatively approved a “B” level claim,

which was reproduced verbatim in the Planters ads. 

 The second advertisement was for a real dietary supplement, and contained a qualified

health claim for lycopene and a reduced risk of heart disease. As detailed below, the evidence

for this diet-disease relationship is limited and in some areas inconsistent. For the purposes of

our research, the relationship was assigned a rating of “D.” The advertised product is hereafter

referenced as “Product X.”

Two versions of the lycopene ad were tested. The first was the original ad with no

modifications. In the second version, the qualifying language in the original ad was replaced by

language FDA has suggested as appropriate for a “D” claim. Finally, based on an idea

developed by FDA staff, a test cell was reserved for a novel approach to assessing the efficacy of

qualified health claims. Respondents in this test condition were shown a non-promotional “Fact

Sheet” that summarized in nontechnical terms the state of the scientific support for the lycopeneheart disease relationship. As explained below, this provided an upper-bound reference point for

determining the maximum performance that could be expected from qualifiers appearing in the

context of a promotional advertisement. 

The latest copy test was conducted by U.S. Research in eight geographically dispersed

shopping mall facilities.9

 As indicated, there were 480 respondents, with 60 respondents

assigned to each of eight treatments. 
5

There is much scientific debate about whether 

antioxidant vitamin supplements reduce the risk of some 

kinds of cancer. Most studies have failed to show that 

these vitamin supplements reduce the risk of cancer. 

B. Ad Treatments in Detail

1. Antioxidant Vitamin Supplement Treatments

The various versions of the antioxidant supplement print ads used in this and the three

prior copy tests were for a fictional product called “ACE.” The four treatments used in the most

recent test are shown in Figures 1-4. The first ACE treatment is the “Tombstone Control” ad,

which was used to provide background information on respondents’ beliefs concerning the

certainty of the science supporting a relationship between antioxidant vitamins and a reduced

cancer risk. The ad is identical in appearance to the ads that contain health claims, but the main

text is confined to basic content information:

New ACE provides the complete antioxidant group in one convenient

capsule. Take ACE everyday to make sure you get the antioxidant 

vitamins you want.

The second and third ACE treatments are, respectively, the “Box Disclaimer”and “FDA”

ads, which were also used in the second follow-up copy test. The Box Disclaimer uses a strong

disclaimer in a text block below the sales message:

Some scientific evidence suggests that consumption of antioxidant vitamin

supplements may reduce the risk of certain kinds of cancer. New ACE provides

the complete antioxidant group in one convenient capsule!

 

The FDA ad contains verbatim the language sanctioned by the FDA as acceptable for use

by the litigants in its settlement of the Whitaker case. The main text of this ad is as follows:

Some scientific evidence suggest that consumption of antioxidant vitamin 

supplements may reduce the risk of certain kinds of cancer. However, FDA has

determined that this evidence is limited and not conclusive. 
6

Figure 1

Ace Tombstone Control

The Complete 

Antioxidant 

Group 

New ACE provides the complete antioxidant 

group in one convenient capsule. Take ACE 

everyday to make sure you get the antioxidant . . vltamms you want. 

Try New ACE! 

ACE. The Complete Antioxidant Group 
7

Figure 2

Ace Box Disclaimer

i Thle Complete 

Antioxidant 

Group 

What We Know About 

Antioxidants and Cancer 

Some scientific evidence suggests that consumption of antioxidant vitamin 

supplements may reduce the risk of certain kinds of cancer. New ACE provides 

the complete antioxidant group in one convenient capsule! 

There is much scientific debate about whether 

antioxidant vitiamin supplements reduce the risk of some 

kinds of cancer. Most studies have failed to show that 

these vitamin supplements reduce the risk of cancer. 

ACE. The Complete Antioxidant Group 
8

Figure 3

Ace FDA Disclosure

The Complete 

Antioxidant 

Group 

What We Know About 

Antioxidants and Cancer 

Some scientific evidence suggests that consumption of antioxidant vitamin 

supplements may reduce the risk of certain kinds of cancer. However, FDA has 

determined that this evidence is limited and not conclusive. 

Try New Ace 

Ace Antioxidant Supplement provides the complete antioxidant group in one 

convenient capsule! 

ACE. The Complete Antioxidant Group 
9

Figure 4

Ace Report Card

II TI." Complete 

Antioxidant 

Group 

What We Know About 

Antioxidants and Cancer 

Some scientific evidence suggests that consumption of antioxidant vitamin 

supplements may reduce the risk of certain kinds of cancer. FDA evaluated the 

scientific evidence and gave it a "e" rating, based on a scale from A (strongest 

evidence) to 0 (weakest evidence). 

Try New Ace 

Ace Antioxidant Supplement provides the complete antioxidant group in one 

convenient capsule! 

ACE. The COlTlplete Antioxidant Group 
10 Marketing research has reported high levels of consumer skepticism concerning

the truthfulness of advertising. For a discussion of this research, See Calfee, J.E., & Ringold,

D.J., The Seventy Percent Majority: Enduring Consumer Beliefs about Advertising, Journal of

Public Policy and Marketing 1994 (13): 228-238. 

10

The last ACE ad is in the “Report Card” format that FDA has proposed as a possible

method of communicating the strength of scientific support for a given diet-disease relationship. 

Rather than relying on a direct description of the state of the science, the report card disclosure

format would use one of four letter grades (“A” through “D”) to characterize the relative strength

of the evidence. These grades correspond to the four levels of scientific support discussed

earlier. 

As noted, FDA now considers the antioxidant vitamin-cancer risk relationship to be

backed by a “C” level of scientific evidence. The ACE “report card” ad used the following

wording:

Some scientific evidence suggests that consumption of antioxidant vitamin 

supplements may reduce the risk of certain kinds of cancer. FDA evaluated the

scientific evidence and gave it a “C” rating, based on a scale from A (strongest 

evidence) to D (weakest evidence). 

2. “Product X” Treatments

The first of the three Product X treatments was the original ad with no modifications. 

This ad prefaced the lycopene-heart health relationship with the qualified phrase:

“Emerging science suggests that a remarkable nutrient, Lycopene, is one of the 

ingredients in tomatoes that may help reduce the risk of heart disease.” 

In the second treatment, the “emerging science” qualifier was replaced by a presumptively

stronger qualifier that FDA has tentatively approved for use with category “D” claims:

“Very limited and preliminary evidence suggests that lycopene may help reduce

 the risk of heart disease.”

The third test condition was a non-promotional “Fact Sheet” that described the types of

evidence relevant to the lycopene-heart health claim and the principal limitations and

inconsistencies in that evidence. The intent was to provide a “best case” control condition that

would establish an upper bound on the performance that realistically could be expected from

qualified health claims in advertisements. Respondents presumably would be less skeptical of an

educational Fact Sheet than an advertising message.10 Further, assuming the Fact Sheet was

properly crafted, respondents should gain a more accurate understanding of the certainty of the

science than they would from the much briefer qualifiers in an ad. Thus, if respondents in all
11 See, for example, Sesso, H.D., et al., Tomato-Based Food Products and

Cardiovascular Disease in Women, J Nutr. 2003;133(7): 2336-41.

12 Rissanen, T.H., et al., Serum Lycopene Concentrations and Carotid

Atherosclerosis: The Kuopio Ischaemic Heart Disease Risk Factor Study, Am J Clin Nutr.

2003;(77):133-138; Kohlmeier, L., et al., Lycopene and Myocardial Infarction risk in the

EURAMIC Study, Am J Epidemiol. 1997;(146): 618-26. 

13 Positive results for LDL levels and/or resistance to oxidative stress are reported in

Rao, A.V., Lycopene, Tomatoes, and the Prevention of Coronary Heart Disease, Exp Biol Med.

2002 Nov;227(10): 908-13; Maruyama, C., et al., Effects of Tomato Juice Consumption on

Plasma and Lipoprotein Carotenoid Concentrations and the Susceptibility of Low Density

Lipoprotein to Oxidative Modification, J Nutr Sci Vitaminol., 2001 Jun;47(3): 213-21. But

negative results have been reported in several studies, including Hininger, I.A., et al., No

Significant Effects of Lutein, Lycopene or Beta-Carotene Supplementation on Biological

Markers of Oxidative Stress and LDL Oxidizability in Healthy Adult Subjects, J Am Coll Nutr.

2001 Jun;20(3): 232-8; Dugas, T.R., et al., Dietary Supplementation with Beta-Carotene, But

Not With Lycopene, Inhibits Endothelial Cell-Mediated Oxidation of Low-Density Lipoprotein,

Free Radic Biol Bed.1999 May;26(9-10): 1238-44.

11

three test conditions overestimated the certainty of the science supporting the diet-disease

relationship at issue, irrespective of whether they saw an advertisement or the Fact Sheet, we

might conclude that the cause was not deficiencies in the specific advertising disclosures tested,

but rather the inherent difficulty of communicating such information to consumers in any format. 

Figure 5 presents the Fact Sheet used in the test. The first paragraph describes the

established negative correlation that has been found between consumption of tomato products

and the risk of heart disease, and raises the issue of whether the observed protective effects may

be due to the high levels of lycopene that tomatoes contain.11

The second paragraph explains the types of research that have been conducted on this

issue, and the outcomes. Much of the evidence comes from studies that have found depressed

levels of lycopene in the serum and adipose tissue of subjects with heart disease.12 The tissue

studies are described in very simple terms in the first sentence of the second paragraph of the

Fact Sheet. Such evidence cannot isolate whether lycopene is the active agent, or merely a

marker for some other agent or agents in tomatoes.

There also have been a number of clinical studies that have investigated whether

lycopene supplementation reduces serum LDL cholesterol levels in test subjects, or enhances the

resistance of LDL to oxidation. The result of these studies have been mixed.13 These

inconsistent outcomes are described in the last two sentences of the Fact Sheet’s second

paragraph. Finally, the research on lycopene contains no “gold standard” long-term controlled

studies establishing whether supplementation with this nutrient will in fact reduce the risk of

heart disease. 
12

Figure 5

Fact Sheet for Product X

 What We Know About Lycopene and Heart Disease

A number of studies have found that people who eat diets

rich in tomatoes and tomato products tend to have fewer heart

attacks and other heart problems. Scientists have also studied

whether some of this benefit may be due to lycopene, which

is a nutrient found mostly in tomatoes. 

So far, we have learned that people with heart disease

have less lycopene in their bodies than heart-healthy people

do. Some studies have reported beneficial effects on

cholesterol in the blood when people take lycopene

supplements. But other studies have not found any benefits.

At present there are no long-term studies of whether

people who take lycopene will actually lower their risk of

having a heart attack. So we do not know whether there is

any benefit from taking lycopene supplements. Carefully

controlled and long-term clinical studies will be needed to

answer this question.
13

The last paragraph of the Fact Sheet describes this gap in the literature and its implications. The

wording is quite strong, which seems appropriate in view of the inconsistent results of the

shorter-term serum cholesterol studies.

3. Planters Peanuts Treatment

As shown in Figure 6, the study included one widely-distributed ad for Planters Peanuts

that was devoted exclusively to the possible heart-health benefits of peanuts. At the time the ad

ran, FDA had tentatively approved the following qualified health claim for peanuts: 

Scientific evidence suggests but does not prove that eating 1.5 ozs per day of most 

nuts, such as dry roasted peanuts, as part of a diet low in saturated fat and cholesterol

may reduce the risk of heart disease. 

FDA intended this language to communicate a “weight-of-the-evidence” or level “B” standard,

which would be the first level of qualification below the approved unqualified claims supported

by significant scientific agreement. 

The Mr. Peanut ad included this language verbatim, but preceded it with the following

language in much larger print:

Studies indicate a handful of nuts a day can be good for your heart. 

Finally, a reason to love science.

The ad also placed a heart symbol behind the Planters name logo.

C. Interview Process and Questionnaire Design

Potential respondents were approached while shopping and asked to answer a series of

screening questions to determine eligibility. Subject to gender and age quotas, consumers were

eligible if they passed the usual check for any family employment that would constitute a

conflict of interest (such as employment by a marketing firm or a manufacturer of one of the test

products), and had purchased an item from the relevant product category for themselves or a

family member during the past 30 days. The final sample was 75% female, and ages were

evenly distributed across four categories: 21-29, 30-39, 40-49, and 50 and above.

Consenting respondents were escorted to an interview room, where they were allowed to

see the test ad twice. After the first viewing, the ad was removed from sight and the respondent

was asked to identify the name of the product advertised. The respondent was then allowed to

read the ad again, after which the ad was removed from sight and the main questionnaire

administered. (Interviews were terminated with any respondent who could not correctly identify 
14

Figure 6

Mr. Peanut
15

 1 2 3 4 5 6 7

 Not At All Slightly Somewhat Very 

 Certain Certain Certain Certain

the product advertised after a second exposure). The questionnaire for the Ace Antioxidant

Vitamin Supplement ad is presented in Appendix I. The other questionnaires are essentially

identical. 

The first main question was a completely open-ended request for the main ideas that the

ad communicated to the respondent. This was followed by a “yea-saying” control question that

asked whether the ad said or suggested anything about sodium (which it did not). Respondents

were not eliminated if they gave a positive reply, but the subsequent statistical analysis of the

key close-ended question excluded the responses for these individuals. Respondents were then

asked whether the ad said or suggested anything about the diet-disease relationship that was at

issue in the ad. For example, viewers of the ACE ads were asked whether the ad said or

suggested anything about whether taking antioxidant vitamin supplements reduces the risk of

certain kinds of cancer. Only those respondents who saw a health claim in the ad were asked the

subsequent close-ended ad communication question concerning the certainty of the science

supporting that advertised diet-disease relationship. This question was:

Based on what the ad said or suggested, how certain is the evidence that 

peanuts (lycopene) (antioxidant vitamins) reduce(s) the risk of heart disease (cancer). 

Respondents responded using a card that displayed the following 7-point certainty scale:

The questionnaire then asked respondents to answer the same question, but this time

expressing their personal opinion rather than simply reporting what they thought the ad

communicated. This “beliefs” question was asked of all respondents, irrespective of whether

they had seen any health claim in the ad. For the ACE vitamin ads, this question provided the

major opportunity to compare pre-existing beliefs, as measured by the answers of respondents

seeing the tombstone control ad, with beliefs of other respondents after they were exposed to the

various qualified health claims. 

The next question asked respondents to rate on a 5-point scale how interested they would

be in purchasing the advertised product (“not at all interested” = 1, “extremely interested” = 5). 

The remaining questions asked for the respondents’ education level and household income in

2002.
16

D. Issues Analyzed

Do Qualifiers Matter? The initial 1998 Generic Copy Test and the three subsequent

follow-up tests have explored three related but distinguishable issues relating to ad

communication. The threshold ad communication question is whether qualified health claims

can in fact communicate a significantly lower level of scientific support than unqualified health

claims. In short, do qualifiers make a difference? In the Generic Copy Test and the first followup study, this question was answered by comparing the average certainty ratings from the key

close-ended question for an unqualified ad treatment with the average scores for the various

qualified treatments. 

This test did not repeat the unqualified ad treatments in the second and third follow-up

copy tests. Further, the format of the close-ended certainty rating question was modified after

the first follow-up test, thus preventing direct comparisons of the average certainty ratings

between tests. Thus, our evidence concerning this threshold issue comes primarily from the

earlier research. 

Do Qualifiers Communicate the “Right” Level of Support? The second communication

issue of interest is whether a qualified ad can communicate a level of scientific certainty that not

only is less than that communicated by an unqualified ad, but that also is correct in an absolute

sense for the diet-disease relationship under investigation. For purposes of this analysis, FDA’s

four-tier (“A” through “D”) evidence ranking system was used as the basis for characterizing the

“correct” level of support for the various diet-disease relationships used in the copy test. It

therefore was necessary to establish a rationale for determining which score, or range of scores,

on the questionnaire’s seven-point certainty scale should correspond to each of FDA’s four

designated levels of scientific support. Although the system that was adopted is somewhat

arbitrary and not necessarily superior to other possible assignment rules, it does appear

reasonable and internally consistent.

By way of illustration, consider a qualified cancer health claim for the ACE antioxidant

vitamin product. As noted, FDA has rated the level of support for this diet-disease relationship

as a “C,” which is below a weight-of-the-evidence standard. Presumably, this means that

average scores above the midpoint (four) on our seven-point scale would be inappropriately high

for an antioxidant vitamin-cancer claim. Since scores at the very bottom of the scale would have

to be reserved for a “D” level claim, this suggests very roughly that average scores between three

(or a little below) and four would correspond to the correct level of scientific certainty as

determined by FDA. This would leave mean scores below three as appropriate for a “D” claim. 

Scores above four, but below 7, might reasonably be considered as appropriate for a “B” claim,

leaving scores near or at seven to designate the “A” rating for significant scientific agreement. 

How Varied is the Consumer Response? The third communication issue investigated in

my copy tests is the degree of agreement among respondents concerning the level of scientific

support communicated by an ad treatment. Depending on the distribution of responses on either

side of the mean, the average certainty score may represent the typical response of most
14 Deception Policy Statement, appended to Cliffdale Assocs., Inc., 103 F.T.C. 110, 

176 (1984).

15 Thompson Medical Medical Company, Inc., 104 F.T.C. 648, 805-806 (1984).

Due to policy planning considerations, however, the Commission generally targets cases where

the proportion of deceived consumers exceeds this minimum legal threshold. 

17

respondents seeing an ad, or it may obscure wide variation--even to the point where virtually no

individual respondents may have chosen the average score. From a consumer welfare

standpoint, the degree of this variance is critical. If qualified health claims lead most consumers

either to overestimate or underestimate the level of substantiation for the claim, then very few

consumers will make what for them is the correct purchase decision. Some will refrain from

purchasing the product when they might have considered the option had they not been overly

skeptical of the science, while others will err in the opposite direction. 

This issue has specific application to the FTC’s deceptive advertising enforcement

program. The Commission may determine that an advertisement is deceptive if the

advertisement communicates a misleading and material message to reasonable consumers.14

Under Commission case law, this standard can be met if an ad misleads as few as twenty-two

percent of respondents in a copy test.15 Accordingly, in any investigation that requires a copy

test to determine ad meaning, the distribution of responses across the various response categories

is key to determining whether an advertisement is actionably deceptive. 

E. Results

The discussion of copy test results begins with a summary of the unprompted responses

consumers gave to the initial open-ended question concerning ad meaning, and then focuses on

the responses to the key close-ended certainty question in order to answer the three

communications issues posed above. Finally, responses relevant to beliefs, purchase interest,

and demographics are presented. 

1. Analysis of Open-Ended Questions

The least directed view of consumer takeaway from the various ads is provided by the

initial open-ended question that simply asked respondents to describe the main points of the ad.

Of particular interest is the proportion of respondents who referenced the health claim qualifiers

in some manner. For the three qualified Ace vitamin ads, from 40 percent to 60 percent of

respondents played back an unqualified health message as a main point of the ad. From 17

percent to 25 percent of respondents referred to the health claim in qualified terms, such as

“scientific evidence inconclusive,” “probably doesn’t prevent cancer,” or “not FDA approved

yet.” 
18

For the two Product X ads, about 70 percent of respondents in both test conditions

reported an unqualified health claim. Virtually no one expressed the health claim in qualified

terms. For the Fact Sheet, approximately two-thirds of respondents mentioned an unqualified

claim, and one-third reported a qualification, such as “not sure,” “need more studies,” or “no

long term testing.” Finally, for the Mr. Peanut ad, virtually all respondents mentioned an

unqualified heart-health claim, and only two respondents referenced a qualification. 

These results would seem to indicate that most consumers did not notice the disclaimers

in the ads. The picture changes, however, in the open-ended question that followed the key

close-ended question. Respondents were asked why they had selected a particular rating from

the seven-point certainty scale. Over 80 percent of respondents seeing the three qualified ACE

ads cited qualifying language to justify their answer. Only about 10 percent mentioned an

unqualified claim. About 70 percent of the respondents viewing the unedited Product X ad, and

75 percent seeing the “very limited and preliminary” version mentioned some kind of uncertainty

about the science. For the Fact Sheet, the figure was about 80 percent. Finally, 85 percent of the

Mr Peanut respondents mentioned a lack of proof as the justification for their answer. Thus, it

appears that respondents did notice and remember the disclaimers, but chose to describe the

main points of the ad in very general terms in the initial open-ended question. 

2. Analysis of Close-Ended Questions

a.. Yea-Saying Bias

Very few respondents answered the “yea-saying” control question (Q4) incorrectly by

stating that the ad said or suggested something about an ingredient (sodium) that was not

discussed in the test ad. The proportion of yea-sayers ranged from one percent to four percent.

(These respondents are not included in any of the results discussed below.) With the exception

of the ACE tombstone control ad, over 90 percent of respondents answered correctly that the ad

said or suggested something about the diet-disease relationship at issue, and therefore were asked

the key close-ended communication question concerning the certainty of the science. 

b. Key Ad Communication Results

i. Can Qualifiers Reduce Average Certainty Ratings? 

As noted above, the final follow-up copy test did not include any ads with unqualified

health claims, and the scale used to measure communication of certainty differed from that of the

earlier tests. We therefore must rely primarily on our earlier research to answer the threshold

question of whether consumers on average interpret qualified and unqualified health claims

differently. 

The 1998 Generic Copy Test used print advertisements for two fictional products--the

same “ACE” antioxidant vitamin supplement used in subsequent tests--and a margarine (“Better

Blend”) that was free of trans fatty acids. Three levels of claims were tested. The first was an
19

 Scientists have now proven that supplements containing these same 

antioxidant vitamins also reduce the risk of cancer.

Some medical studies are now finding that supplements containing

these same antioxidant vitamins may also reduce the risk of cancer. 

 What This Means to You

It looks promising, but scientists won’t be sure until longer term

research is completed. 

Some medical studies are now finding that supplements containing

these same antioxidant vitamins may also reduce the risk of cancer. 

 What This Means To You

It’s too early to tell for sure. Some recent studies have failed to show

that these vitamins protect against cancer. Longer term research is

needed. 

unqualified “Proof” claim that presented the relevant diet-disease relationship as a proven fact. 

The second was a mildly qualified claim that was drafted to convey some uncertainty in the

underlying science while still preserving a positive sales message. The third claim was more

heavily qualified (although it was not intended to convey a level of certainty below a weight-ofthe-evidence standard). The key portions of these claims are presented below. In all cases the

claims were preceded by the following language:

Scientists have known for some time about the special health benefits of fruits and 

vegetables that are rich in antioxidants like vitamins A, C, and E. Eating plenty of these 

foods can reduce the risk of certain kinds of cancer.

Proof Claim:

 

Mildly Qualified Claim: 

 

 

Qualified Claim: 
20

The primary close-ended question used to test for deception was: “Based on what the ad

says or suggests, how sure are scientists about whether taking antioxidant vitamin supplements

will reduce the risk of certain kinds of cancer? Respondents answered using a 5-point scale. 

The five choices were (1) Very Unsure, (2) Somewhat Unsure, (3) Neither Sure nor Unsure, (4)

Somewhat Sure, and (5) Very Sure. Parallel disclaimers and questions were used for the Better

Blend margarine ads. 

As can be seem from Figure 7, the average certainty ratings for the ACE vitamin ads fell

as the level of qualification increased. In all instances, however, the certainty ratings were quite

low in absolute terms. For example, the mean score for the vitamin Proof Claim was only 3.85

(just below Somewhat Sure), even though the diet-disease relationship was presented as a proven

fact. This rating fell to 2.88 for the Qualified Claim. There was a statistically significant

difference in mean ratings between the Proof Claim and the Qualified Claim for both the vitamin

ads and the margarine ads (not shown). The ratings for the Mildly Qualified ads were

approximately half way between those of the Proof and Qualified ads, but the differences were

not statistically significant.

The first follow-up copy test used the same ad copy, but changes were made in the

primary close-ended question to test whether the arguably extreme wording of the “top box”

option for the Generic Copy Tests’s 5-part certainty question may have discouraged respondents

from selecting it even when they thought the advertised diet-disease relationship was well

established. Respondents may have construed “Very Sure” as describing a state of scientific

consensus rarely attained, and therefore chose the safer “Somewhat Sure” option. In the followup test, the top and bottom box responses were changed, respectively, to “Sure” and “Unsure.”

The mean certainty ratings for the follow-up test are shown in Figure 8. These ratings

were uniformly higher than those recorded in the Generic Copy Test. The mean scores for the

ACE Proof, Mildly Qualified, and Qualified claims were, respectively, 4.24, 3.72, and 3.35,

compared to the original values of 3.85, 3.39, and 2.88. All of the differences in means were

statistically significant for the follow-up test.

These findings suggest that consumers do notice and take into account disclaimers

concerning the degree of scientific support behind a claim. Further, the significant result for

even the mildly qualified claim in the first follow-up test indicates that very strongly worded

disclaimers may not be needed to affect consumer perceptions of scientific certainty. 
21

How sure are scientists?

Ace Vitamins Generic Copy Test

3.85

3.39

2.88

0

1

2

3

4

5

Proof Mildly Qualified Qualified

Mean Response

Somewhat

Unsure

Neither Sure

 nor Unsure

Somewhat

Sure

Very Sure

Very Unsure

Figure 7
22

How sure are scientists?

Ace Vitamins First Follow-Up Test

4.24

3.72

3.35

0

1

2

3

4

5

Proof Mildly Qualified Qualified

Mean Response

Somewhat

Unsure

Neither Sure

 nor Unsure

Somewhat

Sure

Sure

Unsure

Figure 8
23

ii. Can Qualified Claims Communicate the “Correct” Level of Support?

Antioxidant Vitamin Ads

Results from the most recent follow-up copy test are used to answer this question. Figure

9 presents the mean certainty scores for the four Ace vitamin ads. As indicated above, the

maximum average score consistent with an FDA “C” rating would probably be the midpoint

rating of 4. Scores above this level would be more consistent with the “B” weight-of-theevidence standard. A precise lower cut off is difficult to articulate, but presumably anything

much lower than a 3 would have to suggest a “D” rating if that designation is to have any

meaning. 

On this basis, the mean score for each of the Ace vitamin test ads meets the standard for a

“C” rating. The Box Disclaimer (3.33) and FDA (3.71) scores clearly meet the criterion, and the

Report Card’s mean of 4.04 is statistically indistinguishable from a 4.0. Perhaps surprisingly,

the lowest average is for the tombstone control ad, which did not mention any health benefits,

qualified or unqualified. This result must be interpreted cautiously, however, since only eleven

respondents thought the ad said or suggested anything about a cancer risk-reduction benefit, and

therefore were asked to answer the main ad communication question. The real purpose of the

control treatment is to measure prior beliefs, not ad communication. The control ad results

therefore will be of more interest when the responses to the beliefs question are discussed below. 

In terms of relative performance, the Box Disclaimer mean score is significantly lower

than that of the Report Card (P = .047), and the difference between the Tombstone Control and

the Report Card is almost significant at the .05 level (P = .08). There are no other significant

differences. 

Planters Peanuts and Product X Ads

Figure 10 presents the mean certainty scores for the remainder of the test ads. 

It is readily evident that the Mr. Peanut ad communicated a level “C” certainty rating, rather than

the level “B” that is consistent with current research findings. The mean certainty rating of 3.89

is virtually identical to the rating of 3.71 accorded the “FDA” version of the ACE vitamin ad

(“However, FDA has determined that this evidence is limited and not conclusive.”). Given the

prominent positive statements in the Mr Peanut ad, and the use of a red heart symbol, this

outcome is somewhat surprising. It is possible that the smaller-print disclaimer (“Scientific

evidence suggests but does not prove....”) carried a more negative connotation than FDA

intended when it approved the language for a level “B” claim. It is also possible that

respondents 
24

How certain is the evidence?

Ace Vitamins

2.89

3.33

3.71

4.04

0

1

2

3

4

5

6

7

Control Box Disclaimer FDA Report Card

Mean Response

Not at All

Certain

Slightly

Certain

Somewhat

Certain

Very

Certain

Figure 9
25

How certain is the evidence?

Mr. Peanut, Fact Sheet, Product X

3.89 3.87

4.85

4.23

0

1

2

3

4

5

6

7

Mr. Peanut Fact Sheet Original Product

X Ad

Revised Product

X Ad

Mean Response

Not at All

Certain

Slightly

Certain

Somewhat

Certain

Very

Certain

Figure 10
16 From a statistical standpoint, the mean scores for all of the Ace ads except the

Report Card are significantly lower than the Product X score (P =.0002 to .02).

26

were unfamiliar with, and thus skeptical of, the claimed diet-disease relationship for a product

that is usually considered a snack food. 

The opposite problem is evident in the results for the two Product X ads and, to a lesser

extent, the Fact Sheet. The unaltered ad’s mean score of 4.85 is higher than that of any of the

ACE vitamin ads, and is more consistent with a level “B” weight-of- the-evidence standard than

the grade “D” rating that is appropriate for this particular diet-disease relationship.16 Replacing

the original qualifying language with “very limited and preliminary evidence suggests...” does

appear to lower the mean certainty rating (4.23 vs. 4.85), although the difference is not

significant (P =.106) using the conservative 2-tail test. In any event, the score is still not

consistent with a “D” rating.

The mean score for the Fact Sheet (3.87) is below that recorded for the two Product X

ads, and significantly lower than that for the original unaltered ad (P = .005). None of the mean

scores, however, is significantly lower than that of any of the Ace vitamin ads, and thus these 

results are not consistent with a “D” level of scientific support. Although the relatively high

score registered by the original unaltered ad is perhaps not surprising given the generally

positive tone of the ad, the Fact Sheet’s score near the midpoint of the scale suggests that

communicating levels of certainty as low as those intended for a “D” level of scientific support

may prove very difficult. 

iii. Do Consumers Agree on the Level of Scientific Support?

The final ad communication question explored is whether the average certainty scores

reported above reflect the typical consumer’s take-away from a test ad, or whether consumers

disagree widely about the degree of certainty that an ad communicated. Of particular interest is

whether the ads misled a sufficiently high proportion of consumers to be considered deceptive

under FTC case law. As noted, this proportion need not be higher than approximately one-fifth

of respondents seeing an ad (although in practice the Commission usually targets advertisements

that mislead a considerably higher proportion of consumers). Indeed, an even lower proportion

(16 percent in Thompson Medical) has been considered sufficient if the responses have been

adjusted to reflect the results of a suitable control question or advertisement. This adjustment

process is explained in greater detail below.

In practical terms, these minimum thresholds would pertain only to the fraction of

consumers who overestimated the certainty of the supporting science, since advertisers are not

generally held legally accountable for communicating overly cautious messages about their

products. Yet from a broader policy perspective, consumers are also injured if a particular

choice of qualified language leads them to underestimate the level of support for the claim, and
17 The denominator for the distribution calculations is the total number of

respondents who completed the interview for a given ad and did not respond “yes” to the yeasaying bias question that asked about a nutrient that was not mentioned in the ad. The various

numerators exclude all respondents who did not think the ad communicated the relevant health

claim, or who answered “yes” to the yea-saying bias question. 

27

as a result avoid a purchase they should have made. Thus, the overall distribution of responses

around the mean needs to be examined. 

 ACE Antioxidant Vitamin Ads

Figures 11-14 explore this issue for the four Ace vitamin ads. Each graph shows the

percentage of respondents who chose each of the seven possible certainty ratings. The graphs

also indicate our previously discussed assignment of these ratings to FDA’s four levels of actual

scientific support (“A” - “D”).17

As indicated earlier, only 11 of the 60 respondents in the Tombstone Control test cell

reported that the ad said or suggested anything about a cancer benefit, and were therefore asked

the follow-up question about the certainty of the evidence for the diet-disease relationship. Since

the ad did not contain an explicit health claim, the responses of these individuals can be viewed

as representing the impact of prior beliefs on consumer interpretation of ad meaning concerning

the existence of a cancer benefit, and the certainty of the science backing that claimed benefit. 

That is, these respondents presumably would take away an implied cancer health message

from any advertisement for antioxidant vitamin supplements, even if the ad contained no explicit

health message for which the advertiser could be held legally accountable. The various

percentages registered for the tombstone control ad can be subtracted from the corresponding

percentages for the test ad being evaluated in order to purge the effects that are not due

specifically to the qualified health claim at issue. In this case, the impact will be very slight

given the small number of respondents who saw any relevant health message in the tombstone

control ad. The discussion below includes both adjusted and unadjusted distributions. 

Figure 11 shows that the distribution of responses for the Tombstone Control ad is

concentrated between “Not at All Certain” and “Slightly Certain.” Only 3 respondents rated the

science as “Somewhat Certain or Above.” Figure 12 reports the results for the Box Disclaimer

ad before adjustment for prior beliefs. It is immediately evident that there is practically no

central tendency whatever in consumer interpretation of this qualified claim. Virtually equal

proportions of respondents thought the ad communicated that the science was “Not at All

Certain,” “Slightly Certain,” or “Somewhat Certain.” The remaining responses are scattered

across the distribution from a certainty rating of 2 to the highest possible level of 7. 
28

How certain is the evidence?

Responses for Ace Control

5.26%

1.75%

5.26%

0% 1.75% 0% 

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B 

Figure 11
29

How certain is the evidence?

Responses for Ace Box Disclaimer

22.41%

10.34%

22.41%

5.17%

24.14%

1.72%

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D CB 

Figure 12
30

How certain is the evidence?

Responses for Ace FDA Disclosure

20.69%

12.07%

6.90% 8.62%

34.48%

5.17% 

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B 

Figure 13
31

How certain is the evidence?

Responses for Ace FDA Report Card

10.34%

6.90%

15.52% 17.24%

25.86%

3.45%

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B 

Figure 14
32

These results show that about 35 percent of respondents thought the ad portrayed the science as

“Somewhat Certain” or above, and thus higher than the targeted level of “C.” This result is

significantly higher than the 22 percent Thompson Medical standard (P = .045). Net of the

control results, the figure falls to 30 percent, but is also significantly higher than the

corresponding Thompson Medical figure of 16 percent net of control (P =.025). An

approximately equal proportion (33 percent) of respondents gave certainty scores below 3.0, and

thus may actually have underestimated the level of support for this claim. Only 28 percent of

respondents chose “correct” ratings of 3.0 or 4.0.

Figure 13 shows the unadjusted distribution of ratings for the FDA Disclosure. In this

case, the modal response is clearly “Somewhat Certain,” which was the choice of 34.5 percent of

respondents. The remaining responses are broadly distributed across the continuum. Almost

one-half of the respondents (47 percent) rated the science at a level of “B” or above. Although

accounting for the control results reduces this figure to 43 percent, either percentage is

significantly higher than the corresponding Thompson Medical standard (P<.001). 

Finally, Figure 14 displays the results for the FDA Report Card test cell. The distribution

has a small peak at “Somewhat Certain,” but otherwise there is little agreement among

respondents. About 40 percent of respondents overrated the certainty of the science (35 percent

net of control). Either figure is significantly higher than the corresponding Thompson Medical

threshold proportion (P=.008 and .004 respectively). 

These results document serious shortcomings in the performance of the tested qualifiers

for the ACE vitamin supplement ads. Using standards established under FTC case law, none of

the qualifiers succeeded in forestalling a deceptive message for a substantial proportion of

consumers. Further, there was broad disagreement among respondents as to what level of

scientific certainty was being communicated by the various Ace advertisements.

Mr. Peanut Ad

The corresponding results for the Mr. Peanut ad are presented in Figure 15. Here 

we are interested in the proportion of respondents who rated the certainty of the science

supporting a heart-health benefit for peanuts at an “A” level, rather than the actual level of “B.” 

Depending on whether the lower bound for an A is considered a certainty rating of 6 or 7, only

from about 5 to 10.5 percent of respondents chose an inappropriately high rating. (There was no

tombstone control for this ad, and no further adjustment is possible for prior beliefs.). From a

welfare perspective, the only difficulty with the Mr. Peanut ad is its failure to communicate a

sufficiently high certainty rating (above a 4) to about half of the respondents. 

Product X Ads and Fact Sheet

Finally, Figures 16-18 show the response distribution for the Fact Sheet, Original Ad, and

Revised Ad, respectively. They show the same broad range of interpretations as the Ace and Mr.

Peanut ads, and reveal that a substantial proportion of respondents in all the test cells 
33

How certain is the evidence?

Responses for Mr. Peanut

12.28%

8.77%

17.54%

12.28%

35.09%

5.26% 

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B 

Figure 15
34

How certain is the evidence?

Responses for Fact Sheet

21.67%

5.00%

16.67%

8.33%

28.33%

10.00% 

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B 

Figure 16
35

How certain is the evidence?

Responses for Original Product X Ad

3.39%

6.78%

11.86%

6.78%

35.59%

6.78%

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B

Figure 17
36

How certain is the evidence?

Responses for Revised Product X Ad

8.47%

13.56%

6.78%

15.25%

32.20%

11.86%

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

D C B 

Figure 18
37

overestimated the certainty of the supporting science. If we specify conservatively that any

certainty rating above a 3 (Slightly Certain) is inappropriately high for a level “D” health claim,

57 percent of respondents were arguably deceived by the Fact Sheet. This percentage rises to 68

percent for the Very Limited Evidence ad, and 72 percent for the Original ad. Increasing the

demarcation point to a rating of 4 (the unlabeled midpoint of the distribution) reduces these

percentages only slightly. In all cases, approximately one half or more of respondents rated the

science as more certain than even this lenient benchmark. 

iv. Beliefs

As discussed earlier, respondents were also asked to rate the certainty of the science for

the relevant diet-disease relationship relying on their personal opinion, rather than basing their

answer strictly on what they thought the ad was communicating about this issue. In all cases the

beliefs question followed the communication question so as not to bias consumer interpretation

of ad meaning. The beliefs question serves two purposes. 

 First, from a policy planning perspective, it is useful to determine more carefully the

weight that consumers give to any deceptive message they may find in an ad. The results

discussed above indicate that the ACE vitamin ads and the Product X ads communicated a more

certain health message to a substantial proportion of respondents than could be justified by the

current state of the science. If, however, respondents discounted this message and gave as their

personal opinion a rating more consistent with the actual scientific evidence, there might be less

cause for concern (even though from a legal standpoint a firm could still be found liable for

communicating the deceptive message).

Second, the beliefs question allows a more meaningful application of the Ace Tombstone

Control ad. Unlike the communication question, which most respondents did not answer

because they did not see a health message in the tombstone ad, the beliefs question could be

asked of everyone in the test cell. Thus, from a statistical standpoint, the results of the beliefsbased certainty ratings would provide a much more powerful test for any differences in the mean

certainty ratings between the Tombstone Control subjects and the three groups of test subjects

seeing the ACE ads. Similarly, subtracting the Tombstone Control ratings across the seven

possible response categories from the corresponding figures for the for the three ACE test ads

will provide a detailed picture of the net impact of the tested qualified claims on respondents’

beliefs concerning the certainty of the science supporting the antioxidant vitamin-cancer

relationship. 

Figures 19 and 20 compare the mean certainty ratings for communication and beliefs for

the various test ads. It is readily apparent that the two measures track each other very closely.

Indeed, there are no statistically significant differences between the mean communication score

and the corresponding mean belief score for any of the ads. Further, when the belief results are

substituted for the communication results in the statistical analysis reported above, there is no 
38

Comparing communication and belief

Ace Vitamins

2.89

3.33

3.71

4.04

3.18

3.41

3.60

4.21

0

1

2

3

4

5

6

7

Control Box Disclaimer FDA Report Card

Mean Response

Communication

Belief

Not at All

Certain

Slightly

Certain

Somewhat

Certain

Very

Certain

Figure 19
39

Comparing communication and belief

Mr. Peanut, Fact Sheet, Product X

3.89 3.87

4.85

4.23

3.86

3.63

4.36

3.90

0

1

2

3

4

5

6

7

Mr. Peanut Fact Sheet Original Product

X Ad

Revised Product

X Ad

Mean Response

Communication

Belief

Not at All

Certain

Slightly

Certain

Somewhat

Certain

Very

Certain

Figure 20
40

substantive change in the conclusions concerning the “accuracy” of the average certainty ratings,

or the proportion of respondents who overestimate or underestimate the certainty of the science.

The close correspondence is open to several interpretations. The results could indicate

that the perceived message in the ads had actually shaped respondents’ beliefs. It is also possible

that, when asked the initial communication question, respondents read into the ads whatever they

happened already to believe. Finally, we cannot reject the possibility that respondents simply

did not understand the intended distinction between the communication question and the beliefs

question. 

The Ace Tombstone Control ad results do not suffer from this ambiguity, since there

were no explicit or strongly implied health claims in the text to confound the issue. Thus, the

respondent ratings can be construed as reflecting the baseline beliefs of respondents concerning

the certainty of the science supporting the antioxidant vitamin-heart health relationship. 

As shown in Figure 21, the results for the Tombstone Control ad suggest that these

consumers on average brought with them a more skeptical view of the supporting science than

the other respondents expressed after viewing the ACE test ads with explicit health claims. The

mean certainty score of 3.18 for the Tombstone Control is lower than that of the other three cells,

although only the difference between the Report Card ad and the Tombstone Control is

statistically significant (P=.003). Thus, we can dismiss any concern that the three test

advertisements would have had to overcome very strong positive prior beliefs in order to achieve

mean certainty ratings consistent with the correct level of scientific certainty for this diet-disease

relationship.

The final question is whether the tombstone control results ameliorate in any way the

conclusion that a substantial proportion of respondents (although not the mean respondent)

overestimated the strength of support for the antioxidant vitamin-heart health relationship when

asked to express either what the ad communicated or what their personal opinion was about the

certainty of the science. We are interested in comparing the percentage of respondents in the

Tombstone Control cell who believed the certainty of the science was above a 4 with the

corresponding percentage in each of the three test cells. From Figure 22, it can be seen that 31.6

percent of the control respondents gave a certainty rating above a 4. This is virtually identical to

the proportion of respondents in the Box Disclaimer cell who rated the science above a 4 (32.7

percent). It would appear, therefore, that this test ad had no net impact on the proportion of

consumers who overestimated the strength of the science. 
41

In your personal opinion, how certain is the evidence?

Ace Vitamins

3.18

3.41

3.60

4.21

0

1

2

3

4

5

6

7

Control Box Disclaimer FDA Report Card

Mean Response

Not at All

Certain

Slightly

Certain

Somewhat

Certain

Very

Certain

Figure 21
42

Belief

Responses for Ace Control

36.84%

5.26%

14.04%

10.53%

24.56%

1.75%

0%

10%20%30%40%50%60%70%80%90%

100%

123456

% Replying

Not at All

Certain

Slightly

Certain

Somewhat

Certain

Figure 22
43

For the FDA disclosure, there was a net increase of about 10 percentage points (41.4 percent vs.

31.7 percent for the Tombstone Control). This difference in proportions is not statistically

significant, however (P=.14). Finally, for the Report Card Ad, 44.8 percent of respondents rated

the certainty of the science above a 4, for a net increase of 13.1 percentage points. This

difference is almost significant (P=.08). Taken as a whole, these results do not show that the

Ace test ads had an appreciable impact on beliefs. As previously stated, however, FTC case law

on deception is based on ad communication, not beliefs, and these findings therefore have no

legal implications. 

v. Purchase Interest

Respondents in all of the test cells except the Fact Sheet (which did not reference a

specific product) were asked to rate on a five-point scale how interested they would be in

purchasing the advertised product. The choices were “not at all interested,” “not very

interested,” “somewhat interested,” “very interested,” and “extremely interested.” The results

provide another view of respondents’ reactions to the qualifications in the ads. Presumably, the

mean purchase response would tend to follow the same pattern as the mean certainty ratings for

the products. The purchase ratings also allow some indication of how convinced respondents

were by the overall presentation of the ads. Of particular interest is whether purchase interest in

the fictitious ACE vitamin supplement is substantially below that of the two real products in the

test.

For the ACE vitamin ads, the lowest mean score was a 2.21 for the Box Disclaimer ad.

The other three ads recorded very similar scores, ranging from 2.47 for the FDA ad to 2.60 for

the Tombstone Control. Although the pattern of scores for the three ads with health claims do

track the pattern for the certainty scores, the only significant difference was between the Box

Disclaimer and the Tombstone Control (P=.05). The two Product X ads recorded identical

purchase interest scores of 2.88, indicating that respondents did not attach much importance to

the stronger qualifier used in the second of these ads. Given that the overall tone of the Product

X ads was quite positive, and that Product X is a leading brand of its type, the somewhat lower

purchase interest scores for the fictitious ACE vitamin product do not necessarily indicate that

respondents were reacting negatively to the test ads’ appearance or credibility. The Mr. Peanut

ad, with a mean score of 3.42, displayed the most sales appeal of any of the treatments. Absent a

control ad that made no heart health claim, we cannot determine whether respondents were

interested in purchasing the peanuts for a heart benefit or merely as a snack food. 

vi. Demographic Analysis

The test questionnaires included questions on respondent education and income status,

and gender and age information was gathered during the screening process. Probit regressions

were run on the pooled data and the individual test cell data to determine whether respondents

with a particular demographic characteristic were more likely to assign the correct level of

certainty in response to the key communication and beliefs question. 
44

The regression using the pooled data did not reveal any significant relationships between

the demographic variables and the probability of choosing an appropriate certainty rating. If the

analysis is limited to the pooled data for the four ACE vitamin ads, the coefficient for education

was significant and positive (P=.05). Neither education nor any other demographic variable was

systematically related to the choice of certain rating in the data for the individual ACE ads or in

any of the other ad treatments. 

III. CONCLUSIONS

The results of my copy tests suggest several conclusions. First, it would appear that it is

possible to convey a moderate level of scientific uncertainty to consumers. All of the treatments

except the unaltered Product X ad conveyed a level of certainty consistent with a “B” level or

below to at least eighty per cent of respondents. Indeed, for the one ad that should have

generated “B” certainty ratings based on current science (Mr Peanut), respondents discounted the

scientific support too heavily. 

Second, the results from the ACE vitamin ad treatments suggest that very clear and

strong disclaimers, with no contradictory elements elsewhere in the ad, can reduce mean

certainty ratings to below a weight-of-the-evidence standard. Further, our results suggest that

FDA’s report card approach to conveying scientific certainty may be almost as effective as

stronger explicit disclaimers, such as those used in the Box Disclaimer and FDA Disclosure ads. 

Third, based on the results for the Product X ads and the Fact Sheet, it would appear that

it is extremely difficult to communicate a mean level of certainty as low as a “D.” Although it is

true that the Product X ads contained other language that could have detracted from the

qualifying language, the Fact Sheet was crafted specifically to convey in a nonpromotional

format serious limitations in the underlying science. Yet the mean certainty rating for the Fact

Sheet was near the mid-point on the rating scale, and virtually identical to that for the ostensibly

more positive Mr. Peanut ad.

Finally, and perhaps most important, respondents interpreted the qualifying language in

the various ads and Fact Sheet in widely disparate fashion. Almost without exception, there was

no “typical” certainty rating. Although the qualifiers did move the overall distribution of ratings

toward greater uncertainty, these qualifiers were not capable of focusing interpretation narrowly

in one range of certainty, and often were not capable of generating an overall average rating that

was consistent with the actual state of scientific certainty for the relevant diet-disease

relationship. Future researchers may wish to investigate whether alternative disclosure formats,

such as those using figures, graphs, or other symbols, might communicate scientific uncertainty

more successfully.
Appendix I

Main Questionnaire for ACE Vitamin Ad
 ID#_______

 ACE VITAMIN SUPPLEMENT ADVERTISING STUDY

Advertisement: (Circle One) A B C D

ESCORT RESPONDENT INTO INTERVIEWING ROOM. SEAT RESPONDENT AT

TABLE. IF RESPONDENT INDICATED EARLIER THAT S/HE WEARS GLASSES FOR

READING, BE SURE THAT S/HE IS WEARING THEM. 

Hello, my name is_________from U.S. Research. As mentioned earlier, we are conducting a

study today about advertising. I am going to show you an advertisement. Please read it

carefully and let me know when you are finished. 

GIVE RESPONDENT AD. WHEN RESPONDENT INDICATES THAT S/HE IS FINISHED

LOOKING, TAKE BACK AD AND REMOVE FROM VIEW.

1 What was the name of the product that was advertised?

1 ACE

2 OTHER

9 DON=T KNOW, DON=T REMEMBER OR NOT SURE

Since people often read ads more than once, I would like you to look at the ad again. Please take

time to review it carefully. When you are done, I will take back the ad and then ask you some

questions. There are no right or wrong answers to these questions. If you don=t know an answer,

that=s o.k., just say AI don=t know.@

GIVE RESPONDENT AD. WHEN RESPONDENT INDICATES THAT S/HE IS FINISHED

LOOKING, TAKE BACK AD AND REMOVE FROM VIEW.

2 Although you may have told me this before, what was the name of the product that was 

 advertised?

1 ACE (CONTINUE)

2 OTHER (TERMINATE)

9 DON=T KNOW, DON=T REMEMBER OR NOT SURE (TERMINATE)
3 What were the main ideas that the ad communicated to you? (RECORD VERBATIM.

 PROBE UNTIL UNPRODUCTIVE WITH: Anything else?)

4 Did the ad say or suggest anything about sodium?

1 YES, IT DID 

2 NO, IT DIDN=T

9 DON=T KNOW, NOT SURE OR DON=T REMEMBER 
5 Did the ad say or suggest anything about whether taking antioxidant vitamin supplements 

 will reduce the risk of cancer?

1 YES, IT DID (GO TO Q6A)

2 NO, IT DIDN=T (GO TO Q7)

9 DON=T KNOW, NOT SURE OR DON=T REMEMBER (GO TO Q7)

(HAND RESPONDENT CARD A)

6a Now I am going to ask a question that I want you to answer using this card. After I have read

the question, please select the number on the scale that you think is most appropriate. You may

select any of the numbers shown, not just the ones with descriptions underneath them. 

Based on what the ad said or suggested, how likely is it that taking antioxidant vitamin

supplements will reduce the risk of certain kinds of cancer? Remember, you may select any of

the numbers shown.

RECORD NUMBER SELECTED____________

 9 Don=t Know

6b Why did you select that number? RECORD VERBATIM. PROBE UNTIL 

UNPRODUCTIVE WITH: Anything else?)

(HAND RESPONDENT CARD B)
7) So far I have been asking you questions about what the ad said or suggested. Now I would

like to ask you a question concerning your personal opinion about antioxidant vitamin

supplements. In your personal opinion, how likely is it that taking antioxidant vitamin

supplements will reduce the risk of some kinds of cancer? Please choose the number that seems

most appropriate to you. 

RECORD NUMBER SELECTED _____________

9 Don=t Know 

8 Based on the information in the ad, how interested would you be in purchasing the product?

 (READ CHOICES AND CIRCLE ONE ANSWER)

1 Not at all interested

2 Not very interested

3 Somewhat interested

4 Very interested

5 Extremely interested

(HAND RESPONDENT CARD C)

9 Which of the following describes your education?

1 Some High School or less

2 High School Graduate

3 Some College or Technical School

4 College Graduate

5 Post Graduate

9 Refused (TABULATE BUT DON=T READ)

 (HAND RESPONDENT CARD D)

10 Which letter on this card best describes your household=s total income before taxes in 2002?

1 A. Under $25,000

2 B. $25,000 to $49,999

3 C. $50,000 to $74,999

4 D. $75,000 or more

9 Refused (TABULATE BUT DON=T READ)

That=s all the questions I have for you. Thank you very much. Would you please sign the

certification page so I can show my supervisor that I interviewed you? You may be contacted

later to verify that the interview occurred, but information you provide will be kept confidential

and will not be used to sell you anything.
RESPONDENT CERTIFICATION 

I certify that I was shown a print ad, asked some questions about it, and paid $3.00 for my

participation.

RESPONDENT NAME (PRINT)____________________________________________

SIGNATURE______________________________________

DAYTIME PHONE____________________________________

HOME PHONE, IF DIFFERENT__________________________________

DATE___________________

INTERVIEWER CERTIFICATION

I hereby certify that all of the above information was obtained by me from the respondent named

above, who is not personally known to me. I agree to provide this affidavit under oath,

immediately upon request.

INTERVIEWER NAME (PRINT)____________________________________________

SIGNATURE______________________________________

DATE__________________