ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

ESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDETO OPINION POLLS AND
PUBLISHED SURVEYSESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

All ESOMAR world research codes andguidelines, including latest updates, areavailable online at www.esomar.org© 2009 ESOMAR/WAPOR. All rights reserved. No part of this publication
may be reproduced or copied in any form or by any means, or translated,without the prior permission in writing of ESOMAR or WAPOR.
The ESOMAR/WAPOR Guide to Opinion Polls and Published Surveys wasdrafted in English and the English text is the definitive version.

ESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYSCONTENTS

3

2. Frequently asked questions about polls

4

3. Guideline for the publication of opinion poll and survey results 14
3.1 Introduction
14
3.2 Scope 		
16
3.3 Requirements of the ICC/ESOMAR International Code 		163.3.1 General requirements 		163.3.2 Specific requirements for published		public opinion surveys 		173.3.3 Arrangements between the Research		
Organisation and its Client 		214. Specific guidance on conducting pre-election
opinion polls and polls on voting intention 		244.1 Introduction 		244.2 The Guidelines		25
4.2.1 Timing of fieldwork 		254.2.2 Sample size 		254.2.3 Sample distribution 		264.2.4 Telephone interviewing 		274.2.5 Online polls 		274.2.6 Weighting 		284.2.7 Adjustments 		284.2.8 Survey content 		294.2.9 Time series 		294.2.10 Consistent design 		294.2.11 Indicator variables 		294.2.12 Unusual poll results		30
5. Guideline on conducting exit polls and election result forecasts 		315.1. Introduction 		315.2. Specific requirements for exit polls 		315.2.1. Items for minimal disclosure 		325.2.2. Election projection methods and their disclosure
34

ESOMAR WORLD RESEARCH CODES & GUIDELINES

for researchers

1. Opinion polls and democracy – an overview

faq

2

for researchersand commissioners

Introduction to the guideESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

ESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDE TO OPINIONPOLLS AND PUBLISHED SURVEYS

Introduction to the guide
Public opinion polls are regularly conducted and published in many countries. They measure not only support
for political parties and candidates, butalso public opinion on a wide rangeof social and political issues. They are
published frequently by a variety ofprint, online and broadcast media.
The public discussion of opinion polls
is not always well informed. However,the case for restricting the publicationof polls during election campaigns ishard to support with rational argument
or empirical evidence. ESOMAR and
WAPOR have produced the present
guide in order to help those interested
in the subject of opinion polls to reacha more informed judgement about
the value of such polls and the mostappropriate ways of conducting andreporting them.
There are five sections in this guide.
First there is a statement on the roleof opinion polls in democratic systems.
ESOMAR and WAPOR’s position on
this issue is quite clear. We believe that
there should be no restriction on the
conduct or publication of opinion polls
which have been carried out according
to the ICC/ESOMAR International
Code on Market and Social Research

ESOMAR WORLD RESEARCH CODES & GUIDELINES

and published according to the
ESOMAR Guideline for the Publication of Opinion Poll Results. The ICC/ESOMAR International Code and thisassociated Guideline are observed bymembers of ESOMAR working in morethan 100 countries around the world.
Second there is a section on frequently
asked questions about how to judge
the quality of an opinion poll. These
questions are designed to help journalists, politicians, academics, the public
and other interested parties understandhow to judge the quality of an opinion
poll.
Third there is the ESOMAR Guideline
for the Publication of Opinion Poll andSurvey Results which is intended toreduce the risk of the public being
misled by research which is inadequate
or badly presented or interpreted.
Fourth there is a section which provides specific guidance for practitionerson the conduct of pre-election polls.
Finally, there is a Guideline for the
Conduct of Exit Polls and Election Results Forecasts. These last two guidelines are designed to help ensure thatelection polls are carried out to highstandards.They provide technical guidance to researchers and backgroundinformation but they are not intended toESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

Why opinion polls are essential for democracy

be a “How to do it” manual. The subjectof public opinion research and themeasurement of voting intention continue to evolve and each election may
bring new circumstances the researcher must address. The professional skills
and previous experience of pollingorganisations are essential componentsof effective public opinion research. It is
not possible to write them into guidelines or codes of practice.
1. Opinion polls anddemocracy – an overview
Public opinion is a critical force inshaping and transforming society.
Properly conducted and disseminatedsurvey research gives the generalpublic an opportunity for its voice to beheard. Through opinion research the
public, politicians, the media and otherinterested groups have access toaccurate measures of public attitudes
and intentions.
“Scientific” polling is among the mostsuccessful political developments of thelast century. Public opinion polls help
guide policy by giving decision-makers
impartial information about what the
public wants. Polls also alert the public
to their own hopes, desires, and political goals. They are mirrors, permitting
individuals to understand where they fit
into the political system. Media reports

ESOMAR WORLD RESEARCH CODES & GUIDELINES

3of the results of opinion polls tell readers and listeners that their opinions areimportant, and can even sometimes bemore important than the opinions of theelite.
The democratic urge towards participation and the journalistic desire to ask
questions have merged to create theextensive media polling of the last 70
years. Imagine a political system where
the public is told what it thinks by its
political leaders, where election winners have the ability to tell voters why
they voted the way they did and where
the government, when it looks for public input, asks only its friends what
the public thinks. The alternative toproperly conducted polls is a public anda government exposed only to unscientific and probably inaccurate assertions
about what people believe, in manycases presented by partisan individuals
or organisations with a political agenda.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

4

Understanding opinion polls

2. Frequently asked
questions about polls
1. What is an opinion poll?
An opinion poll is a scientific and representative survey designed to measure
the views of a specific group – forexample a country’s electors (for mostpolitical polls) or parents or trade union
members.
2. What makes a survey “scientific”?
The two main characteristics of scientific surveys are a) that respondents
are chosen by the research organisation according to explicit criteria toensure representativeness, rather thanbeing self-selected, and b) that questions are worded in a balanced way.
For example, if the population being
sampled contains 52% who are womenand 30% who are over 55, then a scientific opinion poll will represent thosegroups appropriately and the questions will be balanced and not lead the
respondent towards a particularanswer.
3. How does a poll choose a sample
that is truly representative?
There are two main methods. The firstis “random” sampling, the second “quota sampling”. With random sampling, apolling organisation either uses a list of
randomly-drawn telephone numbers or
email addresses (for telephone or some

ESOMAR WORLD RESEARCH CODES & GUIDELINESInternet polls); or visits randomlydrawn addresses or names from a list
such as an electoral register (for someface-to-face surveys). The pollingorganisation then contacts people on
those telephone numbers or at thoseaddresses, using a random selection procedure, and asks them to takepart in the survey. “Quota” sampling
involves setting quota controls – for example, age and gender – and letting theinterviewer seek out different people
who, together, match those characteristics. Surveys based on quota
sampling are often used in face-to-facesurveys. In addition, some Internet polls
employ quota sampling to select representative samples from a database ofpeople who have already provided suchinformation about themselves. Quota
sampling may be used, in otherwise
randomly sampled telephone surveys,
to select the person to be interviewed
within the household, in order to speed
up the fieldwork process.
4. Do polling companies do anythingelse to achieve representative
samples?
Yes, they do. While well-conducted
random and quota samples provide
a broad approximation to the public,
there are several reasons why theymight contain slightly too many of somegroups and slightly too few of others.
What normally happens is that polling

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

5

faq

Understanding opinion polls

companies ask respondents not only
about their views but also about themselves. This information is then used tocompare the sample with, for example,
census statistics. The raw numbers
from the poll may then be adjusted
slightly, up or down, to match the profile of the population being surveyed.
If, for example, a poll finds that, when
its survey-work is complete, it has 100
members of a particular demographic
group, but should have 110 of them (ina poll of, say, 1,000 or 2,000), then it
will “weight” the answers of that groupso that each of those 100 respondentscounts as 1.1 people. This way, thepublished percentages should reflect
the population as a whole.
5. Are other kinds of surveys boundto be wrong?
No. Just as a stopped clock tells theright time twice a day, unscientificsurveys will occasionally produce rightpercentages. But they are far more
likely to be inaccurate. The most common forms of unscientific surveys arephone-in polls conducted by television programmes and self-selecting
surveys conducted over the Internet.
These contain two defects. First, theirsamples are self-selecting. Such polls
tend to attract people who feel passionately about the subject of the poll,rather than a representative sample.
Second, such polls seldom collect

ESOMAR WORLD RESEARCH CODES & GUIDELINES

the kind of extra information (such as
gender and age) that would allow somejudgement to be made about the natureof the sample.
6. But surely a phone-in or write-inpoll in which, say, one million
people take part is likely to be moreaccurate than an opinion poll sample
of 1,000?
Not so. A biased sample is a biased
sample, however large it is. One
celebrated example of this was the US
Presidential Election in 1936. A magazine, Literary Digest, sent out 10 million
post cards asking people how theywould vote, received almost 2.3 million
back and said that Alfred Landon was
leading Franklin Roosevelt by 57-43
percent. The Digest did not gather
information that would allow it to judge
the quality of its sample and correct, or“weight”, groups that were under- or
overrepresented. Since the Literary
Digest sent its postcards primarily toindividuals with telephones and automobiles, their “sample” included few
working class people. A young pollster
called George Gallup employed a muchsmaller sample (though, at 50,000, it
was much larger than those normallyused today), but because he ensured
that it was representative, he correctly
showed Roosevelt on course to win bya landslide.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

6

Understanding opinion polls

7. How can you possibly tell what
millions of people think by asking
just 1,000 or 2,000 respondents?
In much the same way that a chef canjudge a large vat of soup by tasting
just one spoonful. Providing that thesoup has been well stirred, so that the
spoonful is properly “representative”,
one spoonful is sufficient. Polls operateon the same principle: achieving
representative samples is broadly akinto stirring the soup. A non-scientificsurvey is like an unstirred vat of soup.
A chef could drink a large amount fromthe top of the vat, and still obtain amisleading view if some of the ingredients
have sunk to the bottom. Just as the trickin checking soup is to stir well, rather thanto drink lots, so the essence of a scientificpoll is to secure a representative sample,
rather than a vast one.
8. But isn’t there some risk ofsampling error in a poll of 1,000 or
2,000 people?
Yes. Statistical theory allows us toestimate this. Imagine a country that
divides exactly equally on some issue– 50% hold one view while the other
50% think the opposite. Statisticaltheory tells us that, in a random poll of
1,000 people, with a 100% response
rate, then 19 times out of 20, a poll
will be accurate to within 3 percentage
points. In other words, it will record
at least 47%, and no more than 53%,

ESOMAR WORLD RESEARCH CODES & GUIDELINES

for each view. But there is a one in 20
chance that the poll will fall outside thisrange. With a sample of 2,000, the poll
will be within 2 percentage points 19
times out of 20.
9. You say those calculations apply
to “a random poll with a 100%
response rate”. Surely that’s pie inthe sky?
Fair point. Many polls are non-random,and response rates are often verymuch lower – well below 50% in many
countries for polls conducted over justa few days.
10. So isn’t the real margin of error
much larger?
Possibly – but possibly not. Here are
two examples, at opposite extremesof this issue. Return to our example ofan equally divided country. Suppose
everyone who holds view A lives in
the northern half of the country, while
everyone who holds view B lives in thesouthern half. In that case, if pollsters
ensure that half of each survey isconducted in the north, and half in thesouth, then their polls should be exactly
accurate. Structuring polls in this kindof way is called “stratification”. Properly
done, stratification can help to increasea poll’s accuracy.
Now make a different assumption about
our mythical, equally divided country.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

7

faq

Understanding opinion polls

Suppose people who hold view A are
far more likely to be members of areligious or ethnic minority who refuse
to answer questions than people who
hold view B. Unless the polling company is aware of this bias, and knows
how big it is, it could well produce
results showing that view B is far more
popular than view A. This is an example
of a systematic error.
To measure the “true” margin of error,
we would need to take account ofrandom sampling error, and the effectsof stratification, and possible systematic
errors. The trouble is that it is hard,
and arguably impossible, to be sureof the true impact of stratification andsystematic errors. (If the impact of all
systematic errors were known, a competent survey company would adjust its
results to compensate for them.)
11. Doesn’t this mean that polls can’t
really be trusted at all?
No. Polls may not be perfect, butthey are still the best way of measuring
what the public thinks. In mostcountries where poll results can becompared with actual results (suchas elections), well-designed polls areusually accurate to within 3 percentage
points, even if they occasionally strayoutside that margin of error. Moreover,much of the time, polls provide a good
guide to the state of opinion, evenallowing for a larger margin of error. If

ESOMAR WORLD RESEARCH CODES & GUIDELINES

a well-designed, representative survey
finds that the public divides 70-30%
on an issue, then a margin of error ofeven 10 percentage points cannot alter
the fact that one view is expressed far
more widely than the other. However, it
is true that in a closely-fought election,a polling lead (in a sample of 1-2,000)
of less than 5 percentage points for one
candidate or party over another cannot
be regarded as a certain indicator of
who was ahead at the time the survey
was taken – let alone a guarantee of
who will in the days, weeks or months
ahead.
12. Can I Trust an Internet Poll?
The answer depends on the people
being sampled and the answer is thesame for telephone or face-to-faceinterviewing as well, not just theinternet. If a representative sample ofpeople can be reached by any of theseinterviewing modes, the answer isyes, the poll can be trusted. There areplaces and populations where it is very
difficult for an interviewer to get a representative sample using face-to-faceinterviewing, just as there are places
and populations where only certainclasses of people have telephones oraccess to the internet. Obviously if theinterviewing mode cannot be correctedby stratification and weighting to allow
for this bias, the interviewing modecannot provide a representative sample

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

8

Understanding opinion polls

and should not be trusted. In somecases the coverage of the internet may
be sufficiently high for internet interviewing to be a good, or even the best,
way of carrying out surveys, but withthe level of access to the internet still
low and unrepresentative in most countries, internet polls of the general publicshould be treated with caution.
Internet surveys use panels of people
who have given permission to becontacted for surveys. The ESOMAR
26 Questions to Help Research Buyersof Online Samples provides help on
how to assess the quality of an internet
panel sample.
13. I have seen polls conducted bydifferent, well-regarded, companies
on the same issue produce very different results. How come?
There are a number of possible reasons, beyond those issues related tosampling error.
a) The polls might have been conducted at different times, even if they
are published at the same time. If theviews of many people are fluid, andliable to change in response to events,
then it might be that both polls werebroadly right, and that the public moodshifted between the earlier and thelater survey.
b) The polls may have used different
definitions of the group that they
are representing (e.g. different age,

ESOMAR WORLD RESEARCH CODES & GUIDELINES

regions, ethnic groups etc.)c) They might have been conductedusing different methods. Results canbe subject to “mode effects”: that is,some people might, consciously or
sub-consciously, give different answers
depending on whether they are asked
questions in person by an interviewer,
or impersonally in self-completion
surveys sent by post or email/Internet.
There is some evidence thatanonymous self-completion surveys
may secure greater candour on somesensitive issues, than face-to-face or
telephone surveys.
d) The polls might have asked different
questions. Wording matters, especiallyon subjects where many people do nothave strong views. It is always worth
checking the exact wording when polls
appear to differ.
e) There might be an “order effect”.
One poll might ask a particular question“cold”, at the beginning of asurvey; another poll might ask thesame question “warm”, after a seriesof other questions on the same topic.
Differences sometimes arise between
the two sets of results, again whenmany people do not have strong views,
and some people may give different
answers depending on whether they
are asked a question out of the blue orafter being invited to consider some
aspects of the issue first.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

9

faq

Understanding opinion polls

14. Does the way the question is
asked influence the answers?
There is a great deal of knowledge
about how questions should be worded,based on what we know about how
people process information. But this isreally a matter of common sense. It is
important to look at the exact question
which was asked and, if possible, tocheck the questions asked before it.
Questions can contain concepts withinthem which lead the respondent ina certain direction e.g. “There seemto be fewer policemen on the streets
and a lot of people around here areconcerned about rising crime, do you
think the police in this area are overstretched?” or questions which containmore than one concept but where only
one answer is reported e.g. “How wellis the city council dealing with traffic
congestion and the lack of public transport?” reported as the level of concern
with public transport. Questions likethese will not provide clear or helpful
answers about what people really thinkof the police or public transport.
The context in which questions are
asked can obviously influence the way
in which people respond. If a question
about concern with crime is asked aftera series of questions about whether
people have ever felt nervous on public
transport or have a relative or friend
who has been mugged etc. it is likely

ESOMAR WORLD RESEARCH CODES & GUIDELINES

that more people will say they areconcerned than if this question had
been asked before the others.
When using answers to questions like
this, it is important to be aware thatthe questions were biased or ambiguous and therefore the answers cannot
be an accurate reflection of what thepeople answering them really believe.
This type of questioning is particularly
popular with pressure groups, who usethem to try and get media coverage for
their point of view. Responsible journalists and commentators should notreport those polls, or they should draw
attention to misleading questions whenreporting the results of opinion polls.
15. When I read or see a report of apoll, how can I tell whether to take itseriously or not?
Check the following:a) Who conducted the poll?
Was it a reputable, independent pollingorganisation? If not, then regard its findings with caution. If you are not sure, then
one test is its willingness to answer thequestions below. Reputable polling firms
will provide you with the information
you need to evaluate the survey.
b) Who paid for the poll and why
was it done?
If it was conducted for a respected
media outlet, or for independent

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

10

Understanding opinion polls

researchers, there is a good chance
it was conducted impartially. If it was
conducted for a partisan client, such as
a company, pressure group or politicalparty, it might still be a good survey(although readers/listeners/viewers
should be told who the client was). The
validity of the poll depends on whether
it was conducted by an organisationthat used a scientific approach to sampling and questionnaire design, whether
it asked impartial questions, and whether
full information about the questions asked
and results obtained are provided. If suchinformation is provided, then the quality ofthe survey stands or falls according to its
intrinsic merits. If such information is notprovided, then the poll should be treatedwith caution. In either event, watch out forloaded questions and selective findings,
designed to bolster the view of the client,rather than report public opinion fully
and objectively.
c) How many people wereinterviewed for the survey?
The more people, the better – althougha small-sample scientific survey is
ALWAYS better than a large-sample
self-selecting survey. Note, however,that the total sample size is not always
the only relevant number. For example,
voting intention surveys often show
figures excluding “don’t knows”,
respondents considered unlikely tovote, and those who refuse to disclose

ESOMAR WORLD RESEARCH CODES & GUIDELINES

their preference. While excluding thesegroups ensures that, the poll reports
the opinion of the most relevant group– “likely voters” – reported voting-intention sample size may be significantly
lower than the total sample, and therisk of sampling error therefore greater.
Likewise, be careful when comparing
sub-groups – for example men andwomen. The sampling error for each
sub-group could be significantly higher
than for the sample as a whole. If thetotal sample is 500, and made up ofequal numbers of men and women,
the margin of error for each gender(counting only random errors anddisregarding any systematic errors) isaround 6 percentage points.
d) How were those people chosen?
Is it clear who is included in the sample
and who was left out? If the poll claims
to represent the public as a whole (ora significant group of the public), has
the polling company employed one of
the methods outlined in points 2,3 and4 above? If the poll was self-selecting– such as readers of a newspaper ormagazine, or television viewers writing,telephoning, emailing or texting in –then it should NEVER be presented asa representative survey. If the poll was
conducted in certain locations but notothers, for example, cities but not rural
areas, then this information should bemade clear in any report.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

11

faq

Understanding opinion pollse) When was the poll done?
Events have a dramatic impact on poll
results. The interpretation of a pollshould depend on when it was conducted relative to key events. Even the
freshest poll results can be overtakenby events. Poll results that are several
weeks or months old may be perfectly
valid, for example, if they concern
underlying cultural attitudes or behaviour rather than topical events, but thedate when the poll was conducted (asdistinct from published) should always
be disclosed. The date of the fieldwork
is particularly important for pre-election
polls where voting intention can changeright up to the moment the voter
records their vote.
f) How were the interviews conducted?
There are four main methods: inperson, by telephone, online or by
mail. Each method has its strengths
and weaknesses. Telephone surveys
do not reach those who do not havetelephones. Email surveys reach only
those people with Internet access. All
methods depend on the availability andvoluntary co-operation of the respondents approached; response rates canvery widely. In all cases, reputablecompanies have developed statisticaltechniques to address these issues andconvert their raw data into representative results (see points 3 and 4 above).

ESOMAR WORLD RESEARCH CODES & GUIDELINESg) What were people asked?
Try to get a copy of the full questionnaire, not just the published questions.
A reputable organisation will publish
the questionnaire on its website, or
provide it on request. Decide if thequestions were balanced and be cautious about the results if the interview
was structured in a way which seemed
to lead the respondent towards aparticular conclusion.
h) Are the results in line withother polls?
If it is possible, check other polls to see
if the results are similar or very different. Surveys aiming to cover the same
topic should come to similar conclusions. If the answers are very different,
the reasons may become apparent
when the questionnaire or the sampling
method is examined.
i) Was it a “push poll”?
The purpose of “push polls” is tospread rumours and even outright lies
about opponents. These efforts are notpolls, but political manipulation trying
to hide behind the smokescreen of apublic opinion survey. In a “push poll,”a large number of people are called by
telephone and asked to participate in apurported survey. The survey “questions” are really thinly-veiled accusations against an opponent or repetitions of rumours about a candidate’sESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

12

Understanding opinion polls

personal or professional behaviour. The
focus here is on making certain the
respondent hears and understands theaccusation in the question, not in gathering the respondent’s opinions. “Pushpolls” have no connection with genuine
opinion surveys. The best way to guard
against “push polls” is to find out who
conducted the survey. Reputable companies have nothing to do with “pushpolls”, a phenomenon that has grown inrecent years in a number of countries.
j) Was it a valid exit poll?
This question applies only at elections.
Exit polls, properly conducted, are an
excellent source of information about
voters in a given election. They are theonly opportunity to survey actual votersand only voters. They are generally
conducted immediately after people
have voted, and are therefore able (intheory) to report actual behaviour. Preelection surveys, even those conducted
the day before the vote, cannot entirely
avoid the danger that some people may
change their mind, about whether tovote or which party/candidate to support, at the very last minute.
Properly conducted, they are an
excellent source of information about
voters in a given election. In addition toanswering the question “Who won?”they provide information to answer thequestions: Who voted for the winner

ESOMAR WORLD RESEARCH CODES & GUIDELINES

and why did candidate/party (a) or
candidate /party (b) win?. Exit polls
are characterised by an elaborate
design and a much higher numberof interviews than pre-election polls;
often tens of thousands and, in somecountries, hundreds of thousands areinterviewed.
Exit polls have four distinct sources oferror, apart from pure random error:• Supporters of one candidate/party may be more willing to disclose
their vote than supporters of another.
This phenomenon, “differential
nonresponse”,is especially hard tojudge accurately in exit polls.
• Some people may genuinely havethought they voted for a particular
candidate/party, but may inadvertently
have voted for someone else, or spoiled
their ballot paper or (when using voting
machines) not have completed theprocess properly.
• Exit polls may not have been

conducted on an absolutely representative group of polling stations. Even if
the total sample is very large – say,
5,000 or more – it may suffer from aneffect known as “clustering”. If, say, 50
polling stations are selected, and 100
voters questioned at each, the figures
could be wrong if the overall political

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

13

faq

Understanding opinion pollsbalance of those 50 polling districts iseven slightly askew.
• There may be operational difficulties

in sampling voters accurately at pollingplaces – either because of interferenceby local election officials or because oflegal restrictions on where interviewers
my stand. Reputable polling organisations go to considerable lengths
to avoid these problems. Other organisations may conduct exit polls ina minimal number of voting locations
using interviewers who do not haveexperience or specialist training in thismethod of polling.

ESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

14

How to publish opinion poll and survey results

3. Guideline for the
publication of opinion
poll and survey results
3.1. Introduction
1. Public opinion research – the studyof people’s attitudes and beliefs andbehaviours about political, social andother issues – forms a part of the total
market and social research field. It is
subject to exactly the same professional and ethical requirements asother forms of survey research. These
requirements are set out in the ICC/ESOMAR International Code on Market
and Social Research.
2. However, public opinion researchoften concerns an especially ‘sensitive’area. It deals with issues which arousegreater public interest and emotion
than do most market research projects.
In addition, its findings are much more
widely published and debated, and may
sometimes be presented in a provocative or even tendentious way. ESOMAR
has therefore set out specific recommendations about the publication ofsuch research.
3. Opinion polls and surveys have avaluable role to play in present-day
society. Because the data from polls
and surveys have far-reaching implications, it is important that the generalpublic, politicians, the media and other

ESOMAR WORLD RESEARCH CODES & GUIDELINES

interested groups should, throughresearch, have access to accurate andunbiased measures of public attitudes,
intentions and behaviours. We recognise concerns regarding the effectsof such poll data on voting or otherbehaviour. However, the alternative isthat the public would be exposed only
to unscientific and probably inaccurate
assertions about the situation, in manycases presented by individuals ororganisations who take an extremely
partisan or ideological approach to presenting the facts. The objective of thisGuideline is to reduce the risk of the
public being misled by research whichis inadequate or badly presented.
4. Recommendations in this Guideline
are based on the previous ESOMAR
Code for the Publication of Opinion
Polls. This was examined by the
Parliamentary Assembly of the Councilof Europe which then recommended
the widespread application of the Codeto govern the publication of polls. The
Code has since been combined with
updated guideline advice.
5. The validity and value of public
opinion surveys depend on three main
considerations:(i) the nature of the research techniques used and the efficiency withwhich they are applied,ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

15(ii) the honesty and objectivity of theresearcher carrying out the study,

be theoretically desirable and what ispracticable.

(iii) the way that the findings are presented and the uses to which they areput.

7. All reputable research organisations
apply the appropriate scientific methods
and operate with professional objectivity. In doing so they conform to the
ICC/ESOMAR International Code onMarket and Social Research. There isalso general agreement among themon the principles which should underlie the publication of research results.
However, normal professional practice
may vary between countries and incertain countries additional information
to that specified in this Code will also
customarily be provided as part of thestandard key material.

This section of the Guide concentrates
primarily on the second and third ofthese issues. Specific guidance ontechniques and the conduct of pre-election polls and exit polls is given in thefollowing two sections – 4 and 5.
6. Major misunderstandings can arise
when opinion poll or survey findings are published and debated. It is
unrealistic and unreasonable to expect
the media to quote the full technicalbackground of a survey when presenting its findings: they have limitations ofspace and must also hold the interestof their audience. However, there iscertain basic information which must
be provided if that audience is to havethe opportunity of judging for itself
the evidence presented and deciding whether or not it agrees with any
conclusions drawn from the research.
This Guideline is primarily concernedwith trying to ensure that the public hasreasonable access to this key information about the survey, and that published reports of the findings are notmisleading. The Guideline tries to strikea realistic balance between what would

ESOMAR WORLD RESEARCH CODES & GUIDELINES

Research organisations have a particular responsibility in the field of
public opinion polls, to make sure thatboth the client and the public have areasonable understanding of the special
problems and limitations involved inmeasuring attitudes and beliefs as
distinct from behaviour. Such researchfrequently deals with complex and sensitive issues about which respondents
have varying degrees of knowledge andinterest, and where their views may
often be half-formed, confused andinconsistent. High professional integrity
and skill is essential if the researchitself is to be unbiased and meaningful,and if the findings are to be presented

for researchersand commissioners

How to publish opinion poll and survey results

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

16

How to publish opinion poll and survey results

and interpreted clearly and accurately.
It is important also that the researchbudget available is sufficient to carryout a valid study. ESOMAR and WAPOR
fully recognise that such considerations
are vital if public opinion polls are to
merit public confidence and support.
8. If as a result of past experience, a
researcher has reason to believe thata particular client will not fairly present
opinion poll results in the publishedversion of the findings, the researcher
has a responsibility to stop carrying out
polls for publication by that client.
9. Finally, one particular type of work
which can be confused with an opinion
survey is a consultation exercise or
straw poll. In this type of study people
reply to questions in the same way as
in a sample survey, but the individuals
who answer the questions select themselves rather than being chosen as partof a scientific sample to represent a
particular universe or population. It isextremely important that the results ofthis type of work, which can provide
insight and engage the public in the
democratic process, are never reported
in a way which could lead to it being
confused with a representative surveyof opinion.

ESOMAR WORLD RESEARCH CODES & GUIDELINES

3.2. Scope
1. This Guideline must be read in
conjunction with other ESOMAR codes,
guidelines and principles available atwww.esomar.org
2. Market, social and opinion research
involve the gathering and further
processing of personal data, which isregulated by law in many countries. In
addition, certain countries have laws
regulating the publication of pre-election opinion poll results. This Guideline
does not provide a description of thelegislation in force around the world,and cannot replace the advice of legal
experts and self-regulatory bodies.
Rather it sets minimum standards of
ethical conduct to be followed by market research professionals and is to be
applied against the background of any
stricter standards that may be required
in a specific country.
3.3. Requirements of the ICC/ESOMAR International Code
3.3.1. General requirements
1. All researchers who conduct public
opinion polls and surveys must
conform to the ICC/ESOMAR International Code. Particular attention isdrawn to the requirements of Article
1d (concerning the clear separationof research from nonresearch activities), Article 3b (concerning protecting

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

respondents from harm or adverse
effects as a result of participating ina survey), Article 7 (concerning dataprotection and privacy) and Article 11(concerning interpretation, reportingand publication of results).
2. It is important to distinguish between
the requirements which apply to thereporting of public opinion poll results
by a researcher to the original client,and those which apply to the subsequent publishing of any poll findings bythat client to a wider audience. The firstof these situations is largely covered
by Article 11 of the International Codewhich specifies reporting requirements
in detail. This Guideline is intended
to clarify certain additional requirements which arise in connection withthe wider publication of the findings,
and therefore applies especially to thesecond situation.
3.3.2. Specific requirements for
published public opinion surveys
1. This section covers requirements for
all published public opinion surveys,
Section 4 covers specific and moredetailed requirements for surveys ofvoting intention.
2. When public opinion survey findings
are published in print media they must
always be accompanied by a clear
statement of:

ESOMAR WORLD RESEARCH CODES & GUIDELINES

17(a) the name of the research organisation carrying out the survey;(b) the universe effectively represented (i.e. who was interviewed);(c) the achieved sample size and its
geographical coverage;(d) the dates of fieldwork;(e) the sampling method used (andin the case of full random probability
samples the response rate achieved);(f) the method by which the information was collected (face-to-face,telephone interview, internet panel etc.);(g) whether weighting was used toadjust the results and the universe
used for the weights(h) the relevant questions asked. Inorder to avoid possible ambiguity theactual wording of the question should
be given unless this is a standard question already familiar to the audienceor it is given in a previously publishedreport to which reference is made.
3. “Achieved sample size” is thenumber of interviews actually reported.
“Geographical coverage” should state
which broad regions of the country(national or other such as urban areas
only or a specific geographical area)
were represented. Additionally the
number of sampling locations usedshould be given as an indication of theadequacy of the sample design. In referring to the number of “locations” theobjective is to provide a realistic picture

for researchersand commissioners

How to publish opinion poll and survey results

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

18

How to publish opinion poll and survey resultsof the extent to which the sample iswidely distributed geographically. The
best terms to use would vary by country – for example “Départements”
might be best in France, “Parliamentary
Constituencies” in the U.K.
4. It is important for the reader to begiven some general indication of thesampling method used since this may
in certain cases have very strong implications for the likely representativenessof the sample. A published report cannot necessarily provide all the relevant
data for a technical assessment; buteven a limited reference can be helpful.
In the case of probability sampling, themain objective is to identify those studies where an unusually low response
rate has been achieved, for whateverreasons. Although it is preferable
wherever possible to quote the actual
response rate, the main requirement
is therefore to indicate if the response
rate is below that regarded in theprofession as “normal” for that type ofstudy (this is a matter for experienced
professional judgement).
5. It is also important to indicate
whether the results quoted have beenadjusted by weighting proceduresor other statistical methods and it isrecommended that the raw data is
made available wherever the findings
reported differ substantially fromESOMAR WORLD RESEARCH CODES & GUIDELINES

the raw data collected in the field. (Thisrecommendation is especially relevantin the case of non-standard weightings – e.g. weighting applied to correctfor disproportionate sampling, likeover-representing particularly competitive states or constituencies to improve
efficiency, under-representing specificethnic or religious groups or to correct
for poor coverage of the population
when using online or telephone interviewing).
6. The guiding principle when deciding
which question wordings are relevant
to publish should be the elimination of
ambiguity and misunderstanding. Thisis particularly important where theactual wording of the question is criticalto the interpretation of the findings,
and where the reported answers canbe affected by the precise form of thequestion or its context – especially onpolitically or socially sensitive issues(for example, attitudes towards abortion). The description should therefore
help the reader understand exactly
what was asked. In some cases thiswill be sufficiently clear from the text
itself and the actual answers reported,
but when in doubt the question wordingused should be reported. Certainly
where tabular data are given, the fullquestion wording must be included.
Experience shows that it is in practiceoften possible to include the questions

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

19

without overloading the publishedreport. Where the company is able topublish information on its website, thefull question should be made available,
together with, as a minimum, theanswers for the weighted sample intotal. These answers should include“Don’t know and non-response”

A further example is:

7. Any recommendations for a standard
publication format in print must takeaccount of the different styles, layouts,
etc. of widely varying types of publication. One example of a suitable form of
wording would be:

8. There are certain specific situations
in which it is clearly difficult to follow all
the recommendations listed:

This survey was carried out by ABC
Research, on behalf of News Inc.,
using a national quota sample of1111 adults of voting age personallyinterviewed in 102 locations betweenstth
1 -5 March 2008.
Another alternative is to use a ‘databox’ of the kind:
Survey carried out by XYZ Research,
on behalf of News Inc. National
survey of 1234 adults aged 18 andabove, interviewed by telephone
between 25th-28th February 2008
using random digit dialling and aquota sample.

ESOMAR WORLD RESEARCH CODES & GUIDELINES

A national online survey of 2222
adults interviewed online on 5th
March 2008 by XYZ research andweighted to represent national voting
shares. Full details of survey availableat www.xyzresearch.”(i) where the survey reported on isvery extensive and complex andwhere the media report can therefore
provide only a relatively brief overview
of the total survey(ii) where an article summarises theresults of a number of surveys, whenagain it would be too complicated togive all the key information for each of
the surveys referred to.
Also, where a given survey is reportedon ‘serially’ (for example in the courseof several consecutive issues of anewspaper) it might be unnecessary to
repeat all the technical details in every
issue.
These situations are exceptions. Most
published reports on public opinion
polls refer to much more limited studies
than these. Even in the more complex

for researchersand commissioners

How to publish opinion poll and survey results

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

20

How to publish opinion poll and survey results

cases it should frequently be possible
to give the key information required.
In all cases where the key information cannot be fully provided, the basic
principle of fair and informative reporting must be followed, and it should
be made clear how and where theserious enquirer can obtain fuller
details. The advent of the internet
makes it possible to publish a considerable amount of information about asurvey, including details of sampling,weighting and questionnaire whichmight be difficult to provide in the original publication or broadcast. ESOMAR
recommends that the survey company
publishes full details of all public polls
on a website and that the website’s
address is given to the public by the
media, so that people can check details
if they wish to.
9. In the case of broadcast media it
may not always be possible to giveinformation on all these points. As
a minimum, points (a) through (f) inparagraph 2 above must be covered inany broadcast reference to the findingsof a public opinion poll, preferably invisual (written) form where practical.
Some research organisations currently
arrange with their client to provide apress release at the same time as thebroadcast report on a survey. Such apress release can more easily include
some kind of fact sheet covering all the

ESOMAR WORLD RESEARCH CODES & GUIDELINES

basic pieces of information referred toin paragraph 2 above. The publicationof this type of information on the agencies or broadcaster’s website is also a
way of making fuller detail available.
10. The percentages of respondents
who give ‘don’t know’ answers (andin the case of voting-intention studies,of those who say they will not vote)must always be given where they arelikely to affect the interpretation of thefindings significantly. When comparing
the findings from different surveys,
any changes (other than minor ones)in these percentages must also be
indicated. There are many occasions inwhich the interpretation of particularfindings will be quite different if the levelof ‘don’t know’ answers is 5% or 50%.
In the case of voting-intention studies
the same consideration also applies to’will not vote’ answers. A researcher
must apply their experience and professional judgement in deciding when suchsituations arise. It may not be necessary
to include all the ‘don’t know’ percentages in any tables given, althoughwhere this is possible it is frequently
the best way of dealing with the issue.
It may be quite sufficient, for example,
to make a general comment such as:“the proportion of ‘don’t knows’ wasnever higher than 5%” - or to comment
specifically on those instances where
the proportion was much higher.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

11. Whatever information may be givenin the published report of the survey,
the publisher and/or the researchorganisation involved must be preparedon request to supply the information
described in paragraph 2 of this Guideline on the survey method, as required
by Article 11 of the ICC/ESOMAR Code.
Where the questions formed part of amore extensive or ‘omnibus’ survey,
this must be made clear to any enquirer, including a general indication of theplacement of the questions in the larger
questionnaire. There is no obligation
under the Code for further information
beyond this to be supplied – althoughorganisations will normally be prepared
to discuss their research methods inmore detail with bona fide enquirers.
3.3.3. Arrangements between the
Research Organisation and its Client
1. In order to ensure that the requirements of the ICC/ESOMAR Code arefollowed, and to avoid possible misunderstandings, the research organisationmust make clear in advance to its
client:(i) that the research organisation itself
is bound by the requirements of thegeneral International Code.
(ii) that subsequent wider publicationof the research findings should be inaccordance with this guideline.

ESOMAR WORLD RESEARCH CODES & GUIDELINES21

It is therefore the responsibility of
the research organisation to draw its
client’s attention to this Guideline on
Publication of Results and to use its
best endeavours to persuade the client
to follow the Code’s requirements.
2. The research organisation andthe client each have a responsibility in the public interest to ensurethat the published report on a public
opinion poll does not misrepresent or distort the survey data. For
example, misleading comments basedon non-significant differences must be
avoided. Special care must be taken to
ensure that any graphs or charts used
do not convey a misleading impression of the current survey’s results orof trends over time. It is also importantthat the reader or listener should be
able to clearly distinguish between thesurvey findings as such and any editorial or other comments based uponthese findings. Wherever feasible, theresearch organisation must approve inadvance the exact form and content ofpublication as required in Article 11b of
the ICC/ESOMAR Code.
3. In preparing material for publication,
journalists and others connected withthe media themselves normally follow
their own professional codes of practice and ethics. The present Guideline
is not intended in any way to substitutefor researchersand commissioners

How to publish opinion poll and survey results

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

22

How to publish opinion poll and survey resultsthese but rather to support them. (Inthis context, ‘published report’ covers
non-print as well as print media.)
Article 11a of the Code emphasises theimportance of distinguishing, as muchas possible, the results that emerge
directly from the questions asked, andany commentary/interpretation basedon these results. Although the dividing
line is not always a simple one todefine, in most cases the distinction
between ‘fact’ and ‘comment’ is inpractice a workable one.
Article 11d of the Code requires thatthe research institute must reserve theright to publish the total study and notjust the technical specifications in theevent of:• a shortened version of the publication

distorting the analysis of the results• an unforeseen and abridged versionof the publication• a publication which does not conformto the prior agreements
4. The research organisation cannot
normally be held responsible for any
subsequent use made of public opinion
poll results by people other than theoriginal client. It should however
be ready to immediately issue suchcomments or information as may benecessary to correct any cases of

ESOMAR WORLD RESEARCH CODES & GUIDELINES

misreporting or misuse of results whenthese are brought to its attention.
5. In the event that a client releases datafrom a survey which was not originallyintended for publication, this Guideline
will apply to it as if it had originally beencommissioned for publication.
6. ESOMAR strongly advises theuse of contracts between researchorganisations and their clients toensure adherence to the ICC/ESOMAR
Code (see ESOMAR Guideline on themutual rights and responsibilities ofresearchers and clients). For example,
some contracts stipulate that theagency has the right to examine andapprove a copy of the publication basedon its research. Where the agency
reserves the copyright of the findings
this can also help to reduce some ofthe problems involved in misleading‘secondary reporting’ of the findings byother people. In addition to any otherrequirements it is suggested that sucha contract should cover:• clarification of the point that thecontract binds both the survey sponsor
and the media involved, where theseare different parties• some measure of control by the
research organisation over the published form of the results including
figures and graphs

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

Certain contracts also provide that if
research findings commissioned for
publication are not in fact published,such findings can subsequently (after
a specified period of time) be releasedby the research organisation itself; oralternatively the organisation is free torepeat the survey for another client. It
is also increasingly common practicein certain countries for data tapes frompublic opinion surveys to be lodged
with appropriate archives for subsequent secondary research by academic
researchers and others. Such steps
can help to reduce the danger that polls
may be thought sometimes to be used
in a ‘manipulative’ way by less scrupulous clients.
7. Any code of practice in this area has
obvious limitations, in that researchers
can exercise only restricted control
over how their results are presented
in the media, and still less influence
over any comments and interpretations(sometimes misguided and tendentious) based on the findings. A code
therefore depends on spreading the useof ‘best practice’ and influencing media
clients to avoid misleading presentationof survey results. ESOMAR expects its
members to follow this Guideline withthis objective firmly in mind.

ESOMAR WORLD RESEARCH CODES & GUIDELINES

23

for researchersand commissioners

How to publish opinion poll and survey results

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

24

Best practice guidance for researchers

4. Specific guidance on
conducting pre-election
opinion polls and polls
on voting intention
4.1. Introduction
1. This section concentrates on the
conduct of pre-election polls and assumes that the requirements of the
ICC/ESOMAR Code described in thepreceding section will be met. At first
it may seem strange to have specificguidelines on pre-election polls since
they are just one particular type of political poll. While all opinion polls require
high technical standards, it is pre-election polls that feature most frequently
in the debate about polls, and which
are restricted in some countries. These
guidelines have two main objectives –to protect the interests of the voter in a
democracy and to protect the credibilityof market and opinion research.
2. The first objective of these guidelines is to ensure that polling organisations take all possible technical steps
to ensure that polls published close to
the vital decision point for voters are
an objective guide to the state of public
opinion and voting intentions. The
process of sampling cannot guarantee
highly precise measurement by every
single poll. Also, the measurementof stated intentions to vote is only an
estimate and cannot guarantee that all

ESOMAR WORLD RESEARCH CODES & GUIDELINES

electors will actually vote in line with
their earlier stated voting intentions.
People do change their mind, someeven in the second before marking theirvote on the ballot slip. Polling organisations have a responsibility to electors toensure that polls, especially those polls
published in the last few days of anelection campaign, provide reliable andobjective information.
3. The second objective of these guidelines is to protect the public reputation of market research using sample
surveys. Pre-election opinion polls
which are published in the final daysof an election campaign have a major
influence on this. While it is true that
opinion polls are a snapshot of intentions at a specific point of time, they
are inevitably seen as predictions of theelection result when they are published late in the campaign. In general,
pollsters have not effectively challenged
this use of polling data, partly because
the track record of the polls in “predicting” the result is good.
4. In some countries where the publication of polls in the final stages of a
campaign is restricted, polls based onnational samples are often conducted
on polling day, or the day before, forpublication within minutes of the closeof the polling stations. Also carrying out
exit polls (interviewing voters as they

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGEBest practice guidance for researchers

5. Pre-election polls are a very public
test of sampling theory and survey
research in action. Polls have a goodtrack record for accuracy but the occasional poll which appears to be wrong
gets extensive media coverage. “Polls
wrong” is news and always gets major
coverage. “Polls accurate” is a headlinethat is seldom written. ESOMAR hopesthat these guidelines will contribute to
the technical education of journalists
responsible for poll reporting. However,special care must still be taken by polling organisations to minimise the riskof “getting it wrong”.
4.2. The Guidelines
4.2.1. Timing of fieldwork
Anyone attempting to prepare a criticalcase about polls would certainly decidethat the date of publication of a poll is a
key piece of information. Regardless ofwhen the interviewing took place, thepublication date is the important fact injudging the contribution of the poll tothe electoral process.

ESOMAR WORLD RESEARCH CODES & GUIDELINES

Polling organisations must be responsible for ensuring that polls publishedin the very late stages of an election
are likely to be a fair representation of
public opinion as close as possible tothe end of the campaign.
Guideline Polling companies should try
to reduce the risk of “getting it wrong”
by minimising the time elapsed between fieldwork and publication. A pollis more likely to achieve a good representative sample if the fieldwork period
includes some time in the evening
and during week-ends when potential
electors in full-time employment areavailable for interview.
4.2.2. Sample size
The measurement of the share of thevote for a party is subject to the normal
statistical confidence limits for sample
surveys. Two factors affect the size ofthe confidence limit of any party share.
The first is the absolute level of support
for a party. The closer this gets to 50%,
the wider will be the confidence limit
around the share estimate. The second
is the size of the sample interviewed inorder to produce the estimate. In mostpre-election polls the size of the sample
is the more important factor.
In countries with a simple proportional
representation system, the pre-election
poll measurement of the share of thefor researchers

leave the polling station) has becomemuch more common. Such polls areeven more likely to be seen as prediction polls and the analysis of theirresults is often used to explain why theelection came out the way it did. Theiraccuracy is equally important to the
public image of market research.

25

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

26

Best practice guidance for researchers

vote to each party is a fair indicator of
the election outcome. In other electoral
systems this may not be the case.
However, the best that the polls can
do is estimate the share of the voteto each party at a national level. The
key statistic reported in the media is
the gap in share of vote between theleading parties, and the measurementof the gap has much larger confidence
limits than those for an individual
party’s share.
Polling organisations frequently represent the margin of error of their polls
as ± 3 percentage points. This may be
accurate for a single party but is rarely
accurate for the key media figure – thegap between leading parties. A poll that
produces a 95% confidence limit of ± 3
percentage points on the share for oneof the leading parties, could produce
an equivalent confidence limit of ± 5.7
percentage points on the gap between
the two leading parties.
Guideline Pre-election polls should
not have a sample of fewer than 1,000
respondents. In circumstances where
the gap between leading parties is
expected to be small, the sample size
should be larger and samples of 1,500
to 2,000 should be used.

ESOMAR WORLD RESEARCH CODES & GUIDELINES

4.2.3. Sample distribution
In European countries there are twomain approaches to selecting samples
for face-to-face interviews.
Method 1
Select sampling points in proportion to
the number of electors. Then, in eachselected area take an equal numberof interviews. For any given sample
size, the rule for samples using thismethodology should be to maximise
the number of sampling points andminimise the number of interviews
conducted at any one sampling point.
This implies minimising the numberof interviews conducted by any single
interviewer.
Guideline There is empirical evidence
that if the number of interviews per
sampling point exceeds 20, then theinfluence of increased interviewer
variation outweighs increased sample
size.
Method 2
Select sampling points with equal
probability but then to take a different
number of interviews in each area
proportional to the size of the electoratein the area.
Guideline Polls using this methodshould still aim to minimise the numberof interviews conducted by any single

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGEBest practice guidance for researchers

Guideline For both methods, thesample design should give priority
to maximising the number ofsampling points and minimising thenumber of interviews conducted byeach interviewer.
4.2.4. Telephone interviewing
In principle telephone surveys offer
high-quality, unclustered, random samples with fast completion of fieldwork.
In countries with limited telephoneownership or availability, ownership isfrequently correlated with voting intention, i.e. those who cannot be contactedby phone are more likely to support one
rather than another of the parties in theelection. This may also be the case forthose telephone owners who are notlisted in the telephone directory, which
is why random digit dialling is to bepreferred, or those whose telephones
may not be part of the sampling frame(e.g. mobile/cell phone households).
Guideline If telephone ownership is
not high and/or likely to produce an
unrepresentative sample, this methodof interview should not be used forpre-election polls, or should be supplemented with data collected using
another mode of interviewing. If telephone samples are used for polls, theESOMAR WORLD RESEARCH CODES & GUIDELINES

polling organisation must use methods
to correct for any under-representationof supporters of particular politicalparties. Simple demographic profile
adjustments may not be adequate.
4.2.5. Online Polls
Internet surveys are increasingly being
used for opinion polls. As with telephonesurveys, its viability as a method ofcarrying out pre-election polls depends
to a large extent on the accessibility ofa representative sample via the internet.
Given the spread of the internet, there are
a number of countries where this can bedone, if care is taken to include people
who do not have the possibility to use theInternet at home, but connect to the internet at work or in some other place. The
ESOMAR Guideline on Research Using theInternet contains additional guidance onrequirements including online sampling
and the use of access panels.
Guideline This method should not be
used for pre-election polling without
evidence that a representative sampleof electors can be contacted via theinternet. As with telephone surveys,
the polling organisation must correct
for any underrepresentation. Simple
demographic profile adjustments will
generally not be adequate. The organisation should provide information aboutsample coverage and the weighting
variables used to correct the data.

for researchersinterviewer and stick to the maximumof 20.

27

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

28

Best practice guidance for researchers

4.2.6. Weighting
In order to conduct fast surveys withlarge samples, most pre-election opinion polls based on face-to-face interviews will use quota sampling methods.
The application of simple demographic
weights to ensure an accurate sample
balance is standard good practice. If
some parts of the electorate have been
deliberately over-sampled, weighting
should be used to re-establish thecorrect balance (see section 4.2.11 foradditional information).
Guideline The demographic profile ofpre-election polls should be checked
for representativeness and, if necessary, weighting should be applied tocorrectly represent the electorate.
Polling companies should ensure thatthe population profile used is that ofelectors eligible to vote rather than the
more normal all adults profile used incommercial market research.
4.2.7. Adjustments
In some circumstances polling organisations may feel that the result of theirpre-election poll is not an accurate
guide to the likely outcome. The mostobvious example is where the voting
intention of those with a high likelihoodof casting their vote is different fromthe total sample.

ESOMAR WORLD RESEARCH CODES & GUIDELINES

In some countries it is common tomeasure self-reported voting at theprevious election and use this to adjust
the voting estimate from the current
survey. In Denmark and France this is
an important quality improving procedure. However, this approach does notwork as well in other countries.
Experience has shown that voting intention measurements in some countries
need adjustments in order to provide areliable guide to public opinion. In thesecountries the weighting or adjustment
strategy of the polling organisation
may be confidential to that company forcompetitive reasons. In such circumstances, where the adjustment is madevia a stable and systematic procedure,
the polling company may wish to withhold full details of its method.
Guideline Polling organisations should
not make any adjustments to the simple
poll result which cannot be documented
and defined in advance of seeing theresults. Any adjustments must be
capable of being repeated and justified. It is good practice for pre-election
polls to measure key variables suchas likelihood to vote and to consider
whether the simple poll result should
be adjusted. Where adjustments to theunweighted poll findings are made, thisshould be noted in the publication of
the poll findings.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGEBest practice guidance for researchers

Guideline Wherever possible, pre-election polls should measure reasonsfor party choice or attitudes on issues
or other aspects of the campaign.
4.2.9. Time series
Polling organisations use different
methodological designs that may
distinguish one firm from another. The
meaning of a final pre-election poll is
far easier to determine if it is the latestof a series of polls conducted by thesame organisation during the campaign.
Guideline The validity of the methods
used by an organisation can be judged
better if they produce a series of voting
intention estimates during the campaign. Any obvious biases will become
apparent by comparison with the published polls of other organisations.
4.2.10. Consistent design
The ability to compare a final pre-election poll with previous polls fromthe same organisation is weakened if
the organisation changes key aspects

ESOMAR WORLD RESEARCH CODES & GUIDELINES

of its methodology for the final poll. It
could be argued that there is a benefit ifpolling organisations improve the methodological quality of the design they use
for final polls. However, this reduces
the comparability of a series of polls
from the same polling organization, andsuggests that some campaign polls canbe of poorer quality. However, if polls
are to make a valid contribution to informing the political process, they mustall be of high quality. A two-tier quality
policy undermines the value of polls.
Guideline Polling organisations shouldattempt to keep key elements ofmethodology consistent throughout theelection campaign. This particularly
applies to sampling method, question
wording and the positioning of voting
intention questions. It does not apply
to sample size. Where polls are basedon eligible (registered) voters early inthe campaign and subsequently moveto samples of “likely” voters late in the
campaign, this adjustment should beclearly noted and distinguished.
4.2.11. Indicator variables
It is possible to interview a sample representative as far as age, sex and social
grade are concerned, but still have apolitically unrepresentative sample.
For example, section 4.2.1 refers to theissue of representing working electors
by ensuring that the fieldwork period

for researchers

4.2.8. Survey content
Pre-election opinion polls will havegreater political and social value if
they do not confine themselves only
to measuring voting intention but
also explore the reasons for party
choice and opinions on importantcampaign issues.

29

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

30

Best practice guidance for researchers

includes some evening and weekend
interviewing.
It is good practice in a pre-election poll
to include the collection of information
which is correlated with voting behaviour but which is not part of the quota
control. The item should have a known
distribution from alternative sources.
Variables such as religion or language
spoken may be possible indicator variables if they have not been used in thedesign of the survey quota controls.
Guideline Polling companies should
be encouraged to develop an indicatorvariable for sample weighting purposes. In the period between elections it ispossible to calibrate the electoral implication of an over or under representation of the indicator variable.
4.2.12. Unusual poll results
Probability theory suggests that onepoll in twenty may give results outside
the normal 95% confidence limits. A
polling organisation may therefore finditself with a pre-election poll that is outof line with all the campaign evidence
available up to that point. Usually there
will be little time available between getting the final result and writing the copy
for publication.
Guideline It is unacceptable tosuppress a pre-election poll that looks

ESOMAR WORLD RESEARCH CODES & GUIDELINES

out of line with earlier polls unless avalid technical reason why the poll iswrong has been established. It is also
unacceptable to make a “gut feeling”
adjustment. The poll should be
published with appropriate warning
about the unusual poll result. The odds
are 20:1 in favour of the poll being correct and voters do change their minds
even in the polling booth.

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGEBest practice guidance for researchers

5.1. Introduction
An increasingly popular component of
the television coverage of electionnights is the exit poll. These are notbased on nationally representative
samples of voters asked their voting
intention. Exit polls are polls of voters,interviewed after they have voted, andno later than Election Day. They may
include the interviewing before Election
Day of postal, absentee and other early
voters. In some countries Election Day
polls cannot be conducted at the pollingplace, but in most cases, interviewing
takes place at the polling location.
Exit polls can serve three different
functions that are not mutually exclusive:• predicting election results• describing patterns of voter support

for parties, candidates, and issues• supporting extensive academic

research efforts
The main difference between these
may be the speed with which the results are formulated and disseminated.
Exit polls used for projections should
be reported as soon as is practical after
the polls close. Any delay in dissemi-

ESOMAR WORLD RESEARCH CODES & GUIDELINES

nating the results will inevitably raisequestions about the legitimacy of theeffort, especially with regard to estimating the outcome of the election. If
analysis is the only purpose of the exit
poll, prompt release is less important.
In some countries, election laws
prohibit the publication of exit poll
data until after the polls have closed.
WAPOR and ESOMAR oppose regulation of the conduct and reporting ofpolls in principle. Nevertheless, statements about the outcome of an election
based on exit polls must not be published before all the polls in the contest
have closed. In national elections, thismeans polls relating to election resultsfor elections in smaller voting units canbe reported when all the polling places
have closed in those locations, rather
than waiting until all polling placesused for voting that day have closed.
Descriptive information other than voting behaviour may be published before
the polls have closed, unless this isprohibited by local legislation or codesof practice.
5.2. Specific requirements
for exit polls
Survey researchers in general andthose conducting exit polls in particularneed to follow certain broad principles
in conducting their research:

for researchers

5. Guideline on
conducting exit polls andelection result forecasts

31

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

32

Best practice guidance for researchers

1. Exit polls conducted for public
consumption must be impartial andnon partisan. Exit polls are designed
to collect data from voters and report
information on electoral outcomes.
They are not tools for partisan advocacy.
2. Methods must be transparent, public,
and well-documented. These goals canbe achieved by publicly describing themethods prior to conducting the exit
poll and by adhering to the standardsof minimal disclosure delineated in thisGuideline. It is also recommended thatwhen the exit poll is used for analysis,
the data set (without individual identifiers) along with appropriate survey
documentation be deposited in publicarchives and/or on websites for generalaccess.

ing. Limitations and weaknesses in thedesign of an exit poll, its execution, andthe results must be noted in all reports
and analyses. Results must be released
to the public and other interested
parties through the general media andsimultaneously made accessible to all.
5. The identity of respondents in exit
polls must be protected. Identifying
information (e.g. name, address, orother IDs) must not be maintained withthe voter-level records, and the data
set should not allow deductive disclosure of respondents’ identities. To limit
the chances of deductive disclosure,small-area geographic details such as
the specific polling place in which votes
were cast must not be revealed.

3. Researchers must adopt study
designs for their exit polls that are suitable for producing accurate and reliable
results and that follow specific procedural and technical standards stipulated
in this document.

6. Poll methods used must be thosegenerally accepted as good survey
practice and must be disclosed inadvance of the conduct of the exit
poll, as well as with any projection or
analysis or subsequent public releaseof the dataset.

4. When reporting results from exit
polls, researchers must observe the requirements for publication of surveys in
the ICC/ESOMAR Code and be careful
to keep their interpretations and statements fully consistent with the data.
Speculation and commentary should
not be labelled as data-based report-

5.2.1. Items for minimal disclosure
1. These items must be disclosed withany exit poll report or when any projection is made. Good practice would be todisclose as much of the methodology
in advance as possible, particularly
those items in italics, which must be
disclosed before Election Day.

ESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGEBest practice guidance for researchers

ESOMAR WORLD RESEARCH CODES & GUIDELINES• A description of the precision of
the findings, including estimatesof sampling error• Monitoring and validation
procedures (if any)
• Weighting procedures• Response rates (using one ofthe definitions in the AAPOR/		WAPOR “Standard Definitions: 		Final Dispositions of Case Codes andOutcome Rates for Surveys”) anditem non-response on vote questions.
• Any known non-response bias
• General description of how estimates
are made and the kinds of variables
that are being used, and whether
adjustments for non-response havebeen made• Known design effects
2. Political parties may sometimes
make claims about private data. These
claims also require documentation. Any
public statement referring to exit poll
results must abide by the disclosure
principles and requirements above.
3. Those conducting exit polls must
always use generally accepted statistical methods. However, there are anumber of good practices that apply
specifically to exit polls.
• Exit polls typically employ clusteringin their sample designs. Because of thepossibilities that various groups might

for researchers• Sponsor of the exit poll
• Name of the polling company or
principal researcher; prior experience(if any) in exit polling; and whether
the data collector has any businessor personal ties to political parties,
candidates, political organizations or
governmental bodies.
• Name of the organization responsible
for analysis and projections, if different.
• Number of interviews• Number of polling stations or
sampling points if this is not where
the sampling takes place• Sampling frame• Geographic dispersion and coverage• How sampling points are selected• Where and how interviews are		
conducted: at polling places, 		
in person at homes, by phone, by 		
self-administered questionnaire etc.
• Any legal limits on data collection 		that might affect polling accuracy (e.g.
minimum distance of interviewers
from the polling place)• Time of day of interviewing• Whether interviewers are part ofa permanent field staff or hired forthe occasion• How respondent anonymity is
guaranteed (paper questionnaires, etc.)• The interview schedule orquestionnaire and instructions• Which results are based on parts		
of the sample, rather than thewhole sample

33

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

34

Best practice guidance for researchersattempt to influence voters and/or exit
poll respondents, researchers are notexpected to disclose the actual sample
points or locations.
• Exit polls must collect information

across the whole of the polling day.
Probability sampling (or full census)
for interviews conducted at the pollingplace is the only acceptable selection
method. Quotas are not appropriate forsampling at the polling place.

smallest voting unit level or because
the sampling units do not coincide withvoting units.
5.2.2. Election projection methods
and their disclosure
1. Election projections can be made inother ways than by interviewing votersas they exit the polling place. Whilemost projections are based on interviews with voters after they have voted
at a polling place, other forecastingmodels may include:• A national exit poll must represent theentire country, with 95% of the target
population included in the samplingframe. If the sampling frame covers
less than 95% of the target population,
there must be an explanation for thatdecision.
• Researchers must keep in mind therelationship between small units forwhich the votes are tabulated andcan also serve as clusters for exit pollinterviews. One way to evaluate an exit
poll is to compare the actual election
results with the estimates derived
from the exit poll interviews for thesesame units. This comparison of small
unit accuracy, typically at the precinct
or polling place level, is one of thebest ways to understand the exit poll’s
success. But there are situations where
this will not be possible, either because
no tabulations are reported at the

ESOMAR WORLD RESEARCH CODES & GUIDELINES• interviews in person, by telephone,or by other means of communication
with voters after or before having casttheir votes• counts of official votes in a sample ofprecincts, often known as quick counts• a mix of methods
2. A projection is an estimate that leads
to a conclusion about the outcome ofan election in a jurisdiction such as anation, a state or a district. This may
occur in two different situations:• If the winner is based on the popular

vote for an office or a party, then aprojection of the division of that voteis a projection of the outcome in thejurisdiction.
• If the winner is based on the vote inmultiple jurisdictions, such as electionof a Parliament where votes are cast inESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGEBest practice guidance for researchers

The projection need not reach a conclusion about each sub-jurisdiction. It
need only reach a conclusion about thejurisdiction outcome.
The objective of any projection is aconclusion about an election for somejurisdiction. A sample of that jurisdiction must be adequate to reach an
unbiased conclusion with sufficient/appropriate confidence in the estimate.
A national projection typically requires
the coverage of the entire country, withat least 95% of the target population in
the sampling frame.
There will be times that a subset ofthe country will be used (for example,
only competitive districts). But if thesampling frame is used that includes
something less than the entire voting
population of a jurisdiction, thenthe researcher should define in adisclosure statement what is and is
not included in the sampling frame.
The researcher must also publish arationale to justify the researcher’sability to make an unbiased conclusion

ESOMAR WORLD RESEARCH CODES & GUIDELINES

about the election outcome based uponcollecting information from a subset of
all jurisdictions. Similar rules to thosefor the provision of basic information
apply. The following information should
be made available before election day:• Sponsor of the exit poll
• Name of the polling company or

principal researcher; prior experience(if any) in projections; and whether
the data collector has any businessor personal ties to political parties,
candidates, political organizations or
governmental bodies.
• Name of the organization responsible
for analysis and projections, if different.
• Source of the data used to make theprojections (i.e. interviews in person, by
telephone, or by other means of communication with voters after or before
having cast their votes).

for researchers

districts or of a President where votes
are accumulated based on victories
won in many jurisdictions, a conclusion
about which party has a plurality ofseats in the new Parliament or which
presidential candidate has a winning
number of votes is a national projection.

35

ESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

“Market research, which includes
social and opinion research, is thesystematic gathering and interpretationof information about individuals ororganisations using the statistical andanalytical methods and techniques of theapplied social sciences to gain insight or
support decision making.
The identity of respondents will not be
revealed to the user of the information
without explicit consent and no sales
approach will be made to them as adirect result of their having provided
information.”
Definition of market research contained in the ICC/ESOMAR International CodeESOMAR
Eurocenter 2
Barbara Strozzilaan 384
1083 HN Amsterdam
The Netherlands
Tel +31 20 664 2141
Fax +31 20 664 2922
E-mail professional.standards@esomar.org
www.esomar.org

ESOMAR WORLD RESEARCH CODES & GUIDELINESESOMAR/WAPOR GUIDE TO OPINION POLLS AND PUBLISHED SURVEYS

TO CONTENTS PAGE

ESOMAR is the world organisation for enabling better
research into markets, consumers and societies.
With 5000 members in over 100 countries, ESOMAR’s
aim is to promote the value of market and opinion
research in illuminating real issues and bringing about
effective decision-making.
To facilitate this ongoing dialogue, ESOMAR createsand manages a comprehensive programme ofindustry-specific and thematic events, publications andcommunications, as well as actively advocating selfregulation and the worldwide code of practice.

Founded in 1947, the World Association for Public Opinion
Research (WAPOR, www.wapor.org) is dedicated toadvancing the scientific study of public opinion.
Its members around the world work to maintain highstandards for collecting, analysing, and disseminating
public opinion data. Each year, the organisation holds
conferences and workshops involving academics,
practitioners, journalists, and policy-makers.
WAPOR sponsors the multidisciplinary research journal,
the International Journal of Public Opinion Research.

ESOMAR WORLD RESEARCH CODES & GUIDELINES