Second European Union minorities and discrimination survey : technical report.
Date: 2017-11-28
Institutions: European Union Agency for Fundamental Rights
Subjects: anti-discriminatory measure, crime against individuals, discrimination based on disability, discrimination on the basis of nationality, discrimination on the basis of sexual orientation, ethnic discrimination, fundamental rights, integration of migrants, job access, protection of minorities, psychological harassment, racial discrimination, rights of minorities, social survey

EU-MIDIS II

Second European Union Minorities 
and Discrimination Survey

Technical report

Europe Direct is a service to help you find answers 
to your questions about the European Union

Freephone number (*):
00 800 6 7 8 9 10 11

(*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

More information on the European Union is available on the internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2017

FRA – print: 

ISBN 978-92-9491-764-5 

doi:10.2811/546676 

TK-02-17-830-EN-C

FRA – web: 

ISBN 978-92-9491-763-8 

doi:10.2811/326094 

TK-02-17-830-EN-N

© European Union Agency for Fundamental Rights, 2017

Reproduction is authorised provided the source is acknowledged. For any use or reproduction of photos contained 
herein, permission must be sought directly from the copyright holder.

Printed by Imprimerie Centrale in Luxembourg

Printed on elemental chlorine-free bleached paper (ECF)

Second European Union Minorities 
and Discrimination Survey

Technical report

Acronyms

AAPOR 
ACS 
CAPI 
CCT 
CEH 
CIH 
CIS 
CS 
DEGURBA 
EFTA 
EU 
EU-MIDIS II 
Eurostat 
EU-SILC 
FE 
FRA 
HH 
ISCED 
ISTAT 
LAU 
LFS 
LGBT 
MIDIS 
NGO 
NSE 
NSI 
NUTS 
OECD 
PAPI 
PPS 
PSU 
TRAPD 

American Association for Public Opinion Research
Adaptive cluster sampling
Computer-assisted personal interviewing
Central coordination team
Confirmed eligible household
Confirmed ineligible household
Sociological Research Centre
Contact sheet(s)
Degree of urbanisation
European Free Trade Association
European Union
Second wave of the European Union Minorities and Discrimination Survey
Statistical office of the European Commission
European Union Statistics on Income and Living Conditions
Focused enumeration
European Union Agency for Fundamental Rights
Household
International Standard Classification of Education
Italian National Institute of Statistics
Local administrative unit
Labour Force Survey
Lesbian, gay, bisexual and transgender
Minorities and Discrimination Survey
Non-governmental organisation
National survey expert
National statistical institute
Nomenclature of territorial units for statistics
Organisation for Economic Co-operation and Development
Paper and pencil interview
Probability proportional to size
Primary sampling unit
Translation, review, adjudication, pre-test and documentation

3

Country and target group codes

Country

EU Member 
State

Country target 
group code

Target group

Spain

Austria

Belgium

Denmark

Finland
France

Estonia
Greece

Bulgaria
Cyprus
Czech Republic
Germany

AT – TUR
AT – SSAFR
BE – TUR
BE – NOAFR
BG – ROMA
CY – ASIA
CZ – ROMA
DE – TUR
DE – SSAFR
DK – TUR
DK – SSAFR
EE – RUSMIN
EL – ROMA
EL – SASIA
ES – ROMA 
ES – NOAFR
FI – SSAFR
FR – NOAFR
FR – SSAFR
HR – ROMA
HU – ROMA
IE – SSAFR
IT – SASIA
IT – NOAFR
IT – SSAFR
LT – RUSMIN
LU – SSAFR
LV – RUSMIN
MT – SSAFR
NL – TUR
NL – NOAFR
PL – RIMGR
PT – ROMA
PT – SSAFR
RO – ROMA
SE – TUR
SE – SSAFR
SI – RIMGR
Slovenia
SK – ROMA
Slovakia
United Kingdom UK – SASIA
UK – SSAFR

Lithuania
Luxembourg
Latvia
Malta
Netherlands

Croatia
Hungary
Ireland
Italy

Romania
Sweden

Poland
Portugal

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa
Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from North Africa
Roma
Immigrants and descendants of immigrants from Asia
Roma
Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa
Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa
Russian minority
Roma
Immigrants and descendants of immigrants from South Asia
Roma
Immigrants and descendants of immigrants from North Africa
Immigrants and descendants of immigrants from Sub-Saharan Africa
Immigrants and descendants of immigrants from North Africa
Immigrants and descendants of immigrants from Sub-Saharan Africa
Roma
Roma
Immigrants and descendants of immigrants from Sub-Saharan Africa
Immigrants and descendants of immigrants from South Asia
Immigrants and descendants of immigrants from North Africa
Immigrants and descendants of immigrants from Sub-Saharan Africa
Russian minority
Immigrants and descendants of immigrants from Sub-Saharan Africa
Russian minority
Immigrants and descendants of immigrants from Sub-Saharan Africa
Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from North Africa
Recent immigrants 
Roma
Immigrants and descendants of immigrants from Sub-Saharan Africa
Roma
Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa
Recent immigrants 
Roma
Immigrants and descendants of immigrants from South Asia
Immigrants and descendants of immigrants from Sub-Saharan Africa

AT

BE

BG
CY
CZ
DE

DK

EE
EL

ES

FI
FR

HR
HU
IE
IT

LT 
LU
LV
MT
NL

PL
PT

RO
SE

SI
SK
UK

4

Contents

ACRONYMS  ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������ 3

COUNTRY AND TARGET GROUP CODES  ��������������������������������������������������������������������������������������������������������������������������������� 4

INTRODUCTION  ����������������������������������������������������������������������������������������������������������������������������������������������������������������������� 7

Report structure  ....................................................................................................................................................................  9

1  DEVELOPING THE SURVEY  ��������������������������������������������������������������������������������������������������������������������������������������������  11

1.1.  Stakeholder and survey expert consultations  ......................................................................................................  11

1.2.  Cognitive pre-test study  ...........................................................................................................................................  12

1.3.  EU-MIDIS II target groups  ......................................................................................................................................... 14

1.4.  Project management  ................................................................................................................................................  15

2  DEVELOPMENT AND TRANSLATION OF FIELDWORK MATERIAL AND SURVEY TOOLS  �������������������������������������������  17

2.1.  Questionnaire development  .................................................................................................................................... 17

2.2.  Questionnaire translation  ........................................................................................................................................  20

Step 1. Translatability assessment  ..........................................................................................................................  21

Step 2.  Two translations prepared in parallel  ........................................................................................................  21

Step 3. Adjudication  ..................................................................................................................................................  21

Step 4. Adjudication meeting  ..................................................................................................................................  21

Step 5. Final proofreading  ........................................................................................................................................ 22

Step 6. Automated checks  ....................................................................................................................................... 22

2.3.  Development and translation of other survey material  ..................................................................................... 22

Respondent-friendly paper version questionnaires  ............................................................................................ 22

Showcards ................................................................................................................................................................... 22

Contact sheets  ........................................................................................................................................................... 22

Interviewer training manual  .................................................................................................................................... 23

Introductory letter and postcard  ............................................................................................................................. 23

3 

INTERVIEWER SELECTION AND TRAINING  �����������������������������������������������������������������������������������������������������������������  25

3.1.  Central project briefing  ............................................................................................................................................. 25

3.2. 

Interviewer selection and training  ......................................................................................................................... 25

4 

SAMPLING  ���������������������������������������������������������������������������������������������������������������������������������������������������������������������  29

4.1.  Target populations and sample requirements  .....................................................................................................  29

Immigrants and descendants of immigrants in EU-MIDIS II  ..............................................................................  30

Ethnic minorities in EU-MIDIS II  ............................................................................................................................... 33

4.2.  Sample sizes  ............................................................................................................................................................... 33

Roma target group  ...................................................................................................................................................  34

Immigrants and descendants of immigrants from North Africa  .......................................................................  36

Immigrants and descendants of immigrants from South Asia  .........................................................................  36

Immigrants and descendants of immigrants from Sub-Saharan Africa  ........................................................... 37

Recent immigrants  ...................................................................................................................................................  38

4.3.  Sample frames and mapping of available information for sampling  ..............................................................  38

4.4.  Overview of sampling methodologies used  ........................................................................................................  43

Coverage and efficiency of design  ........................................................................................................................  43

4.5.  Direct single-stage sampling (design 1)  ................................................................................................................  49

4.6.  Multi-stage area sampling (designs 2a, 2b, 2c)  ...................................................................................................  49

Oversampling concentration strata  .......................................................................................................................  50

Selection of addresses .............................................................................................................................................. 52

Random route  ............................................................................................................................................................ 52

Focused enumeration  ............................................................................................................................................... 55

5

Second European Union Minorities and Discrimination Survey – Technical report

Adaptive cluster sampling  ......................................................................................................................................  56

Stopping and dropping rules  .................................................................................................................................... 57

Sampling by referrals  ...............................................................................................................................................  58

Selection of dwelling units, households and individuals  ...................................................................................  58

Managing fieldwork targets  ...................................................................................................................................  58

4.7.  Location sampling (design 3)  ..................................................................................................................................  59

Regional mapping, selection of regions and sample allocation  .......................................................................  59

Listing centres in each region  .................................................................................................................................  59

Selection of locations for inclusion in fieldwork  .................................................................................................  60

4.8.  Non-probability sampling (design 4)  ..................................................................................................................... 61

5  PILOTING  ������������������������������������������������������������������������������������������������������������������������������������������������������������������������ 63

5.1.  Methodology  .............................................................................................................................................................. 63

5.2.  Key findings, recommendations and action taken as a result of the pilot  ......................................................  64

Sampling  ....................................................................................................................................................................  64

Gaining respondent cooperation  ...........................................................................................................................  66

Contact sheets  ..........................................................................................................................................................  66

Questionnaire  ............................................................................................................................................................  66

6 

FIELDWORK  �������������������������������������������������������������������������������������������������������������������������������������������������������������������  69

6.1.  Fieldwork dates and progress  ................................................................................................................................  69

6.2.  Contacting respondents – recruitment and interview format  ............................................................................ 71

6.3. 

Interview length  ........................................................................................................................................................ 73

6.4.  Fieldwork support material  ...................................................................................................................................... 75

6.5.  Fieldwork outcomes  .................................................................................................................................................  76

6.6.  Field force and fieldwork control  ...........................................................................................................................  78

6.7.  Data processing and data cleaning  ........................................................................................................................  80

7  WEIGHTING  �������������������������������������������������������������������������������������������������������������������������������������������������������������������  83

7.1.  Weighting procedures: clustered and unclustered samples (sample designs 1 and 2)  .................................  83

Design weights  .........................................................................................................................................................  83

PSU selection (clustered samples)  .........................................................................................................................  83

Non-response weights  ............................................................................................................................................  85

Post-stratification or calibration (individual and household) and final weights  ............................................  86

Trimming weights  .....................................................................................................................................................  87

7.2.  Weighting procedures – location sampling (sample design 3)  ..........................................................................  87

Calculations for locations  ........................................................................................................................................  88

Individual weight  ......................................................................................................................................................  88

Post-stratification weight  ........................................................................................................................................  89

7.3.  Weighting procedures – quota sampling (sample design 4)  .............................................................................  89

7.4.  Weighting efficiency  ................................................................................................................................................  89

8 

LESSONS LEARNED  �������������������������������������������������������������������������������������������������������������������������������������������������������� 91

ANNEX 1:  QUESTIONNAIRE FLOWCHART  ��������������������������������������������������������������������������������������������������������������������������� 95

ANNEX 2:  LOCAL CONTRACTORS FOR FIELDWORK ����������������������������������������������������������������������������������������������������������  96

ANNEX 3:  QUALITY ASSURANCE PLAN  �����������������������������������������������������������������������������������������������������������������������������  97

REFERENCES  ������������������������������������������������������������������������������������������������������������������������������������������������������������������������ 105

6

Introduction

The  European  Commission  developed  a  European 
Union (EU) Framework to guide national Roma integra-
tion strategies in April 2011, and in December 2013 the 
Council of the European Union provided detailed recom-
mendations for enhancing the effectiveness of national 
measures. In parallel, and for the first time, the legal 
framework of the EU’s main investment policy tool, the 
European Structural and Investment Funds, allocated 
€454 billion for 2014–2020, included a specific ex-ante 
conditionality for allocating funds under the thematic 
objective  on  social  inclusion  and  poverty.  There  is 
increasing awareness among EU institutions and rel-
evant stakeholders in individual Member States of the 
need for robust data to underpin policies and invest-
ments that aim to improve the situation and integration 
of Roma.1 In December 2013, the European Parliament 
urged Member States “to produce disaggregated data 
with the assistance of the European Union Agency for 
Fundamental Rights (FRA) and to develop, in coopera-
tion with the Commission, the baseline indicators and 
measurable targets that are essential for a robust moni-
toring system in order to ensure reliable feedback on 
the progress made in the implementation of the National 
Roma Integration Strategies and in improvement of the 
situation of Roma”.2 In 2016, the European Court of Audi-
tors issued a special report to assess the impact of EU 
policy initiatives and financial support on Roma inte-
gration.3 Recommendation 8 (b) of this report calls on 
the European Commission to encourage Member States 
to collect comprehensive statistical data on ethnicity 
within the next 2 years. It further suggests that Eurostat 
could include relevant questions in the EU Statistics on 
Income and Living Conditions (EU-SILC) and its Labour 
Force Survey (LFS). The European Commission, however, 
rejected this recommendation, commenting that col-
lecting statistical data on ethnicity through European 
statistical instruments is technically difficult, expensive 
and legally challenging in some countries.4 

FRA’s mandate is to provide the relevant institutions, 
bodies, offices and agencies of the EU and its Mem-
ber States with independent, evidence-based assis-
tance and expertise relating to fundamental rights. 
Data collection (including comparative data collection 
in the form of survey research) on the situation of fun-
damental rights in the EU provides the basis for FRA’s 
assistance and expertise. For several years, FRA has 
promoted the collection of data that 1) can be disaggre-
gated by ethnic origin; 2) are based on information about 
respondents’ self-identification; and 3) are collected on 

FRA and United Nations (2016).
European Commission (2014).
European Court of Auditors (2016).

1 
2 
3 
4  FRA (2016).

a voluntary basis and fully respecting EU and national 
data protection regulations. In this context, FRA regularly 
conducts surveys on members of ethnic, religious and 
other minorities, including the Second European Union 
Minorities and Discrimination Survey (EU-MIDIS II).5

Article 17 of the Racial Equality Directive requires Mem-
ber States, every 5 years, to communicate to the Euro-
pean Commission all the information necessary for the 
Commission to draw up a report for the European Parlia-
ment and the Council of the EU on the application of this 
directive. In doing so, the European Commission shall 
take FRA’s views into account.6 FRA has identified vari-
ous factors that prevent more effective implementation 
of legal provisions and other mechanisms designed to 
improve protection against discrimination. For exam-
ple, FRA collects existing official data related to racism, 
xenophobia and other intolerance in the field of racist 
and racism-related hate crimes, and has consistently 
shown that not all Member States collect and/or publish 
comprehensive official data on such incidents. The rea-
sons are complex. They reflect, among other things, the 
cultural and historical response to ‘racist’ and related 
crimes in the individual country. Where such data have 
been collected, divergent legal definitions that have 
determined the scope of data collection, the purpose 
for which data are intended, and diverse methodologies 
for data collection have all impeded direct comparisons. 
FRA undertakes scientific research and data collection 
on selected population groups, in line with EU policy 
priorities and where FRA research would have added 
value. FRA enriches methodological development with 
its rights-based approach to data collection and its 
expertise in comparative EU-wide survey design, given 
that Europe’s increasingly diverse populations present 
a challenge to traditional survey research. This means 
that we need to develop and adopt survey methods 
that capture the experiences of persons belonging to 
vulnerable groups, including those categorised as ‘hard 
to reach’. ‘Hard to reach’, ‘elusive’ or ‘hidden’ popula-
tions are socially disadvantaged and disenfranchised 
groups that are difficult to access, engage and retain 
in research cost efficiently and in large numbers.7 FRA 
is the organisation that surveys them most.8

FRA has launched several surveys to compensate for 
the absence of official data and to document the situ-
ation on manifestations of discrimination, racism and 
related intolerance more generally (beyond the specific 
field of hate crime). In 2015, it launched EU-MIDIS II to 

FRA and United Nations (2016). 

5 
6  Council of the European Union (2000), Art. 17 (2). 
7  See, for example, Bonevski, B. et al. (2014); Reichel, D. and 

Morales, L. (2017).

8  FRA and United Nations (2016). 

7

Second European Union Minorities and Discrimination Survey – Technical report

assess progress made since the first wave of the survey. 
The first EU-MIDIS survey was in 2008, and the first 
results appeared in 2009.9 It was the largest EU-wide 
survey, and the first of its kind. It interviewed a ran-
dom sample of 23,500 immigrants and minority ethnic 
groups, such as Roma, face to face concerning their 
experiences of discrimination, racist victimisation and 
awareness of rights.

This second wave of EU-MIDIS collected comparable 
data in all 28 EU Member States in 2015–2016, to assist EU 
institutions and policy makers in developing evidence-
based legal and policy responses to respect, protect 
and fulfil the rights of persons with immigrant or eth-
nic minority backgrounds, including Roma. The survey 
explores a range of issues concerning discrimination 
based on grounds prohibited by international human 
rights law, such as skin colour, ethnic origin, religion or 
religious belief, as well as respondents’ experience of 
harassment, hate-motivated violence and experiences 
of discriminatory profiling. Other thematic areas that 
EU-MIDIS II covers include rights awareness, civic and 
political participation, and experiences of corruption 
and inter-group relations. The survey also collected 
a number of relevant demographic characteristics at 
the level of both individuals and households, to facilitate 
analysis of immigrants and ethnic minorities’ housing, 
income and living conditions. The results of this work 
provide valuable evidence on the impact of EU and 
national social inclusion efforts. This helps policy mak-
ers develop targeted responses in the absence of other 
surveys that would enable us to compare fundamental 
rights outcomes for immigrants and ethnic minorities 
across the EU.

The overall objectives of EU-MIDIS II are 1) to collect 
comparable data in all 28 EU Member States to sup-
port the EU in protecting the fundamental rights of per-
sons with immigrant or ethnic minority backgrounds, 
including Roma; 2) to refine research methodologies 
for sampling and surveying hard-to-reach groups; 3) 
to populate core indicators for measuring progress in 
the implementation of the EU Framework for National 
Roma Integration Strategies and selected indicators on 
immigrant integration; 4) to enable analysis of trends 
over time where possible; and 5) to deliver results, 
analysis and FRA opinions that meet the needs of the 
survey’s key stakeholders.

To achieve these objectives, EU-MIDIS II interviewed 
25,500  persons  with  immigrant  or  ethnic  minority 
backgrounds across the EU, including Roma, in nine EU 
Member States. Experienced, trained interviewers inter-
viewed respondents from the selected target group(s) 
face to face. In each country, one to three groups were 
selected for the survey, and interviewees were chosen 

using random probability sampling methods and inter-
viewed using computer-assisted personal interviewing 
(CAPI), either at home or in locations that the target 
groups frequented. The main reference design in EU-
MIDIS  II  is  a  household-based  survey  of  individual 
persons. The data collection methodology used in EU-
MIDIS II built upon experience gained from carrying out 
FRA’s first survey on immigrants and ethnic minorities 
in 2008 (EU-MIDIS I), the Roma survey in 201110 and 
FRA’s other surveys collecting data on specific minority 
groups, such as Jewish people or lesbian, gay, bisexual 
and transgender (LGBT) persons, or issues, such as vio-
lence against women. Compared with earlier surveys, 
the set of questions was extended in EU-MIDIS II and 
it improved the coverage of the survey’s target groups 
through the use of refined sampling methodologies.

Following an EU-wide open call for tenders, FRA com-
missioned  Ipsos  MORI,  a  large  international  survey 
company based in the United Kingdom, to coordinate 
data collection in all 28 EU Member States under the 
supervision of FRA staff, who monitored compliance 
in accordance with strict quality control procedures. As 
an illustration, while Ipsos MORI organised the imple-
mentation of the fieldwork by subcontracting national 
research agencies in each country, FRA staff partici-
pated in interviewer training sessions and observed 
data collection activities in selected Member States.

In surveying hard-to-reach populations, EU-MIDIS II 
faced a number of challenges during its implementa-
tion. Unexpected events, including negative political 
rhetoric and measures with respect to the refugee crisis, 
and the terrorist attacks in Paris in 2015 and Brussels in 
2016, created difficulties during fieldwork, especially in 
those Member States where immigrants and descend-
ants of immigrants were surveyed. One of the main 
challenges when surveying hard-to-reach groups is the 
lack of, or deficiencies in, sampling frames – that is, lists 
or registers of persons or households that could be used 
for sampling purposes. A cross-country and/or cross-
cultural survey design introduces additional complexity 
for surveying immigrants and ethnic minorities, as the 
proportions of different ethnic groups and their overall 
composition might vary across the countries surveyed.

The fieldwork was commissioned in December 2014 
with a view to delivering all outputs within 16 months 
of the contract signature. However, it needed an addi-
tional 6 months to complete all the activities, because 
of the time needed to improve the sample designs and 
secure access to sampling frames from national sta-
tistical institutes (NSIs), the general complexity of the 
project design and associated survey tools, and a lower 
than expected concentration of the target population 
in some countries. The terrorist attacks in Paris and 

9  FRA (2009).

10  FRA (2013).

8

Brussels produced a more difficult and less safe envi-
ronment for carrying out the interviews in some areas. 
These issues resulted in a longer fieldwork period than 
had initially been planned.

Report structure

This technical report provides the relevant information 
needed to assess the quality and reliability of the EU-
MIDIS II survey data, as well as considerations for inter-
preting the survey results. Survey researchers often 
assess quality in terms of absence or presence of vari-
ous types of bias, which affect all surveys. The various 
sections of this technical report clearly identify poten-
tial sources of bias, stemming from survey design or 
decisions made when implementing the survey. Given 
the innovative nature of the survey and the advanced 
methods used in balancing costs with potential sources 
of bias, this report can also be used as a reference when 
developing other national or multinational surveys on 
immigrants and ethnic minorities. The following chap-
ters of the report cover the procedures used in the 
development and administration of the survey.

The first three chapters of this report describe and 
assess the various stages of developing the methodo-
logical design of the survey, and the approach to devel-
oping interview content (i.e. questionnaire, showcards 
and other data collection tools) and other interviewer 
material (Chapter 1); the translation process (Chap-
ter 2); and the approach to interviewer selection and 
training (Chapter 3).

In most countries where an address-based sampling 
approach was the primary sampling approach used, this 
was supplemented with a secondary sampling method: 
1) adaptive cluster sampling (ACS), or 2) focused enu-
meration (FE). Chapter 4 provides full sampling details 
for each country.

The survey was piloted before the main-stage fieldwork 
in all 28 Member States, to test the questionnaire, all 
fieldwork material and sampling approaches. Chap-
ter 5 discusses the methodology of the pilot and the 
resulting amendments to the fieldwork material and 
sampling. Chapter 6 describes the main-stage field-
work, with details concerning the achieved sample as 
well as fieldwork progress, procedures and outcomes. 
Chapter 7 explains the weighting applied to the data.

Introduction

9

1  

Developing the survey

The development of the EU-MIDIS II survey started in 
2013 with comprehensive research on existing inter-
national and national surveys among immigrants and 
ethnic minorities, as well as the development of the 
questionnaire.  FRA  hosted  expert  and  stakeholder 
consultations at its premises, involving representatives 
from EU institutions, international organisations, civil 
society and academia. In addition, a cognitive pre-test 
study was conducted in 2014 based on a draft question-
naire. This study was implemented in eight EU Member 
States to test selected questions and other fieldwork 
material needed for the full-scale survey. The results of 
the pre-test study informed the further development 
of the survey questionnaire, which was finalised in the 
beginning of 2015. In January 2015, FRA organised a sec-
ond expert consultation meeting to inform the develop-
ment of the full-scale survey, with the participation of 
17 survey experts.

1�1�  Stakeholder and survey 
expert consultations
FRA set out to ensure that the content of the survey 
meets the needs of policy makers at both EU and Mem-
ber State levels, and contributes to filling the gap in the 
availability of data concerning immigrants and ethnic 
minorities’ experiences of discrimination, racist victimi-
sation and related intolerance more generally (beyond 
the specific field of hate crime).

The first stakeholder meeting, in March 2014, gathered 21 
stakeholders representing EU institutions, international 
organisations, civil society and academia as follows:

 • European  Commission  –  Directorate-General  for 
Justice and Consumers, Unit C1 – Fundamental rights 
and rights of the child

 • European Parliament – Policy Department Citizens’ 

Rights and Constitutional Affairs

 • European Commission – Eurostat, Unit F2 – Population

 • Council of Europe

 − European Commission against Racism and Intol-

erance (ECRI)

 − Roma and Travellers Division, Committee of Ex-

perts on Roma and Travellers (MG-S-ROM)

 • Organisation  for  Security  and  Co-operation  in  Eu-
rope, Office for Democratic Institutions and Human 
Rights (OSCE-ODIHR)

 • EQUINET – European Network of Equality Bodies

 • ENAR – European Network against Racism

 • PICUM – Platform for International Cooperation on 

Undocumented Migrants

 • MPG – Migration Policy Group

 • ICMPD  –  International  Centre  for  Migration  Policy 

Development

 • CCIF – Collective Against Islamophobia in France

 • ERRC – European Roma Rights Centre

 • CRAN  –  Conseil  Représentatif  des  Associations 

Noires

 • Muslim Council of Britain – Research and Documen-

tation Centre

 • Office of the Greek Ombudsman

11

Second European Union Minorities and Discrimination Survey – Technical report

 • Centre  for  Equal  Opportunities  and  Opposition  to 

Racism

 • Centre 

for  Policy  Studies,  Central  European 

University

 • Centre of Migration Research, University of Warsaw.

The meeting focused on identifying thematic areas and 
issues relevant to the survey, and the target groups that 
should be covered in the research. The experts brought 
in their various perspectives and expertise to discus-
sions on the survey’s content and scope. The valuable 
contributions and issues raised during the meeting were 
highly beneficial for the development of EU-MIDIS II.

Immediately after that, there was an expert meet-
ing involving 13 survey research experts. This meet-
ing concentrated on practical challenges related to the 
survey, from questionnaire design to sampling and 
fieldwork methods. The list of leading specialists in 
the area of survey research who attended the meet-
ing included (in alphabetical order by surname) Aniko 
Bernat (TARKI Social Research Institute, Budapest), Gian 
Carlo Blangiardo (Department of Statistics and Quan-
titative Research, University of Milan), Agata Gorny 
(Centre of Migration Research, University of Warsaw), 
Inga Jasinskaja-Lahti (Department of Social Psychol-
ogy,  University  of  Helsinki),  Piotr  Juchno  (Eurostat, 
Unit F2 – Population), Martin Kroh (Institute for Social 
Sciences, Humboldt University Berlin), Mónica Méndez 
Lago (Sociological Research Centre (CIS), Madrid), Mar-
wan Muhammad (spokesman for the Collective Against 
Islamophobia in France (CCIF), Silke Schneider (GESIS – 
Leibniz Institute for the Social Sciences), Philipp Schnell 
(Institute for Urban and Regional Research, Austrian 
Academy of Sciences), Rainer Schnell (Institute of Soci-
ology, University of Duisburg Essen), Julia Szalai (Cen-
tre for Policy Studies, Central European University) and 
Vijay Verma (Department of Economics and Statistics, 
University of Siena).

In January 2015, FRA hosted another two-day expert 
meeting to discuss the questionnaire, and key method-
ological aspects of the survey such as sampling frames 
and sampling design. The meeting gathered 17 survey 
experts  representing  EU  institutions,  international 
organisations, civil society and academia, including (in 
alphabetical order) Iris Andriessen (The Netherlands 
Institute for Social Research), Johann Bacher (Johannes 
Kepler University, Linz), Aniko Bernat (TARKI Social 
Research Institute, Budapest), Gian Carlo Blangiardo 
(Department of Statistics and Quantitative Research, 
University of Milan-Bicocca), Han Entzinger (FRA Scien-
tific Committee), Claudia Ionela Grosu (National Roma 
Agency,  Romania),  Kenneth  Horvath  (University  of 
Education, Karlsruhe), Thomas Huddleston (Migration 
Policy Group, Brussels), Piotr Juchno (Eurostat, Unit F2 

– Population), Tadas Leoncikas (Eurofound), Thomas 
Liebig (International Migration Division, Directorate for 
Employment, Labour and Social Affairs, Organisation 
for Economic Co-operation and Development (OECD)), 
Agnieszka Litwińska (Eurostat, Unit F4 – Quality of Life), 
Farhad Mehran (independent expert, formerly Interna-
tional Labour Organization (ILO)), Mónica Méndez Lago 
(CIS, Madrid), Laura Morales (Department of Politics and 
International Relations, University of Leicester), Pat-
rick Simon (National Institute for Demographic Studies 
(INED), Paris) and Chris Skinner (Department of Statis-
tics, London School of Economics and Political Science).

The experts acknowledged the challenge of obtain-
ing a representative probability sample of the target 
populations, due to the general absence of sampling 
frames in most countries. They discussed a variety of 
alternative sampling methods, including focused enu-
meration, adaptive cluster sampling, location sampling 
(‘centre  sampling’),  telephone  name  sampling  and 
respondent-driven sampling (RDS). The use of focused 
enumeration, adaptive cluster sampling and location 
sampling were deemed feasible alternative sampling 
methods, but there was also scepticism about the use 
of respondent-driven sampling in the context and time 
frame of the project.

1�2�  Cognitive pre-test study

In preparation for the second EU-MIDIS survey, FRA 
published an open call for tender, as a result of which 
it commissioned Ipsos MORI in 2014 to cognitively pre-
test a questionnaire in eight EU Member States with 
the objectives to:

 • develop  and 
questionnaire;

improve 

the  EU-MIDIS  survey 

 • explore  the  feasibility  of  collecting  valid,  reliable 
and comparable information through a population-
based survey of selected immigrant and ethnic mi-
nority groups;

 • report  the  findings  to  develop  a  ‘best-practice’ 

model.

To meet these objectives, the pre-test used qualita-
tive, cognitive testing methods to investigate the ways 
in which respondents in different EU Member States 
understand key concepts related to their fundamen-
tal rights. More specifically, questions were tested to 
understand their fitness for purpose in terms of meas-
uring and describing respondents’ experiences of dis-
crimination and crime victimisation, and their living 
conditions. The eight EU Member States selected for 
the pre-test were Belgium, Croatia, Greece, Hungary, 
Italy, Malta, Romania and the United Kingdom.

12

Developing the survey 

The questions selected by FRA for pre-testing – the 
interview content – were first developed by FRA and 
then  further  developed  in  cooperation  with  Ipsos 
MORI’s central research team.

In the next step, FRA asked nine academic experts, 
specialised in questionnaire design and/or research with 
ethnic minority and migrant groups, to provide written 
feedback on the pre-test questionnaire. The experts 
included Barbara Herzog-Punzenberger (Federal Insti-
tute for Research in Education, BIFIE Austria), Kenneth 
Horvath (University of Education, Karlsruhe), Gijs van 
Houten (Eurofound), Mónica Méndez Lago (Centro de 
Investigaciones Sociológicas, Madrid), Alita Nandi (Insti-
tute for Social and Economic Research, University of 
Essex), Peter Schmidt (Justus Liebig University, Gießen), 
Ilona Tomova (Institute for Population and Human Stud-
ies, Bulgarian Academy of Sciences), Georgios Tsiaka-
los (Aristotle University, Thessaloniki) and Malina Voicu 
(GESIS – Leibniz Institute for the Social Sciences). Experts 
were asked to provide written feedback on selected 
questions using a structured, evaluative framework 
that asked them to consider each question in terms of:

(1) The content standard: Does the question ask the 

right thing? Will answers to this question measure 
what is intended?

(2) The cognitive standard: Will the question make 

sense to respondents? Will respondents be able to 
answer it accurately?

(3) The usability standard: Can this question be easily 
read aloud/administered in respondents’ homes? 11

Following their written feedback, the nine experts were 
invited to a pre-test Expert Panel Meeting, which was 
held in Vienna on 20–21 February 2014. The purpose of 
the meeting was to discuss the feedback from the aca-
demic experts, to develop and improve the EU-MIDIS II 
questionnaire ahead of its cognitive pre-testing in the 
field. Prior to the event, each expert was sent the draft 
pre-test questionnaire with comments, including issues 
to be discussed over the course of the two-day meeting.

The input of experts into the development of the inter-
view content was valuable: their expertise facilitated 
a broad-based, structured assessment of the interview 
content, which considered the coherence of each ques-
tion, as well as the likely challenges of use in the field, 
particularly when interviewing Roma.

Following the expert consultation, the central research 
team  of  Ipsos  MORI  summarised  the  feedback  to 

11  Groves, R. M., Fowler, F. J., Couper, M., Lepkowski, J. M., 

Singer, E., and Tourangeau, R. developed this framework. See 
Groves et al. (2009).

formulate recommendations for action to take on each 
survey question assessed by the experts.

In  preparation  for  the  cognitive  testing,  FRA  and 
Ipsos MORI developed a detailed probing approach to 
test respondents’ comprehension related to the sur-
vey  questions.  Tailored  probes  were  inserted  after 
each question to allow the respondents, after having 
answered the question, to explain in their own words 
the way they understood the question or certain terms 
used. After FRA’s approval the final version of the pre-
test questionnaire was translated into eight languages 
used in the pre-test.

As a part of the pre-test study, 280 cognitive inter-
views were conducted between January and July 2014 
with  people  from  diverse  backgrounds,  including 
Roma. While it was not a requirement to test every 
question in each interview, it was required that each 
interview cover a certain number of sections of the 
pre-test questionnaire. The final version of the pre-
test questionnaire included 173 questions, 15 of which 
varied across respondents.

The  pre-test  findings  showed  that  designing  ques-
tions to measure prevalence of discrimination, physi-
cal violence and harassment among immigrants and 
minorities is challenging, because one has to translate 
rather difficult legal concepts into easily understand-
able questions, which is not always straightforward. 
Moreover, respondents from the selected target groups 
are often not fluent in the national language of the 
relevant country, and may have low levels of educa-
tion and short periods of residency, which means they 
are often unfamiliar with the cultural context of terms 
used in the pre-test questionnaire and therefore have 
difficulties in understanding some specific questions.

The findings from the pre-test suggested that more 
emphasis should be placed on trying to address the 
first of these challenges (the translation of theoretical 
concepts into questions), while considerable work had 
already been accomplished on the second. The recom-
mendations from the cognitive pre-test study aimed 
to simplify questions to make them more accessible to 
the survey’s target audience, while at the same time 
ensuring that definitions remain accurate and questions 
deliver the detailed information required. The findings 
and recommendations of the cognitive pre-test pro-
vided a solid basis for the further revision of the survey 
questionnaire, with a view to finalising it for use in the 
full-scale survey in all 28 EU Member States.

Based on the results of the pre-test study and other 
inputs, including those received through the expert con-
sultations, FRA launched an open call for tender in 2014 
to select the contractor for the full-scale survey covering 
the 28 EU Member States. Many of the key parameters 

13

Second European Union Minorities and Discrimination Survey – Technical report

of the survey were fixed by FRA in the technical specifi-
cations of the call for tender, such as target groups to be 
surveyed per country and minimum net sample size(s).

 • recent immigrants;

 • Roma;

1�3�  EU-MIDIS II target groups

The target groups in EU-MIDIS II varied by country. 
Details are in Chapter 4 on sampling. They include:

 • immigrants  and  descendants  of  immigrants  from 

North Africa;

 • immigrants  and  descendants  of  immigrants  from 

Turkey;

 • immigrants  and  descendants  of  immigrants  from 

Sub-Saharan Africa;

 • immigrants  and  descendants  of  immigrants  from 

Asia and South Asia;

 • members of the Russian minority.

Figure 1.1 lists the target groups of EU-MIDIS II by country.

The selection of the target groups is based on a combi-
nation of several considerations. First, the EU-MIDIS II 
survey aimed at a comparative EU-wide survey design, 
which implies inclusion and coverage of all 28 EU Mem-
ber States. Second, some of the target groups in EU-
MIDIS II had to be the same as the groups interviewed 
in the first EU-MIDIS survey (2008) to allow for trend 
analy sis and comparison of results between survey 
waves. At the same time, new target groups are included 
in EU-MIDIS II to be able to respond to the increasing 
diversification of European societies and emerging vul-
nerabilities. Third, as in the first EU-MIDIS survey, FRA 
asked the competent equality bodies in each Member 

Figure 1�1: Overview of target groups in EU-MIDIS II by EU Member State

Africa Sub−Saharan

Africa

North

Asia and
South Asia

immigrants
Recent

Roma

Russian
minority

*

Turkey

AT
BE
BG
CY
CZ
DE
DK
EE
EL
ES
FI
FR
HR
HU
IE
IT
LT
LU
LV
MT
NL
PL
PT
RO
SE
SI
SK
UK

Note: 

* Cyprus covered only immigrants and descendants of immigrants from Asia, excluding those from South Asia.

Source:  FRA, EU-MIDIS II 2016

14

Developing the survey 

State to identify the most common grounds of discrimi-
nation and the persons/groups most vulnerable to or at 
risk of discriminatory treatment and criminal victimisa-
tion, also including potentially ‘racially’, ‘ethnically’ or 
‘religiously’ motivated discrimination and victimisation. 
Fourth, EU-MIDIS II takes into consideration the size of 
the immigrant and ethnic minority groups in each EU 
Member State, as well as the particular situation in each 
Member State with respect to its history of past and 
recent immigration and settlement. Furthermore, it was 
important to define the survey’s target population in 
a way that maximises the possibilities for comparative 
analysis and examining the results for the same group 
in different countries, to the extent possible. Finally, the 
resources available for the survey, in combination with 
the above criteria, also served to determine the number 
of groups to be interviewed in each EU Member State.

1�4�  Project management

FRA managed the survey in close cooperation with the 
contractor, Ipsos MORI. Ipsos MORI’s Central Coordina-
tion Team was responsible for the coordination and 
management of the implementation of the fieldwork 
in the 28 EU Member States. The Central Coordination 
Team was made up of professionals from the Ipsos 
MORI Social Research Institute (ISRI), all of whom have 
extensive experience in delivering large, multi-coun-
try studies. The Central Coordination Team was led by 
a project coordinator, who was supported by a project 
manager, a sampling director and a sampling manager. 
The Ipsos MORI team took the lead in all local (national) 
subcontractor liaison and data quality management to 
ensure that EU-MIDIS II was delivered with maximum 
consistency and quality standards across the surveyed 
countries. Two external academic experts supported 
the core team of Ipsos MORI. Peter Lynn (University of 
Essex) was the Senior Sampling and Weighting Expert 
and provided critical review of the sample and weight-
ing designs. Anthony Heath (University of Manches-
ter) was the Senior Survey Expert providing input into 
the questionnaire design and training material. Due to 
the large number of countries included in EU-MIDIS II, 
Ipsos MORI assigned four ‘hub’ coordination manag-
ers to facilitate communication between the Central 
Coordination Team and the country teams. Each hub 

manager was responsible for day-to-day correspond-
ence between Ipsos MORI and seven local contractors.

FRA worked closely with Ipsos MORI, building on FRA’s 
prior experiences with implementing EU-wide surveys 
on hard-to-reach groups. FRA was supported by the 
sampling  and  weighting  expert  Vijay  Verma  in  the 
capacity of a sub-contracted external expert. Follow-
ing the first expert meeting to discuss the development 
of EU-MIDIS II in March 2014, Vijay Verma supported 
FRA in the assessment of the contractor’s initial sam-
pling design and provided a critical evaluation of the 
overall sampling plan and the country-specific sampling 
approaches, together with a number of suggestions for 
their further improvement. His advice regarding further 
sampling-related information that should be collected 
was very helpful for the development and finalisation 
of the weighting procedure.

The national fieldwork partners (local subcontractors) 
in the 28 EU Member States comprised both local Ipsos 
MORI offices and third-party agencies from Ipsos MORI’s 
wider network. (For a list of agencies see Annex 2). In 
Ireland and the Netherlands, the local contractor origi-
nally selected for the project had to be changed in the 
course of the survey implementation. In Ireland, the 
original network partner informed the Ipsos central 
team that it would not have the capacity to conduct 
the fieldwork in accordance with the original timetable. 
An alternative partner had to be appointed. In the Neth-
erlands, concerns were raised about the quality of the 
work of the originally appointed subcontractor, during 
and after the pilot. Ipsos MORI decided to seek another 
supplier. It appointed Labyrinth to conduct EU-MIDIS II in 
the Netherlands in November 2015. In both cases, the 
appropriate documents were provided to FRA to allow 
a formal contract amendment to be made.

A quality assurance plan was agreed with Ipsos MORI 
at the beginning of the project. This outlined the pro-
cedures that would be used to monitor quality at all 
stages of the survey life cycle, and detailed how their 
achievement would be documented. The quality assur-
ance plan in Annex 3 of this report, and the relevant 
sections of the report, describe the quality assurance 
procedures relevant to various activities, such as sam-
pling, translations and interviewing.

15

2  

Development and translation  
of fieldwork material  
and survey tools

This chapter presents the main activities related to 
development and translation of the EU-MIDIS II survey 
questionnaire – the main data collection tool – and other 
fieldwork material. For the EU-MIDIS II survey, the fol-
lowing fieldwork materials were developed and used:

 • EU-MIDIS  II  survey  questionnaire,  which  was  im-
plemented  by  means  of  face-to-face  interviewing 
using  computer-assisted  in-person  interviewing 
(CAPI) in all the survey countries;

 • a  paper  questionnaire  in  the  national  language(s) 
used in the interview, for use in exceptional cases 
such  as  if  technical  problems  arose  with  the  CAPI 
system,  or  when  using  CAPI  was  not  possible  for 
other  reasons  –  mainly  security  concerns  in  some 
selected primary selection units, e.g. in Portugal;

 • paper  versions  of  the  questionnaire  in  selected 
languages  –  mostly  other  than  the  national  lan-
guage  –  to  facilitate  communication  between  the 
interviewer  and  the  respondent,  where  relevant 
and  possible;  these  translations  of  the  question-
naire  into  selected  languages  are  also  referred  to 
as respondent-friendly questionnaires;

 • showcards  for  interviewers  and  respondents  to 
use during the interview; for each question where 
showcards were used, the cards list the answer cat-
egories relevant for that question;

 • contact sheets for screening respondents for eligi-
bility to take part in the survey, and for fieldwork 
monitoring;

 • a training manual for interviewers;

 • an  introductory  letter  to  help  interviewers  intro-

duce the survey to potential respondents;

 • an information postcard about the survey.

2�1�  Questionnaire 

development

The EU-MIDIS II questionnaire was developed by FRA 
and finalised in close cooperation with Ipsos MORI. The 
questionnaire development took into account experi-
ences of FRA’s earlier surveys – most importantly the 
first EU-MIDIS survey and the 2011 Roma survey – as 
well as inputs collected in the expert consultations and 
the results of the pre-test. (For more details about the 
pre-test, see Chapter 1). The final review of the ques-
tionnaire was done based on the results of the pilot 
surveys in all 28 EU Member States. (For the details on 
the pilots see Chapter 5 – Piloting). The final English-lan-
guage questionnaire (master version used to produce 
the translated versions) is available on the FRA website.

The  final  questionnaire  consists  of  ten  sections, 
as  illustrated  in  Table  2.1.  Full  details  concern-
ing the questions asked can be found by consulting 
the survey questionnaire.

All respondents were asked the first eight sections, 
from Introduction to Socio-economic background. Some 
questions within these sections were relevant only for 
specific respondent groups such as Roma, or immi-
grants and descendants of immigrants, e.g. residence 
status or family reunification. Questions in the section 
on Location sampling information were asked only in 
countries where location sampling was applied. (For 

17

Second European Union Minorities and Discrimination Survey – Technical report

Table 2�1: Overview of EU-MIDIS II questionnaire structure

Section

Introduction 

Topics covered

 • Household information (household grid)
 • Child information grid
 • Housing and living standards

Rights awareness, 
perceptions and attitudes

 • Level of attachment to various areas, e.g. neighbourhood, city, country or EU
 • Self-identification dimensions
 • Prevalence of discrimination
 • Awareness  of  support  organisations,  equality  bodies,  existing  antidiscrimina-
tion legislation, recent antidiscrimination campaigns in the relevant country

 • Worry about being discriminated against when out in public
 • Avoidance behaviour

Employment 

 • Employment situation
 • Experiences of discrimination on any ground, and specifically related to ethnic 

Experience of 
discrimination in the 
following areas:
 • Health
 • Housing
 • Education
 • Other services

Corruption experiences
Police stops experiences

or immigrant background when looking for work and at work
 • Reporting of the last incident of discrimination to any organisation
 • Level of satisfaction with the way the complaint was handled
 • Reasons for not reporting an incident of discrimination

 • Subjective assessment of own health condition
 • Unmet medical care needs
 • Highest level of education attained
 • Discrimination experiences while using health care services, when trying to rent 
or buy an apartment or a house, or when in contact with school authorities
 • Discrimination  experiences  while  using  various  other  services  such  as  enter-
ing a bar or a restaurant, a shop; at administrative offices or public services; in 
public transport

 • Reporting of the last incident of discrimination to any organisation
 • Level of satisfaction with the way the complaint was handled
 • Reasons for not reporting an incident of discrimination
 • Awareness of discrimination experiences among friends and family
 • Expectations to pay a bribe
 • The governmental official involved
 • Police stops experience in different situations
 • Reasons for being stopped
 • Level of police respectfulness
 • Prevalence of physical assault by the police
 • Reasons for not reporting an incident of physical assault by the police

Victimisation: experiences 
of harassment 
and violence

 • Prevalence of harassment and victimisation incidents
 • Characteristics  of  the  last  incident:  forms,  frequency,  perpetrators,  reporting, 
and reasons for non-reporting, satisfaction with handling of complaint by police

Societal participation

Socio-economic 
background

 • Residence status, family reunification
 • Application for country citizenship
 • Migration and mobility
 • Level of religiosity, wearing religious symbols
 • Political and civic participation
 • Group relations, collective identities

 • Marital status
 • Household income
 • Support received by the household
 • Monetary remittances
 • Making ends meet
 • Household possessions
 • Prevalence of household members going to bed hungry

18

Development and translation of fieldwork material and survey tools   

Section

Topics covered

Location sampling 
information

Interviewer questionnaire

Frequency of visiting various locations in the city/town/village 

Interviewer’s observations concerning the setting of the interview 
(e.g. presence of other people, language of the interview and 
respondent’s fluency, interest in the topics of the interview) 

Source:  FRA, EU-MIDIS II 2016

more details concerning location sampling see Chap-
ter 4 – Sampling). The interviewer filled in an inter-
viewer questionnaire after each completed interview. 
It includes the interviewer’s assessment of the charac-
teristics of the interview.

Some of the answer categories and terms in the ques-
tionnaire were tailored either according to the target 
group or country. A generic term ‘ethnic or immigrant 
background’ was used throughout the questionnaire 
to  address  different  target  groups.  It  was  changed 
to  ‘Roma’  when  interviewing  self-identified  Roma 
respondents in Bulgaria, Croatia, the Czech Republic, 
Hungary,  Greece,  Portugal,  Romania,  Slovakia  and 
Spain. When the interview involved Russian respond-
ents in Estonia, Latvia and Lithuania, it was replaced 
with ‘ethnic minority background’. The interviews with 
immigrants and descendants of immigrants used ‘ethnic 
or immigrant background’.

Furthermore, specific answer categories were tailored 
to reflect the categories relevant to each country. Exam-
ples include questions concerning the level of educa-
tion achieved, income, ability to meet unexpected but 
necessary expenses, and references to the national 
equality bodies. The contractor sought expert advice 
during the development of the questionnaire about how 
to measure education across countries.

The  International  Standard  Classification  of  Educa-
tion (ISCED) 201112 categories were used to record the 
highest level of education gained by respondents. The 
National Survey Experts provided the country-specific 
categories for use in their respective countries, along 
with an explanation of how to map these categories 
back into the harmonised ISCED coding frame to be used 
for the comparative analysis. Immigrants and descend-
ants of immigrants were also asked about the highest 
level of education they had achieved in a country other 
than the one in which they currently reside. Given the 
potentially large number of countries where people may 
have obtained their qualifications, it was not possible 
to provide country-specific categories for every pos-
sible country, including immigrants’ countries of ori-
gin, migration and residence. Generic categories were 
developed, based on ISCED 2011 categories, to capture 

the highest level of education that immigrants might 
have obtained outside the survey country.

Standardised income bands were used across all coun-
tries. The original coding frame was taken from the 
questionnaire used for the fifth wave of the European 
Working Conditions Survey, which Eurofound conducted 
in 2010.13 This was Eurofound’s most recently published 
European Working Conditions Survey when the EU-
MIDIS II questionnaire was developed. To account for 
the lower levels of average income among immigrants 
and ethnic minorities in some EU Member States, two 
additional income bands were added to the lower end 
of the scale. The original scale was in euros, and for 
countries outside the Eurozone the exchange rate from 
the European Central Bank on 12 May 2015 was used 
to convert the income bands into national currencies. 
To allow comparability in the final dataset, informa-
tion that the respondents provided in national curren-
cies was converted back to euro equivalents using the 
same exchange rate.

For  the  question  measuring  whether  a  household 
could afford an unexpected but necessary expense 
the amount specified was set at 1/12 of the national 
at-risk-of-poverty  threshold  for  a  one-person 
household in 2013.14

To  measure  awareness  of  equality  bodies  in  each 
country, a list of up to three relevant bodies in each 
country was used. The list included only those bod-
ies with which an individual can lodge a complaint 
after experiencing discrimination.

A range of different CAPI software programmes were 
used for the data collection. In nine survey countries, 
the same software was used with the contractor script-
ing the questionnaire centrally. The master script was 
then overlaid with the translated versions of the ques-
tionnaire. In the other 19 survey countries, 11 different 
CAPI software programmes were used for scripting and 
data collection. In all of these cases, the questionnaire 
was scripted locally following the filtering and routing 
instructions provided in the master questionnaire, and 
the scripting was checked by Ipsos MORI.

12  For further information concerning ISCED see the 

Commission’s webpage on ISCED. 

13  Eurofound (2010).
14  Eurostat (2016). 

19

Second European Union Minorities and Discrimination Survey – Technical report

2�2�  Questionnaire translation

The English master version of the EU-MIDIS II survey 
questionnaire  was  translated  into  22  EU  languages 
as  well  as  into  Arabic,  Kurdish,  Russian,  Somali, 
Tamazight and Turkish. The translations largely fol-
lowed the scheme outlined in FRA’s technical speci-
fications for the survey. In addition to the language 
versions  originally  requested  by  FRA,  the  decision 
was made to translate the questionnaire into Kurdish, 
based on feedback from local contractors interview-
ing immigrants and descendants of immigrants from 
Turkey. Furthermore, a translation into Tamazight was 
produced for use in the Netherlands, based on pilot 
results indicating that a large proportion of the North 
African target group spoke this language. Table 2.2 at 
the end of this chapter presents an overview of the 
various survey tools and the translations produced. 
The questionnaire translation procedure followed the 
TRAPD (Translation, Review, Adjudication, Pre-test and 
Documentation) model as illustrated in Figure 2.1. Ipsos 
MORI subcontracted cApStAn – an agency specialised 
in translatability assessments – and cApStAn’s sister 
agency BranTra to translate the questionnaire and some 
of the non-EU language material.

A team of translators produced each language version 
of the questionnaire. The team consisted of two lin-
guists, who each produced independently an original 
translation of the source questionnaire (Translation 1 
and Translation 2), and one adjudicator responsible 
for merging and adjudication of the two translations. 
Translator 1 (T1) and the adjudicator were appointed 
by cApStAn while Translator 2 (T2) was appointed by 
the local fieldwork sub-contractor. Web-based training 
seminars were organised by BranTra/cApStAn both for 
the translators and for the adjudicators before the start 
of the translation activities.

In preparation for translation, FRA and Ipsos MORI pre-
pared a list of key terms to be used in the survey. The 
list included a glossary compiled by FRA with definitions 
of key terms and their translations in various languages. 

It also included the names of equality bodies referred 
to in the questionnaire, compulsory school age across 
the survey countries, and the name and definition of 
each target group.

The EU-MIDIS II questionnaire included three questions 
regarding respondents’ assessment of their overall 
health, which were taken from the European Health 
Interview Survey15 coordinated by Eurostat. Available 
translations were applied for the EU-MIDIS II survey, 
except in Spain, where the translations from the EU-SILC 
survey (Statistics on Income and Living Conditions)16 
were used. The three questions on overall health status 
are also referred to as the Minimum European Health 
Module, which Eurostat includes in relevant EU surveys.

The following convention to gender neutrality in multi-
lingual surveys was adopted for the translation of the 
EU-MIDIS II survey questionnaire. If gender was differ-
entiated in the source questionnaire (e.g. “he/she…” or 
“his/her…”), translators and adjudicator were instructed 
to also differentiate if applicable in the relevant lan-
guage, and to insert a comment if local usage deviates 
from the master version in this respect. If gender was 
not differentiated in the source questionnaire but two 
forms were possible or necessary in the translated ver-
sion (e.g. in “are you satisfied?” – in some languages 
“are” + “you” + “satisfied” might each have a masculine 
and a feminine form), translators and adjudicators were 
instructed to adopt the most neutral/common form and 
to avoid double gender, slashes and repetition of words, 
and to include instructions to the interviewers to use 
the appropriate form when addressing the respondent.

The same questionnaire translation process was applied 
for each of the languages, except for Tamazight, for 
which the initial translation was undertaken by an expe-
rienced linguist and then verified by two independent 
linguists.17 All three translators agreed that they should 
use common vocabulary that a range of Berber-speak-
ing respondents could understand, regardless of which 
country they are from. The questionnaire translation 
process for the other languages proceeded as follows.

20

15  Eurostat (2017a).
16  Eurostat (2017b).
17  The need for including Tamazight was raised later and 
therefore a different procedure had to be followed.

Development and translation of fieldwork material and survey tools   

Figure 2�1: Illustration of TRAPD (Translation, Review, Adjudication, Pre-test and Documentation) translation 

procedure used in translating the EU-MIDIS II questionnaire

EU-MIDIS II survey master
questionnaire

cApStAn evaluates the master questionnaire,
reviews it and prepares a version for translation

Step 1
Translatability assessment

cApStAn and Ipsos MORI translate
the same text independently

Step 2
Translations T1 and T2

Adjudicator merges T1 and T2 into
a version with best elements from each

Step 3
Adjudication

Adjudicated version is discussed between translators
and adjudicator, and ﬁnal adjudicated version is prepared

Step 4
Adjudication meeting

Proofreader checks
remaining issues

cApStAn runs automated consistency checks
and delivers  ﬁnal translated questionnaire

Step 5
Final proofreading

Step 6
Automated checks

Source:  FRA, EU-MIDIS II 2016

Step 1� Translatability assessment

Prior to the full translation, a translatability assess-
ment  was  undertaken  based  on  the  master  ques-
tionnaire.  A  translatability  assessment  contributes 
to making the source material fit for translation and 
may raise awareness of potential hurdles that should 
be  taken  into  account  when  finalising  the  master 
questionnaire, in view of adaptations that would be 
needed to make translating the questions into certain 
languages more accurate.

A pool of experienced linguists, representing various 
language groups, received a selection of survey ques-
tions from the master questionnaire. They assessed the 
questions for Slavic languages (Croatian, Czech, Polish 
and Russian), Germanic languages (Danish, German and 
Swedish) and Romance languages (French, Italian and 
Romanian), as well as for Greek and Latvian. They also 
assessed a sample of questions in Arabic and Turkish. 
The linguists produced advance translations to identify 
issues that would confront translators. The results were 
presented in a translatability report, in which special 
attention was given to items for which at least two 
linguists identified problems, and to issues that were 
likely to apply to other languages, providing sugges-
tions for alternate wording or notes for translators to 
circumvent the detected issue.

Both during translation and adjudication/adaptation, 
BranTra/cApStAn set up a helpdesk which provided 
advice  and  answered  questions  and  queries  from 
translators and adjudicators.

Step 2�  Two translations prepared 

in parallel

As a general rule, one translator (T1) translated the 
entire text, while the other translator (T2) translated 
only segments selected for parallel translation.

Step 3� Adjudication

The two translations were collated and then provided to 
the adjudicator whose task was to produce a reconciled 
version while ensuring consistency in the use of terms 
across the questionnaire. The adjudicator’s comments 
and problems faced when having to reconcile between 
the translations served as a basis for discussion during 
the adjudication and/or team review meeting.

Step 4� Adjudication meeting

The meeting brought together the translation team 
members to discuss the outcome of the translation 
and adjudication process, issues raised, and solutions 
proposed for the final review. Each translation team dis-
cussed issues earmarked for discussion one by one, and 
the adjudicator took the outcomes of the adjudication 

21

Second European Union Minorities and Discrimination Survey – Technical report

meeting into account when preparing the final version 
of the translation.

Some language versions were shared by two or more 
countries in the survey  – for example, the questionnaire 
in French was used in France, Belgium and Luxembourg. 
In this case teams in each country appointed adaptors 
who assessed the translation to highlight cases where 
the translation needed to be adjusted to account for 
local dialect or typical terms used which may differ 
between countries. For language versions that were 
shared by two or more countries in the survey, the 
adaptors were also invited to the team review meeting. 
Their role was to contribute to the discussion, to point 
out differences for their adapted version (e.g. different 
German terms or expressions used in Austria and Ger-
many), but also to ensure that remaining errors spotted 
by each adaptor would be corrected in all versions of 
that language, if applicable.

Step 5� Final proofreading

The proofreader’s role was to check that the final ver-
sion of the translation was correct, without reference 
to the master questionnaire. This final step covered 
spelling, grammar, syntax and completeness. The proof-
reader was also to double-check that the translated 
version had corrected errata.

Step 6� Automated checks

Using  automated  quality  assurance  routines,  the 
proofread  translated  version  of  the  questionnaire 
was double-checked with regard to completeness of 
the translation (such as checking that the number of 
answer categories is the same in the master question-
naire and the translation) and consistency in translation 
of agreed key terms.

The translation process was documented and archived 
using a centralised monitoring tool which reflected each 
step of the process. FRA was able to monitor the pro-
cess and its progress.

In a final step, FRA staff members reviewed the lan-
guage versions of the questionnaire for those languages 
in  which  they  were  fluent.  Overall,  FRA’s  in-house 
experts were able to proofread most translations.

2�3�  Development and 

translation of other 
survey material

Besides the questionnaire, a number of EU-MIDIS II 
documents were produced to aid the implementation of 
the survey. Ipsos MORI provided the first draft of these 

fieldwork documents, and FRA provided comments and 
signed off on the final versions. All interviewer and 
respondent related material were translated into the 
relevant languages used in administering the survey. 
Some documents were only required in the 22 national 
languages as they were only to be used by the inter-
viewers (e.g. interviewer instructions, contact sheets or 
training manual) while other fieldwork documents were 
required in additional languages as they were meant for 
the respondents with the aim of increasing participa-
tion (e.g. introductory letters) or aid completion of the 
interview (e.g. showcards). For this material a simplified 
approach to translation was adopted consisting of single 
translation and proof reading by two separate linguists. 
All translated survey material were approved by FRA 
before the mainstage fieldwork.

Respondent-friendly paper version 
questionnaires

The questionnaire was translated for CAPI scripting into 
22 EU languages plus Russian, Somali, Tamazight and 
Turkish. CAPI scripting was not available in Arabic and 
Kurdish – instead a paper questionnaire was prepared 
in these two languages. In addition to CAPI scripting, in 
some countries and languages a so called respondent-
friendly paper version of the questionnaire was pre-
pared (see Table 2.2). These versions were modified to 
allow the respondents to follow the questions as the 
interviewers read them in the national or another sur-
vey languages. This was developed as a way of assisting 
respondents who may have problems in understanding 
the survey questions in the language used in the inter-
views. Interviewer instructions and routing instructions 
as well as all answer categories which were not meant 
to be read out by the interviewer had been removed 
from this paper questionnaire to avoid confusion.

Showcards

Showcards  were  based  on  the  approved  question-
naire translations. For some questions the order of the 
codes on the showcards were presented in standard 
(e.g. answer categories listed from A to E) and reverse 
order (e.g. from E to A) to help mitigate any effect 
which the order of the categories may introduce into 
respondents answers. Each pack of showcards included 
only one version – either standard or reverse – and 
interviewers were given one or the other to use for all 
of their interviews.

Contact sheets

The use of contact sheets was a vital part of the sam-
pling procedure in those countries where sampling 
could not be based on a national population register of 
individuals. The interviewers used the contact sheet 
to collect the necessary information when applying 

22

Development and translation of fieldwork material and survey tools   

different sampling methods across the survey countries. 
The contact sheets were country-tailored and adapted 
with  regard  to  the  target  group  and  the  sampling 
method(s) applied in each country. For example, ques-
tions in the contact sheet related to self-identification of 
respondents were adapted to take into account whether 
the interviewer was screening for Roma or Russian 
minority respondents, as opposed to immigrants and 
descendants of immigrants. Interviewers also recorded 
details concerning the sampling approaches used, such 
as using address registers, random walk, location sam-
pling or other methods, as instructed by the fieldwork 
management team. In total, 13 versions of the con-
tact sheet were developed and used in the fieldwork 
in different countries.

Interviewer training manual

The training manual was the main reference for the 
interviewer training session, and it was designed as 
a resource for the interviewers during fieldwork. The 
manual covered a wide range of topics, such as:

 • background and aims of the EU-MIDIS II survey;

 • sampling  method  for  each  country  and/or  target 

group;

 • how to make contact with and select respondents 

for interview;

 • advice on how to introduce the survey and maxim-

ise response rates;

 • administration of the survey material;

 • ethical, cultural and safety considerations.

Country versions of the training manual were tailored 
as necessary, in accordance with the sampling method 
implemented and target group(s) selected.

Introductory letter and postcard

Following  consultations  with  the  National  Survey 
Experts and feedback from the pilot, the introductory 
letter with information about the survey was translated 
into six further languages (Hindi, Tagalog, Tamazight, 
Tigrinya, Urdu, and Vietnamese) in addition to the other 
languages being used to administer the survey. A post-
card with information about the survey was available 
in all national languages. Both the introductory letter 
and the postcard were shared with the respondents 
to promote the importance of the survey and ensure 
positive contact with potential respondents. Both letter 
and postcard also provided contact details in case of any 
queries about the survey.

23

Second European Union Minorities and Discrimination Survey – Technical report

Table 2�2: Overview of EU-MIDIS II survey material and translations produced

Country

Questionnaire 
CAPI script and 
showcards

Respondent-friendly 
paper version of 
the questionnaire

German, 
Turkish

Somali, English, 
Turkish, French

French, Dutch

Arabic, Turkish, Kurdish French, Dutch

Interviewer 
training manual 
and contact 
sheets

German

Introductory letter

Postcard

German, Turkish, 
Somali

French, Dutch, 
Arabic, Turkish

German

French, Dutch

Bulgarian

Bulgarian

Bulgarian

Bulgarian

Greek, English, Tagalog Greek

Greek, English

Greek, English

Czech

German

French, Turkish, 
Kurdish, English

Danish, Turkish, 
Somali

Turkish, Somali, 
Kurdish

Russian

Greek, English

English

French

Somali

Greek

Czech

German

Danish

Greek

Spanish

Finnish

French

Czech

German, French, 
Turkish, Kurdish, 
English, 

Danish, Turkish, 
Kurdish, Somali

Greek, English

Finnish, Somali

French, Arabic, 
Tigrinya

Estonian

Russian

Spanish, Arabic, French Spanish

Spanish

Finnish

French

Croatian

Hungarian

English

Italian

Lithuanian, 
Russian

French, 
Portuguese, 
English

Latvian, 
Russian

Maltese, 
English

Croatian

Croatian

Hungarian

Hungarian

Somali

English 

English, Somali

English, French, Arabic

Italian

Italian, English, 
French, Arabic

Lithuanian

Lithuanian, Russian

French, Portuguese, 
English

French

French, English, 
Portuguese

Dutch, 
Tamazight 

Arabic, Turkish, 
Kurdish, Tamazight

Polish

Russian, English

Polish

Latvian

Latvian, Russian

Latvian

English

Dutch

Maltese, English, 
Somali, French, Arabic

Maltese, 
English

Dutch, Arabic, Turkish, 
Kurdish, Tamazight

Polish, Russian, 
English, Vietnamese

Dutch

Polish

Portuguese

Romanian

Swedish

Slovenian

Slovakian

English

English, Kurdish

Somali

Portuguese

Portuguese

Portuguese

Romanian

Romanian

Swedish

Slovenian

Slovakian

English

Swedish, English

Slovenian

Slovakian

English, Hindi, 
Somali, Urdu 

Romanian

Swedish

Slovenian

Slovakian

English

Czech

German

Danish

Russian

Greek

Finnish

French

Croatian

Hungarian

English

Italian

Lithuanian, 
Russian

French

AT

BE

BG

CY

CZ

DE

DK

EE

EL

ES

FI

FR

HR

HU

IE

IT 

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

Source:  FRA, EU-MIDIS II 2016

24

3  

Interviewer selection  
and training

The complexity of the survey related to interviewing 
immigrants and ethnic minorities from diverse back-
grounds across the EU-28, as well as to developing sam-
pling designs and field work methods. This required 
teams in each country to be very well informed about 
fieldwork procedures and the roles of country team 
members in the data collection process. To ensure the 
quality of data collection, FRA’s project specifications 
set out detailed requirements both for training of the 
National Survey Experts and the interviewers in each 
country, and FRA took part in these training events to 
monitor the implementation of training activities and 
to note questions raised during these events so that 
it could assess the need for final adjustments to the 
survey methodology before the start of the full-scale 
data collection stage.

3�1�  Central project briefing

A  two-day  central  briefing  for  the  National  Survey 
Experts (NSEs) representing the local subcontractors 
in  27  EU  Member  States  took  place  in  Berlin  at  the 
beginning of June 2015.18 To ensure that ample atten-
tion was paid to the particular target groups and the 
different sampling methodologies, the briefings took 
place over two sessions (each two days), with those 
surveying Roma and Russians being briefed separately 
from countries where only immigrants and descendants 
of immigrants were to be interviewed.

The briefing covered the following topics:

a)  introduction to the project objectives, main 

requirements and content, and an overview of the 

18  The Irish agency and the replacement agency in the 

Netherlands were exceptions; they joined the project later 
and were briefed separately. 

main challenges associated with implementing 
a project of this scope and nature;

b)  detailed definitions of the target groups, with 

particular focus on how eligibility to participate in 
the survey needs to be established;

c)  the sampling approaches – random route 

sampling as well as the adaptive cluster sampling 
(for those countries implementing this method);

d)  practising and testing use of the contact sheets 
(forms to be filled in by interviewers when 
screening potential respondents for eligibility);

e)  other issues relevant to the fieldwork, such as use 
of introductory letters, how to convince potential 
respondents of the importance of taking part in 
the survey, tools available to assist respondents 
with limited language skills, and issues related 
to potentially different cultural practices among 
respondents that the interviewers should 
be aware of;

f)  gaining cooperation from local non-governmental 
organisations or community leaders to help gain 
access to surveyed target populations (relevant 
primarily to Roma);

g)  detailed description of the questionnaire 

and its administration;

h)  quality control measures;

i) 

interviewer training.

3�2�  Interviewer selection 

and training

Due to the complex nature of the survey, all interview-
ers were required to have prior experience of work-
ing on surveys that used probability sampling. Where 
possible, National Survey Experts were instructed to 
select interviewers with experience of interviewing 

25

Second European Union Minorities and Discrimination Survey – Technical report

respondents  about  sensitive  topics  to  work  on  the 
survey. To maximise participation and response rates 
in the survey, countries were also asked to facilitate 
interviewer matching with regard to ethnicity and gen-
der where possible. Some countries were able to do this 
more easily than others were, depending on the avail-
ability of suitable interviewers. For example, during the 
pilot the team in Denmark found that having Turkish 
speaking interviewers significantly increased the likeli-
hood of securing respondents’ participation, and conse-
quently the team increased its pool of Turkish-speaking 
interviewers. However, not all countries were able to 
change the composition of their interviewing teams, 
and some countries that did so found that the changes 
did not always translate into improved response rates. 
For example, the United Kingdom found that Sub-Saha-
ran African or South Asian interviewers were not always 
more successful than their white British colleagues, and 
that the interviewer’s polling experience was a much 
more important factor. Belgium noted the same issue. 
These considerations, alongside guidance concerning 
the maximum workload per interviewer, determined 
which interviewers to select.

Table 3.1 presents details of the number of interviewers 
who were trained to work on EU-MIDIS II, and inter-
viewer training sessions. In three of the nine EU Member 
States where Roma were surveyed, interviewers iden-
tifying themselves as Roma were included among the 
field force. In the other six countries where EU-MIDIS II 
interviewed Roma, interviewers with Roma background 
who had the requested experience were not available.

All interviewers were required to attend a two-day 
training. In most countries more than one training ses-
sion was organised, for example to train interviewers 
working in different parts of the country. The training 
was particularly important for countries using new or 
more complex sampling methodology (such as random 
route, adaptive cluster sampling or location sampling). 
The interviewer training sessions followed a similar 
structure as the central briefing, and the National Sur-
vey Experts were instructed to tailor and translate the 
centrally produced material for use in their trainings to 
reflect their country specifics.

The National Survey Experts were advised to pay par-
ticular attention to rules concerning the respondent eli-
gibility criteria and sampling, completion of the contact 
sheet, the questionnaire and fieldwork logistics. NSEs 
were also advised to periodically remind interviewers 
to apply consistently the quality control measures, and 
to share good practice examples – especially from those 
interviewers who earlier had carried out the pilot inter-
views, which took place before the main interviewer 
training sessions. Teams were encouraged to make their 
training sessions as interactive as possible, using the 
centrally produced exercises and role play material.

In the countries where Roma were to be surveyed, the 
NSEs were also encouraged to invite representatives 
from non-governmental organisations working with 
Roma to attend the briefings so that they could share 
their practical insights about the target group.19

Where possible, members of the FRA team attended 
the  training  sessions,  and  upon  request  provided 
clarifications directly to the local teams as necessary 
regarding the fieldwork implementation and ration-
ale of the survey questions. FRA also provided feed-
back about how the briefing sessions could be further 
improved, and Ipsos MORI communicated FRA’s feed-
back to the local sub-contractor in question and other 
countries where relevant.

For some countries, it was necessary to hold addi-
tional briefings to those originally foreseen. Reasons 
for this included:

 • There  was  sometimes  a  gap  between  the  initial 
briefing and the fieldwork starting date. This often 
resulted  from  unforeseen  circumstances  such  as 
a delay in receiving the sample, or due to having to 
delay fieldwork due to other reasons such as overlap 
with  national  or  local  elections.  In  these  instances, 
countries were asked to conduct refresher trainings.

 • It  was  sometimes  necessary  to  recruit  new  inter-
viewers  to  the  project.  This  was  often  because 
interviewers  were  working  on  other  projects  in 
parallel  with  the  FRA  survey,  which  meant  that 
additional  interviewers  had  to  be  recruited  and 
trained to ensure that there was an adequate num-
ber of interviewers to complete the project. Some 
interviewers also decided to leave the project; rea-
sons for this were often specific to the country and 
context.20

19  For example, in Bulgaria, two Roma organisations – World 

Without Borders and Amalipe – attended the central training. 
In Denmark, one member of a community organisation in 
Aarhus, which helped the interviewers to gain access to 
the Sub-Saharan African target group, also attended the 
interviewer training in Aarhus. In Spain, members of the NGO 
Fundación Secretariado Gitano attended the interviewer 
training to give advice and answer interviewers’ questions.

20  For example, interviewers in Belgium and France were 
increasingly hesitant to work in certain areas after the 
terrorist attacks in 2015 and 2016. Interviewers from some 
other countries (including the Netherlands) found the project 
challenging, and preferred to work on other surveys where 
it was easier to recruit respondents. However, adequate 
safeguards were implemented on time and the sampling 
was therefore not affected. For example, when sampling 
the PSUs, a replacement PSU was drawn for each PSU in the 
sample following the same randomization principles as in the 
original sample. Therefore, when the fieldwork could not be 
done in certain areas (PSUs) the replacement PSU was used 
for the interviewing. In this way the representativity of the 
sample was not compromised.

26

Table 3�1: Number of interviewers trained to work on EU-MIDIS II and dates of interviewer training sessions

Country

Total number 
of interviewers 
briefed

Dates of interviewer training sessions

Interviewer selection and training  

AT

BE

BG

CY

CZ

DE

DK

EE

EL

ES

FI

FR

HR

HU

IE

IT

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

31

72

64

19

54

87

56

35

36

54

23

87

29

51

15

72

30

18

16

15

54

16

20

80

25

29

75

78

16–17*, 23–24 October 2015; 14–19 November 2015

12–13*, 22–23, 26–27 October 2015; 5–6, 18–19, 21–22 November 2015; 
3 and 5 December 2015; 25 February 2016

19–20*, 26–27 October 2015

6–7 October 2015; 25–26 February 2016

6–7, 8–9*, 10–11, 12–13 October 2015; 14–15 January 2016

1–2*, 7–8 October 2015

29 September 2015; 1 October 2015; 2 and 5 December 2015; 13, 15 and 
26 January 2016; 10 March 2016; 5 and 27 April 2016; 7 May 2016

2–3*, 9–10 October 2015

25–26 September 2015*; 6–7, 9–10, 11–12 November 2015

22–23 September 2015*; 30 October 2015; 2–6, 13–14, 24–27 November 2015

22–23*, 24–25 September 2015; 5–6 November 2015

13–14*, 20–21, 22–23 October 2015

1–2*, 10–11 October 2015; 5–6 November 2015; 1 February 2016

29–30 September 2015; 21 October 2015; 1 February 2016

13–14 October 2015*; 4 interviewers received separate, one-
to-one training; refresher briefing in March 2016

30 September and 1 October 2015*; 6–7, 8–9, 
13–14 October 2015; 1–2 February 2016

28–29 September 2015*; 25–26 November 2015

14–15 March 2016; refresher training 6–7 April 2016

28–29 October 2015

17–18 November 2015

26–27 November 2015; 8 January 2016; 4–5 April 2016; 12–13, 24–25 May 2016

22–23 September 2015*; 27 November 2015; 4–5 February 2016

13–14 October 2015*; 15 November 2015; 5 January 2016

17–18 September 2015*; 5–6, 12–13, 14–15 October 2015

3–4 November 2015*; January 2016

24–25,26–27 November 2015; 3–4 December 2015; 15–16 January 2016

25–26, 27–28 September 2015; 2–3, 4–5 October 2015; 4–5 January 2016

15–16*, 21–22, 28–29 September 2015; 30–1 October 2015; 24–25 November 2015

Note: 

* Training attended by FRA.

Source:  FRA, EU-MIDIS II 2016

27

4  

Sampling

The survey set out to achieve a random probability 
sample of each selected target group in the respec-
tive EU Member States; to provide survey results that 
would be representative for the target populations with 
immigrant or ethnic background in each country; and to 
allow comparison between countries. The groups cov-
ered in the survey are not covered or are insufficiently 
covered in the comparative general population surveys 
of the European statistical system. The range of avail-
able survey and sampling methodologies has increased 
enormously in recent decades. However, reliable strate-
gies for obtaining probability samples for elusive and/
or hard-to-reach populations, such as immigrants and 
ethnic minorities, are still rare. The survey built on 
the sampling approaches and experiences gained in 
previous FRA surveys – in particular the first EU-MIDIS 
survey in 2008 and the Roma survey in 2011 – to fur-
ther refine sampling methodologies and increase the 
efficiency of selected sampling approaches. However, 
in the absence of a sampling frame in a number of coun-
tries, statistical assumptions were made following close 
cooperation with Professor Vijay Verma (University of 
Siena), the external senior survey and sampling expert 
on elusive populations.

Due to the heterogeneity of the target groups in EU-
MIDIS II and different circumstances in the countries, 
a combination of different sampling approaches was 
applied. The development of the sampling approaches 
started with a mapping exercise to identify the most 
appropriate sample frames to sample the target groups. 
After establishing an overview of the various options, 
FRA and Ipsos MORI developed sampling plans appro-
priate to the situation of each country and target group, 
and best able to meet the requirements of the survey 
using the available resources. The sample sizes were 
also optimised for each target group across countries, 

taking into account the relative sizes of the populations 
and sample design quality.

This chapter describes the various steps and approaches 
to sampling employed in EU-MIDIS II. It begins with 
a detailed definition of the target groups of the sur-
vey, followed by a description of sample frames, and 
planned and implemented samples across countries 
and groups. Furthermore, it provides information on 
the outcome of the mapping exercise, and outlines the 
sampling approaches used and how it was implemented 
in the field.

4�1�  Target populations and 

sample requirements
The survey sampled individuals aged 16 years and older:

 • who self-define as persons of immigrant or ethnic 
minority background; this includes immigrants, de-
scendants of immigrants, as well as ethnic minori-
ties (including Roma and people from the Russian 
minority);

 • whose usual place of residence is in the EU Member 

State surveyed;

 • who have been living in private households in the 
EU Member State surveyed for at least the last 12 
months.21

21 

In a small number of countries, persons who were not living 
in private households were also included in the sample. For 
example, in Malta, the target population (immigrants and 
descendants of immigrants from Sub-Saharan Africa) was 
very small. Without including persons living in institutional 
homes, the coverage of this population would have been 
incomplete.

29

Second European Union Minorities and Discrimination Survey – Technical report

The choice of target groups for EU-MIDIS II was based 
on a combination of criteria, including:

the  survey  country  for  at  least  12  months  before 
the interview.

 • ‘Descendants  of  immigrants’  are  persons  who 
were  born  in  one  of  the  current  28  EU  Member 
States or an EEA/EFTA country, whose usual place 
of residence was in the territory of the EU Member 
State  where  the  survey  was  conducted,  and  who 
had at least one parent not born in an EU or EEA/
EFTA  country  (Liechtenstein,  Iceland,  Norway  and 
Switzerland).

 • In some EU Member States, EU-MIDIS II interviewed 
‘recent immigrants’. These are persons who immi-
grated to an EU Member State in the 10 years be-
fore the survey (i.e. after 2004), whose usual place 
of residence was in the territory of the EU Member 
State  where  the  survey  was  conducted,  and  who 
had been living in the survey country for at least 12 
months before the interview. The country of birth 
of ‘recent immigrants’ can be any country outside 
the EU-28 or an EEA/EFTA country.

EU-MIDIS II covered the following groups under the 
concept ‘immigrants and descendants of immigrants’:

 • immigrants and descendants of immigrants from 

Turkey (in 6 EU Member States);

 • immigrants and descendants of immigrants from 

North Africa (in 5 EU Member States);

 • immigrants and descendants of immigrants from 
Sub-Saharan Africa (in 12 EU Member States);
 • immigrants and descendants of immigrants from 
South Asia and Asia (in 4 EU Member States);

 • recent 

immigrants  from  other  non-EU/EFTA 

countries (in 2 EU Member States).

For higher efficiency in screening up to two people were 
interviewed in each household for the target groups 
‘immigrants  and  descendants  of  immigrants’  and 
‘recent immigrants’. Table 4.1 details which countries 
fall into each category. Table 4.2 details the group or 
groups interviewed in each country.

 • size of the target group in a country;22

 • feasibility  of  survey, 

in  terms  of  costs  and 

accessibility;23

 • risk of ‘racially’, ‘ethnically’ or ‘religiously’ motivat-

ed discrimination and victimisation;24

 • vulnerability to social exclusion;25 and/or

 • comparability with FRA’s first EU-MIDIS survey and 

the 2011 Roma survey.

Finally,  in  combination  with  the  above  criteria,  the 
resources available for the survey also served to deter-
mine the number of groups to be interviewed in each 
EU Member State.26 However, the criteria were applied 
to ensure that the survey covered at least one target 
group in each Member State.

The term immigrants and descendants of immigrants, 
as well as the term ethnic minorities are used in the 
context of EU-MIDIS II as umbrella terms encompassing 
a wide range of groups and diversity of characteristics.

Immigrants and descendants 
of immigrants in EU-MIDIS II

Immigrants, descendants of immigrants and recent 
immigrants included both citizens and non-citizens 
of  the  survey  country  irrespective  of  their  formal 
residence status. For the purpose of the survey, the 
term  ‘immigrants  and  descendants  of  immigrants’ 
encompasses the following:

 • ‘Immigrants’  include  persons  who  were  not  born 
in  an  EU  Member  State  or  a  European  Economic 
Area (EEA)/European Free Trade Association (EFTA) 
country  (Iceland,  Liechtenstein,  Norway  and  Swit-
zerland),  who  had  their  usual  place  of  residence 
in the territory of the EU Member State where the 
survey was conducted, and who had been living in 

22  In some countries, it was decided to sample recent 

immigrants, because there was not one single particular 
group large enough to be surveyed. The 10-year period 
focuses on the period since the relevant country acceded to 
the EU.

23  Potential target populations, such as Roma in France and 
Italy, were excluded, as accessibility of halting sites and 
encampments was very difficult during FRA’s 2011 Roma pilot 
survey. 

24  As identified by the national Equality Bodies.
25  In particular for the selection of countries with Roma 

populations. 

26  For instance, Travellers in Ireland were not surveyed, due to 
costs and the fact that previous research had been done on 
the national level.

30

Table 4�1: Definition of country of birth, or country of birth of at least one of the parents, for target groups 
‘immigrants’ and ‘descendants of immigrants’

Target group

Country or region of birth (of the respondent or at least one of his/her parents)

Immigrants and 
descendants 
of immigrants 
from Turkey

Immigrants and 
descendants of 
immigrants from 
North Africa

Immigrants and 
descendants of 
immigrants from 
Sub-Saharan 
Africa*

Immigrants and 
descendants of 
immigrants from 
South Asia

Immigrants and 
descendants of 
immigrants from 
Asia (Cyprus only)

Turkey

Algeria, Egypt, Libya, Morocco, Sudan, Tunisia, Western Sahara

Angola, Benin, Botswana, Burkina Faso, Burundi, Cameroon, Cape Verde, Central 
African Republic, Chad, Congo, Comoros, Côte d’Ivoire, Democratic Republic of 
the Congo, Djibouti, Equatorial Guinea, Eritrea, Ethiopia, Gabon, Ghana, Guinea, 
Guinea-Bissau, Kenya, Lesotho, Liberia, Madagascar, Malawi, Mali, Mauritania, 
Mauritius, Mayotte, Mozambique, Namibia, Niger, Nigeria, Réunion, Rwanda, Saint 
Helena, São Tomé and Príncipe, Senegal, Seychelles, Sierra Leone, Somalia, South 
Sudan, Swaziland, Tanzania, The Gambia, Togo, Uganda, Zambia, Zimbabwe.

UK and France only: Anguilla, Antigua and Barbuda, Bahamas, Barbados, 
British Virgin Islands, Curacao, Dominica, Grenada, Guadeloupe, Haiti, 
Jamaica, Martinique, Montserrat, Saint Lucia, Turks & Caicos Island

Bangladesh, Bhutan, India, Maldives, Nepal, Pakistan, Sri Lanka
UK only: except India

Afghanistan, Armenia, Azerbaijan, Bahrain, Brunei, Cambodia, China, Hong Kong, Macao, 
Georgia, Indonesia, Iran, Iraq, Israel, Japan, Jordan, Kazakhstan, Kuwait, Kyrgyzstan, 
Laos, Lebanon, Malaysia, Mongolia, Myanmar/Burma, North Korea, Oman, Palestine, 
Philippines, Qatar, Saudi Arabia, Singapore, South Korea, , Syria, Tajikistan, Thailand, 
Timor-Leste, Turkmenistan, United Arab Emirates, Uzbekistan, Vietnam, Yemen

Note: 

* For the purpose of the survey, migrants from French departments such as Guadeloupe, French Guiana, Martinique, 
Mayotte and Réunion are included, but are strictly speaking not immigrants.

Source:  FRA, EU-MIDIS II 2016

Sampling 

31

Second European Union Minorities and Discrimination Survey – Technical report

Table 4�2: EU-MIDIS II target groups by country

Country

Target group

AT

BE

BG

CY*

CZ

DE

DK

EE

EL

ES

FI

FR

HR

HU

IE

IT

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from North Africa

Roma

Immigrants and descendants of immigrants from Asia

Roma

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa

Russian minority

Roma
Immigrants and descendants of immigrants from South Asia

Roma
Immigrants and descendants of immigrants from North Africa

Immigrants and descendants of immigrants from Sub-Saharan Africa

Immigrants and descendants of immigrants from North Africa
Immigrants and descendants of immigrants from Sub-Saharan Africa

Roma

Roma

Immigrants and descendants of immigrants from Sub-Saharan Africa

Immigrants and descendants of immigrants from South Asia
Immigrants and descendants of immigrants from North Africa
Immigrants and descendants of immigrants from Sub-Saharan Africa

Russian minority

Immigrants and descendants of immigrants from Sub-Saharan Africa

Russian minority

Immigrants and descendants of immigrants from Sub-Saharan Africa

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from North Africa

Recent immigrants

Roma
Immigrants and descendants of immigrants from Sub-Saharan Africa

Roma

Immigrants and descendants of immigrants from Turkey
Immigrants and descendants of immigrants from Sub-Saharan Africa

Recent immigrants

Roma

Immigrants and descendants of immigrants from South Asia (excluding India)
Immigrants and descendants of immigrants from Sub-Saharan Africa

Note: 

* Immigrants and descendants of immigrants from South Asian countries were excluded from the definition of the target 
group in Cyprus.

Source:  FRA, EU-MIDIS II 2016

32

Ethnic minorities in EU-MIDIS II

4�2�  Sample sizes

EU-MIDIS II uses the term ‘ethnic minorities’ to refer to 
Roma and persons of Russian background who were 
interviewed for the survey.27

In some EU Member States, ethnic minorities such as 
the Roma or the Russian minorities are recognised in 
law as ‘national minorities.’ Although this is not the case 
in all EU Member States, for the purpose of the survey, 
the generic term ‘ethnic minority’ was defined in the 
context of Roma and Russian minorities as in Table 4.3.

For  the  Roma  and  Russian  ‘ethnic  minority’  target 
groups, only one person per household could be inter-
viewed. In EU-MIDIS II, Roma were interviewed in nine 
EU  Member  States,  and  Russian  minorities  in  three 
countries. Table 4.2 presents the distribution of target 
groups surveyed across EU Member States.

To  increase  sampling  efficiency  and  reduce  screen-
ing costs, up to two people could be interviewed in 
each household for the target groups ‘immigrants and 
descendants of immigrants’ and ‘recent immigrants’. For 
the Roma and Russian ‘ethnic minority’ target groups, 
only one person per household could be interviewed.

EU-MIDIS II achieved a total net sample size of 25,200 
interviews across the 28 EU Member States. In deter-
mining the target sample sizes by country and by group, 
an  optimal  allocation  of  the  sample  was  examined 
taking into account:

 • a minimum sample size of 500 interviews per coun-
try  and  a  minimum  of  400  interviews  per  target 
group;

 • the  size  of  the  target  group  population  –  with-
in  a  country  and  overall  in  the  EU  –  with  the  aim 
of  maximising  the  efficiency  of  cross-country 
weighting.28

For a given group, the sample was distributed across 
countries in relation to the absolute size of the group in the 
country. This procedure optimises the sample allocation 
across countries, but because it is important for EU-MIDIS II 
to also deliver results for each group at the level of indi-
vidual EU Member States, the sample size was adjusted 
to ensure the minimum sample size of 400 interviews 
per group and a minimum of 500 interviews per country. 

Table 4�3: Definitions of ethnic minority target groups in EU-MIDIS II

Target group

Definition

Roma

The term “Roma” is used as an umbrella term in the Council of Europe definition and refers 
to Roma, Sinti, Kale and related groups in Europe, including Travellers and the Eastern 
groups (Dom and Lom). It covers the wide diversity of the groups concerned, including 
persons who identify themselves as Gypsies. The Council of Europe uses the term “Roma 
and Travellers” to encompass the wide diversity of groups covered by the work of the 
Council of Europe in this field: “on the one hand a) Roma, Sinti/Manush, Calé, Kaale, 
Romanichals, Boyash/Rudari; b) Balkan Egyptians (Egyptians and Ashkali); c) Eastern groups 
(Dom, Lom and Abdal); and, on the other hand, groups such as Travellers, Yenish, and the 
populations designated under the administrative term ‘Gens du voyage’, as well as persons 
who identify themselves as Gypsies.” See the Council of Europe’s webpage on Roma.

For the purpose of this survey, the term ‘Roma’ refers to persons who self-identify as 
‘Roma’ or as one of the other groups that the term ‘Roma’ covers. It refers to autochthonous 
‘Roma’ minorities within selected EU Member States and does not encompass ‘Roma’ who 
have moved from to the survey country from another EU Member State or Travellers. 

Russian 
minority

For the purpose of this survey, the term ‘Russian minority’ refers to persons who 
self-identify as belonging to a Russian national or ethnic minority in the EU Member 
States where this group was included in the survey. The term includes both citizens 
and non-citizens of the country surveyed (including persons without citizenship).

Source:  Council of Europe, 2012

27  In practical implementation, in a few countries, residents 

outside private households were also included. For example, 
in Malta, the target population is very small and, without 
including some of the institutional homes, the population 
could not have been covered. In other countries where 
location sampling was applied, a clear distinction cannot 
always be made. 

28  Efficiency means to calculate the optimal sample size with 
the best precision for between- country comparisons and 
within- country estimates.

Sampling 

33

Second European Union Minorities and Discrimination Survey – Technical report

Therefore, the sample allocated to country i with size 
Si of the group concerned is

ni = max (nmin , kSa
i )
where a is a ‘compromise’ parameter 0.5, and constant 
k is determined by the requirement that the country 
allocations ni sum up to the total sample size n assigned 
to the group.

Further adjustments to sample sizes had to be made 
taking into consideration:

 • the  relative  quality  of  the  country-level  sample 
designs (in particular, the population coverage and 
sample efficiency that could be achieved);

increase the number of issued addresses depending on 
the success rate in the field. Once an address was issued 
or a PSU was started it had to be finished by completing 
all required visits to the address or the PSU.

Russian minority

Table  4.4  shows  the  sample  allocation  for  Russian 
minority populations in EU-MIDIS II. The sample size 
for the Russian minority target group in Latvia was 
increased compared with the sample size suggested 
by the optimal allocation method to reflect the larger 
population  in  Latvia  as  opposed  to  the  size  of  the 
Russian minority population in Estonia and Lithuania 
(see Table 4.4).

 • the relative cost of the fieldwork in each country; 

Roma target group

and

 • the heterogeneity of a group within a country, such 
as different countries of origin, or the need to dif-
ferentiate  between  first-  and  second-generation 
respondents as a part of the analysis of the results.

Tables  4.4  to  4.10  present  the  target  group  popula-
tion sizes, the within-target group optimal allocation 
range considered, the final agreed target sample sizes 
and the final delivered sample sizes of interviews. Two 
approaches  defined  an  optimal  allocation  range  by 
assessing the following: 1) the maximum possible cross-
country sample efficiency, while taking into account the 
minimum target group sample size of 400 per country, 
and 2) a slightly less efficient method aimed for a com-
promise between overall cross-country efficiency and 
a more even distribution between countries. The ranges 
were set to sum up to the total recommended sample 
sizes for each country, after adjustments between target 
groups in countries where more than one group was 
interviewed. The targeted sample sizes represent the 
net interviews. To estimate the size of the gross sample 
needed to achieve the net sample size, each country 
started with an assumption on the achievable response 
rate. Over the course of the fieldwork, in some coun-
tries addresses and/or PSUs (Primary Sampling Units) 
were issued in (random) stages to be able to reduce or 

Table 4�4: Allocation of sample sizes: Russian minority

In the majority of the countries where Roma were sur-
veyed in EU-MIDIS II, the final sample size was within 
the range of the optimal allocation calculations (see 
Table 4.5). Following the optimisation procedures, the 
final overall target sample size for the Roma target 
group was increased (from 6,400 interviews at the 
start of the contract, to 7,750 interviews). This adjust-
ment was offset against reductions in sample sizes in 
some of the other countries, which provided additional 
resources in these countries, which were used to make 
improvements to sample quality. In countries with less 
reliable sampling frames (EL, ES, HR and PT) and for 
reasons of cost and available interviewing capacity, 
the planned sample size was not increased up to the 
suggested optimal allocation. This mainly affected the 
sample in Spain. The overall weighting efficiency of the 
cross-country Roma sample was 77 %.

Immigrants and descendants of immigrants 
from Turkey

The overall planned sample size for immigrants and 
descendants  of  immigrants  from  Turkey  was  3,500 
(Table 4.6). Between the six countries where this group 
was interviewed, the sample sizes were adjusted to 
reflect the relative sizes of the target populations in 
a country, and in most cases the target sample size 

Country

Russian 
population 
(all ages)*

Optimal allocation 
range – lower 
bound (N)

Optimal allocation 
range – upper 
bound (N)

Target sample 
size (N)

Achieved sample 
size (N)

EE

LT

LV

321,024

176,913

557,119

Total

1,055,056

410

400

465

449

436

540

400

400

550

1,350

401

404

614

1,419

Note: 

* Figures derived from Census 2011.

Source:  FRA, EU-MIDIS II 2016

34

Table 4�5: Allocation of sample sizes: Roma

Country

Roma population 
(all ages)*

Optimal allocation 
range – lower 
bound (N)

Optimal allocation 
range – upper 
bound (N)

Target sample 
size (N)

Achieved sample 
size (N)

BG

CZ

EL

ES

HR

HU

PT

RO

SK

Total

Note: 

325,343

237,865

62,920

550,000

16,975

315,583

39,233

621,573

402,590

2,572,082

942

840

432

1,105

400

934

400

1,150

1,003

982

864

639

1,277

524

968

588

1,358

1,093

1,050

850

500

700

500

1,050

500

1,450

1,150

7,750

1,078

817

508

776

538

1,171

553

1,408

1,098

7,947

* The estimate for Spain is based on information from the Fundación Secretariado Gitano; for the sources for the other 
countries, see Table 4.11.

Source:  FRA, EU-MIDIS II 2016

is within the range suggested by optimal allocation. 
Population estimates were only available for the first 
generation of immigrants. For the calculation of optimal 
allocation, population sizes were increased by 50 % to 
account for the second generation.

To  reflect  the  large  Turkish  population  in  Germany 
appropriately, it would have been desirable to increase 
the size of this sample even further. However, due to 
a lack of better sample frames, the sample in Germany 
has lower precision and coverage compared with sam-
ples of other countries where immigrants and descend-
ants  of  immigrants  from  Turkey  were  interviewed. 

Taking also into account the fieldwork costs, a further 
increase of the sample size in Germany would not have 
improved the quality of the results to the extent as to 
justify the necessary additional investment. The overall 
weighting efficiency of the Turkish sample across the 
six EU Member States is 41 %, and it would have been 
necessary to increase the German sample size towards 
the upper limit of the optimal allocation range to make 
a notable improvement to this.

After the start of fieldwork, the planned sample size in 
Denmark was reduced to from 500 to 450 interviews 
due to difficulties reaching the original target.

Table 4�6: Allocation of sample sizes: immigrants and descendants of immigrants from Turkey

Country

Estimated size of 
target population 
(all ages) *

Optimal allocation 
range – lower 
bound (N)

Optimal allocation 
range – upper 
bound (N)

Target sample 
size (N)

Achieved sample 
size (N)

AT 

BE 

DE 

DK 

NL 

SE 

Total

Notes: 

239,820

204,857

2,877,000

61,634

396,414

69,219

3,848,944

400

400

912

400

514

400

521

537

1,386

475

590

465

550

570

880

450

600

400

578

628

919

400

617

402

3,500

3,544

In some countries, population estimates were available for first-generation immigrants only, so the available estimates 
have been increased by 50 % to account for the impact of including second-generation respondents in the target group of 
the survey. In Belgium, Denmark, Germany and the Netherlands, estimates of the total target group (first- and second-
generation respondents) were available and no further adjustments were necessary. 
* For the sources of the specified target populations, see Table 4.11.

Source:  FRA, EU-MIDIS II 2016

Sampling 

35

Second European Union Minorities and Discrimination Survey – Technical report

Immigrants and descendants of 
immigrants from North Africa

the actual sample size for the second group in France 
(SSAFR) was higher than planned.29

The overall planned sample size for the target group 
‘immigrants and descendants of immigrants from North 
Africa’ was 3,900 (see Table 4.7). All countries fit or are 
very near to the optimal allocation ranges proposed 
to distribute the available sample. The target sample 
size was not reached in France, as the combined sam-
ple for the two target groups in France yielded higher 
concentrations  for  immigrants  and  descendants  of 
immigrants from Sub-Saharan Africa than expected and 
lower concentrations than estimated for immigrants 
and descendants of immigrants from North Africa in 
the selected PSUs. Consequently, the actual sample size 
for the group of immigrants and descendants of immi-
grants from North Africa was lower than planned and 

Immigrants and descendants of 
immigrants from South Asia

The target sample size for immigrants and descend-
ants of immigrants from South Asia was reached in all 
three countries where this group was interviewed (see 
Table 4.8). In the United Kingdom the target sample size 
was originally set for 700 interviews, but this was later 
reduced to 600 interviews due to lower eligibility rates 
encountered during the fieldwork.

Table 4�7: Allocation of sample sizes: immigrants and descendants of immigrants from North Africa

Country

Estimated size of 
target population 
(all ages)*

Optimal allocation 
range – lower 
bound (N)

Optimal allocation 
range – upper 
bound (N)

Target sample 
size (N)

Achieved sample 
size (N)

BE

ES

FR

IT

NL

Total

Notes: 

486,645

1,149,398

5,313,000

925,242

421,803

8,296,088

556

698

1,131

626

518

671

740

1,501

705

652

700

750

1,100

700

650

3,900

711

787

846

836

653

3,833

Population figures were available for only first-generation immigrants in France, Italy and Spain. These estimates have 
been increased by 50 % to account for the impact of including second-generation respondents in the target group of the 
survey.  
* For the sources for the specified target population estimates, see Table 4.11 (sample frames).

Source:  FRA, EU-MIDIS II 2016

Table 4�8: Allocation of sample sizes: immigrants and descendants of immigrants from South Asia or Asia

Country

Estimated size of 
the population 
(all ages)*

Optimal allocation 
range – lower 
bound (N)

Optimal allocation 
range – upper 
bound (N)

Target sample 
size(N)

Achieved 
sample size(N)

CY**

EL

IT

UK

Total

Notes: 

22,124

84,755

574,151

1,075,331

1,734,236

N/A

400

549

649

N/A

470

582

751

400

500

500

600 (700)

2,000

436

515

517

668

2,136

* For the sources for the population estimates, see Table 4.11. All population estimates for this group were available for 
first-generation immigrants only, so the estimates have been increased by 50 % to account for the impact of including 
second-generation respondents in the target group of the survey. 
**Cyprus is the only country with the Asian target group so optimal allocation is not applicable (N/A).

Source:  FRA, EU-MIDIS II 2016

29  This was also related to difficulties in estimating the 

concentrations in advance, because the detailed data on the 
target group were not available at the PSU level and had to 
be estimated. 

36

Immigrants and descendants of 
immigrants from Sub-Saharan Africa

The sample sizes for the target group immigrants and 
descendants of immigrants from Sub-Saharan Africa 
was adjusted as far as possible to reflect the relative 
sizes of the target populations in the country(see Table 
4.9). In particular, the UK sample size was increased to 
700 interviews to reflect the larger population in this 
country, and the samples were reduced to the minimum 
size of 400 in countries with relatively small popula-
tions (Ireland, Malta and Sweden). In France, the sample 

would have benefited from a further increase, but this 
would have led to a loss of sample efficiency within the 
French sample overall, given that the other target group 
in France – immigrants and descendants of immigrants 
from North Africa – has a population twice the size of 
the Sub-Saharan population in France. After the start 
of fieldwork, the sample sizes in Austria and Denmark 
were also reduced, from 500 to 400 interviews, due to 
difficulties in achieving the original target sample sizes. 
Finally, the target was also reduced in the UK in the 
course of the fieldwork from 700 to 600 due to lower 
than expected eligibility rates.

Table 4�9: Allocation of sample sizes: immigrants and descendants of immigrants from Sub-Saharan Africa

Country

Sub-Saharan 
population 
(all ages)*

Optimal allocation 
range – lower 
bound (N)

Optimal allocation 
range – upper 
bound (N)

Target sample 
size(N)

Achieved 
sample size(N)

AT

DE

DK

FI

FR

IE

IT

LU***

MT

PT

SE

UK

Total

Notes: 

30,831

251,000

39,085

31,059

1,858,500

59,712

452,129

21,607

5,700

532,979

201,048

1,961,738

5,445,388

400

400

400

400

620

400

420

N/A

400

456

400

626

428

499

439

435

851

439

508

N/A

415

518

472

874

400 (500)

500

400 (500)

500

500

400

500

400

400

500

400

600 (700)

5,500

476

500

451

502

794

425

369**

402

411

525

400

548

5,803

* For the sources of the specified population estimates, see Table 4.11 (sample frames). In some countries, population 
estimates were available for first-generation immigrants only, so these estimates were increased by 50 % to account for 
the impact of including second-generation respondents in the target group of the survey. In Denmark, Finland, Germany 
and Malta, estimates of the total target group (first- and second-generation respondents) were available. 
** In Italy, the three target groups were sampled simultaneously and the fieldwork resulted in fewer interviews 
of immigrants and descendants of immigrants from Sub-Saharan Africa than planned (below the minimum of 400 
interviews per group), while more interviews with immigrants and descendants of immigrants from North Africa were 
achieved.  
*** The target group originally planned for Luxembourg was recent immigrants, so Luxembourg was not considered in 
the optimal allocation of the sample concerning immigrants and descendants of immigrants from Sub-Saharan Africa. 
During the implementation of the survey, however, the target group in Luxembourg had to be changed, because it was 
not possible to access the required sample frame to draw a sample of recent immigrants; the decision was made to 
change the target group in Luxembourg to immigrants and descendants of immigrants from Sub-Saharan Africa.  
N/A = not applicable.

Source:  FRA, EU-MIDIS II 2016

Sampling 

37

Second European Union Minorities and Discrimination Survey – Technical report

Recent immigrants

The sample allocations were not revised from the initial 
levels for the remaining target groups. All three target 
groups were set at the minimum country/target group 

level of 400 interviews. This meant that there was no 
room to further optimise the sample allocations for 
these target groups.

Table 4�10: Allocation of sample sizes: immigrants and descendants of immigrants from Asia and recent immigrants

Country

Target group population (all ages)*

Target sample size (N)

Achieved sample size (N)

PL

SI

Total

15,612

36,523

52,135

400

400

800

429

404

833

Note: 

*For the sources for the specified population estimates, see Table 4.11 (sample frames).

Source:  FRA, EU-MIDIS II 2016

4�3�  Sample frames 

and mapping of available 
information for sampling

One of the main objectives for the sampling strategy of 
EU-MIDIS II was to achieve representativeness through 
random probability sampling for all target groups in 
each of the 28 EU Member States. Therefore the follow-
ing aspects were considered when mapping information 
in order to decide upon the specific sampling strategy 
for each target group in each country:

a)  the size of the population (for each target group);

b)  the geographical concentration of the target 

group (based on addresses);

c) 

if no address frame is available in the country for 
sampling purposes, in which locations or ‘centres’ 
the target group congregates; this includes any 
locations of meeting and work, not to be defined 
too narrowly and listed in the order of magnitude, 
with size estimates if necessary;

d)  assessment of difficulty of surveying the 
group and any social network aspect of 
that particular group;

e)  qualitative assessment: how reclusive 

is that population?;

f) 

identification of further lists and assessment 
of possible use, e.g. if telephone screening 
lists are available;

g)  identification of characteristics of the target 

population, including country of birth, years of 
residence, regional information, etc.

In order to carry out this assessment, information from 
a number of available sources were used, including 
available statistics as well as expert assessments. The 
target population size was examined primarily based on 
registers data, results of the 2011 census 2011, Labour 
Force Survey (LFS) and other available survey data.

The Central Coordination Team used the information 
to develop a sampling strategy for each target group 
and country. National Survey Experts were asked to 
provide comments on the sampling strategy as well 
as to assess the information on the basis of which the 
sampling strategy had been developed. The informa-
tion concerning the distribution of the target population 
within the country was used to decide on the survey’s 
regional coverage and how to define a cut off point for 
areas with low density. Screening of addresses becomes 
feasible only at around a 5–10 % concentration of the 
target population. A 5 % concentration means that, on 
average, it is necessary to screen 20 households suc-
cessfully to find one eligible household. This is a very 
costly  approach  that  makes  it  necessary  to  choose 
a cut-off point for areas with low density in countries 
without an individual sample frame.

Given that most of the target groups in EU-MIDIS II can 
be considered hard to reach for survey research – in 
terms of being relatively small in size and/or dispersed 
finding a suitable sample frame in each country was 
a critical part of the preparatory work for the survey. For 
most countries, physical access to the sampling frame 
was needed for optimising the sample design. In several 
cases, FRA had to provide support to the national survey 
companies to obtain access to the full frame, such as the 
full list on census district level with number of persons 
of the target group of country of birth. The statistical 
offices had to comply with data protection rules and 
could grant access only when FRA directly requested 
the access and ensured the purpose of the survey and 
confidential treatment of the data.

38

Table 4.11 lists the sample frames used for each country 
and target group. Sample frames that allowed direct 
identification of eligible individuals – based on up-to-
date population registers – were preferred over other 
frame types because they could offer a high level of 
quality and efficiency. Full access to register-based indi-
viduals from this type of frame was possible in Den-
mark, Finland and Poland.

If a population register was not available or not acces-
sible to the survey research agencies working on EU-
MIDIS II for FRA, the next option was to find a sample 
frame that would allow indirect sampling of the target 
groups. This type of sample frame would allow – in 
the first stage – selecting a sample of small areas of 
the country where the survey’s target group is living, 
followed by screening the areas in a second stage to 
find people who are eligible to take part in the survey 
(see Section 4.4 for details concerning the methods 
in this second-stage sampling). This type of primary 
sampling unit frame was available in a majority of the 
countries (21 EU Member States) where suitable individ-
ual-level frames do not exist or could not be accessed 
in EU-MIDIS II. Due to the need for a second sampling 
stage when using these frames, sources that would 
allow selection of household/dwelling addresses from 
a register was preferred over random route sources. 
These address lists were available in four out of the 
21 countries where two-stage selection was necessary 
(see Table 4.11).

Obtaining adequate sample frames that could be used 
in the EU-MIDIS II survey was both time consuming 
and labour-intensive. The official requests for access 
to sample frames were made to the national statistical 
institutes in February 2015. At the start of the pilot data 
collection in July 2015 sample frames access to sampling 
frames had been obtained in most countries. However, 
it was not possible to obtain sample frames from official 
sources in Germany, Malta or Sweden, or instead of 
the preferred sample frames, another frame had to be 
accepted. Organisations responsible for official records 
in these three countries referred to data protection laws 
as grounds for not granting access to their information. 
In other countries, different problems were encountered 
in various stages of the sampling procedure:

 • In  Finland,  the  main  African  languages  spoken  by 
Sub-Saharans were initially used as a selection cri-
teria  when  drawing  a  sample  of  individuals  from 
the  population  register,  instead  of  basing  the  se-
lection  on  the  country  of  birth  which  also  would 
have been available in the register. This meant that 
Sub-Saharans  who  listed  their  main  language  as 
English, French, Portuguese or Finnish were initially 
excluded. Since this corresponds to some 34 % of 
the target population, and as it turned out the ini-
tial sample was not large enough, a second sample 

was drawn from the population register during the 
fieldwork. This new sample was selected to over-
represent  the  group  that  had  been  excluded  from 
the first sample, so that the final achieved sample 
was well-balanced as to the main language spoken 
by the respondents.

 • In Italy, the data initially available for sampling pur-
poses  referred  to  geographic  units  –  communes, 
which  cover  whole  settlements  including  cities 
–  that  were  too  large  to  be  used  as  primary  sam-
pling  units.  Further,  the  request  made  to  the  Ital-
ian National Institute of Statistics (ISTAT) for data at 
a smaller geographic level could not be processed 
in  time  for  the  pilot  data  collection.  By  the  main 
stage  fieldwork  suitable  sample  frame  data  for 
Italy were received from ISTAT.

 • In Luxembourg, FRA made a request to the national 
registry  holder  for  access  to  their  register  for  the 
purposes  of  EU-MIDIS  II.  However,  despite  signifi-
cant efforts over the course of the year, it was not 
possible  to  obtain  addresses  for  contacting  re-
spondents  face-to-face.  As  a  result,  two  changes 
were  made  to  the  sample  design  in  Luxembourg. 
First,  the  target  group  was  changed  from  ‘recent 
immigrants’  to  ‘immigrants  and  descendants  of 
immigrants from Sub-Saharan Africa’, as the latter 
are a relatively more homogenous group and geo-
graphically less dispersed in Luxembourg. Second, 
in the absence of access to register data the sam-
pling methodology was changed to quota sampling.

 • In  Slovenia,  the  original  sample  design  assumed 
the direct sampling of the target population using 
data  from  the  national  register,  but  direct  access 
to the register was denied with reference to exist-
ing data protection regulations. The sample design 
was  therefore  changed  and  an  indirect  sampling 
approach,  which  required  screening  for  the  target 
group, was adopted for the main stage fieldwork. 
The statistical office offered to deliver instead a list 
of addresses of the general population but with an 
oversample of the target group.

 • In Germany the telephone directory was used – as 
a form of individual-level sample frame – to apply 
an  onomastic  sampling  approach,  which  involved 
pre-screening entries from the telephone directory 
to identify people who are likely to belong to the 
survey’s target groups based on their names. This 
approach was taken due to a change in law, which 
at the time of the survey meant that municipal au-
thorities in Germany were not able to provide sam-
pling  data  for  surveys.  Such  sampling  frame  may 
introduce  a  bias  to  the  sample,  excluding  those 
who cannot be identified via name and persons not 
registered  in  the  telephone  directory.  Additional 

Sampling 

39

Second European Union Minorities and Discrimination Survey – Technical report

sampling by referrals (see Section 4.6) was applied 
to address the possible bias in this regard.30

 • In Malta, the population statistics that would have 
been  necessary  to  inform  the  sample  design  are 
not collected by the authorities.

 • In  Sweden,  legislation  forbids  the  use  of  register 
data for selecting persons on the basis of their race, 

ethnicity,  political  views  or  religious  beliefs.  How-
ever, data at the regional level were obtained and 
proved  helpful  for  planning  the  location  sampling 
in Sweden.

Table 4.11 gives an overview of sampling frames used 
for each target group in each country. Most samples 
were stratified to control for region and urban density.

Table 4�11: EU-MIDIS II sample frames by country and target group

Country

Target group*

Sample frame/source of 
data used for sampling

Sample frame and 
selection stages 

AT

SSAFR

Population register 2014 
and Austrian Post 

TUR 

Population register 2014 
and Austrian Post

Enumeration districts or 
Zählsprengel (PSUs) and 
addresses (selection)

Enumeration districts or 
Zählsprengel (PSUs) and 
addresses (selection)

Stratification variables

NUTS2, DEGURBA

NUTS2, DEGURBA

BE

NOAFR

Census 2011 and Orgassim Census areas or 

NUTS2, DEGURBA

statistische Sector (PSUs) 
& addresses (selection)

TUR 

Census 2011 and Orgassim Census areas or 

NUTS2, DEGURBA

BG

CY

CZ

DE

ROMA

ASIA

Census 2011

Census 2011

ROMA

SSAFR

TUR 

Census 2011 and 
total number of 
Roma estimates

H&S onomastic (survey) 
and Microcensus 2013 
(population adjustments)

H&S onomastic (survey) 
and Microcensus 2013 
(population adjustments)

statistische Sector (PSUs) 
and addresses (selection)

Census areas (PSUs)

NUTS3, rural/urban

City district level 
‘ΕΝΟΡΙΑΣ’ (larger 
cities) or municipality/
community ‘ΔΗΜΟΥ/
ΚΟΙΝΟΤΗΤΑΣ’ (PSUs, larger 
settlements divided)

Municipality (PSUs, larger 
settlements divided)

LAU1, DEGURBA

NUTS3, urban/rural

Settlements (PSUs) and 
individual (selection)

BIK and Nielsen 
territory**

Settlements (PSUs) and 
individual (selection)

BIK and Nielsen territory

DK

SSAFR

Population register 2015

Municipality (for 
mapping and coverage)

N/A

TUR 

Population register 2015

Municipality level 
(for mapping and 
coverage) and individual 
level (selection)

Sample not stratified

30  Research on the application of onomastic sampling among 

immigrants in Germany has shown that there is indeed a bias 
for some outcomes resulting from this method. However, 
the overall bias is not necessarily considerably large. See 
Schnell et al. (2014). The bias for EU-MIDIS II is assumed to be 
mitigated by adding the referral sample.

40

Country

Target group*

Sample frame/source of 
data used for sampling

Sample frame and 
selection stages 

EE

RUSMIN

Census 2011

Sampling 

Stratification variables

LAU1, DEGURBA

NUTS2, rural/urban

Municipality or 
omavalitsus (linn, vald)/
linnaosa (PSUs, larger 
settlements divided)

Municipality (PSUs, larger 
settlements divided)

ROMA

Roma Pilot Survey 2011

SASIA

ROMA

NOAFR

SSAFR

Census 2011

Census areas (PSUs)

NUTS3, DEGURBA

Gitanos.org 2007 data

Municipality or 
municipio (PSUs, larger 
settlements divided)

NUTS2, settlement size

Census 2011

Census sectors (PSUs)

NUTS2, DEGURBA

Population register 2015

EL

ES

FI

IE

IT

FR

SSAFR

Census 2011 and foreign-
born estimates

NOAFR

Census 2011 and foreign-
born estimates

HR

ROMA

Census 2011

HU

ROMA

Census 2011

SSAFR

Census 2011

NOAFR

Census 2011

SASIA

Census 2011

LT

RUSMIN

Census 2011

City (for mapping 
and coverage) and 
individual (selection)

Commune for target group 
and IRIS (‘Aggregated 
units for statistical 
information’ – translated 
from French) for 
foreign-born (PSUs)

Commune for target 
group and IRIS for 
foreign-born (PSUs)

Settlements or 
naselje (PSUs, larger 
settlements divided)

Enumeration 
district (PSUs)

Sample not stratified

NUTS2, DEGURBA

NUTS2, DEGURBA

NUTS3, urban/rural

NUTS2, rural/urban

NUTS2, DEGURBA

NUTS2, DEGURBA

NUTS2, DEGURBA

LAU1

Census areas or sezioni 
di censimento (PSUs)

Census areas or sezioni 
di censimento (PSUs)

Census areas or sezioni 
di censimento (PSUs)

Seniunija (only available 
for Vilniaus) and 
municipality (all others) 
used for target group 
populations, general 
population data on 
electoral districts to 
divide into PSUs

SSAFR

Census 2011 and An Post 
(Irish postal system)

Census small areas (PSUs) 
and addresses (selection)

NUTS3, DEGURBA

LU

LV

SSAFR

RUSMIN

MT

SSAFR

General Social Security 
Inspectorate (IGSS) 
statistics 2015

OMCA registry 
statistics and election 
results estimates

National level only

Sample not stratified

Municipality and 
enumeration district 
‘RĪGAS APKAIMĒM’ (PSUs)

Municipality

No official statistics, 
population size estimated 
from desk research

N/A

Sample not stratified

41

Second European Union Minorities and Discrimination Survey – Technical report

Country

Target group*

Sample frame/source of 
data used for sampling

Sample frame and 
selection stages 

Stratification variables

NL

NOAFR

TUR 

PL

RIMGR

Population register 
2014 and Cendris (Dutch 
register of postcodes)

Population register 
2014 and Cendris (Dutch 
register of postcodes)

Census 2011 and PESEL 
(Universal Electronic 
System of Population 
Register) registry data 
and Centre for Migration 
Research (CMR) estimates

5-digit postcodes (PSUs) 
and address (selection)

Region, DEGURBA

5-digit postcodes (PSUs) 
and address (selection)

Region, DEGURBA

Addresses after 
aggregation of individual 
register (from PESEL)

City area

PT

ROMA

Roma Pilot Survey 2011

Municipality ‘MUNICÍPIO’ 
(PSUs, larger 
settlements divided)

NUTS2, settlement size

RO

SE

SI

SK

UK

SSAFR

ROMA

SSAFR

Census 2011 and 
naturalised immigrant 
estimates

Statistical sectors 
or secção (PSUs)

NUTS3

Census 2011

Census districts (PSUs)

NUTS3, urban/rural

Population register 2014

TUR

Population register 2014

Municipality (for 
mapping and coverage)

Municipality (for 
mapping and coverage)

N/A

N/A

RIMGR

ROMA

SSAFR

SASIA

Population register

Individuals (selection)

LAU2, urban/rural

Atlas 2013

Census 2011 and Post 
Office Address File

Census 2011 and Post 
Office Address File

Municipality (PSUs, larger 
settlements divided)

NUTS3, urban/rural

Census output areas 
(PSUs) and addresses 
(selection)

Census output areas 
(PSUs) and addresses 
(selection)

NUTS2, DEGURBA

NUTS2, DEGURBA

Notes: 

* NOAFR, immigrants and descendants of immigrants from North Africa; SASIA, immigrants and descendants of 
immigrants from South Asia; SSAFR, immigrants and descendants of immigrants from Sub-Saharan Africa; TUR, 
immigrants and descendants of immigrants from Turkey.  
** Measure of population size of a location provided by the BIK Institute (BIK Aschpurwis + Behrens), and regional 
divisions provided by the Nielsen Company.  
N.A. = not applicable.

Source:  FRA, EU-MIDIS II 2016

42

4�4�  Overview of sampling 

methodologies used

This section describes the sampling methodologies 
used for the EU MIDIS II survey. A number of different 
sample designs were used. Proposals on sampling and 
selection strategies were based on statistical grounds 
considering a balance between precision, feasibility 
and costs. The main reference design in EU-MIDIS II 
was a household-based survey of individual persons. 
For most target groups and in most countries, in the 
absence of individual registers with country of birth and 
parents’ country of birth information, an area-based 
sample design was used.

In most cases areas were selected in a single sampling 
stage with probability proportional to target population 
size (stage 1), followed by the selection of households 
and  respondents  within  households  (stage  2).  First 
contact was made face-to-face in all countries except 
in Denmark and Finland, where the sampled persons 
were first screened for eligibility via telephone, and 
an appointment for face-to-face interview was made 
with respondents who were confirmed on the phone 
as being eligible to take part in the survey.

The following sample design approaches were used in 
EU-MIDIS II. In some countries, a combination of sam-
pling designs was used:

1.  Direct unclustered single-stage sampling 
selecting from individual person-level 
registers (three countries).

2.  Multi-stage area sampling, with primary 

sampling units (PSUs) selected at the first stage. 
For the second stage sampling within sample 
areas the choice is generally between: (i) 
systematic sampling from existing population 
lists, where such lists are available, and (ii) 
random route sampling. EU-MIDIS II used these 
approaches as follows:
a.  systematic sampling from address registers (in 

four EU Member States);

b.  addresses identified via random route (in 

c. 

15 EU Member States);
in Germany random walk route was assessed 
as not to be feasible and therefore addresses 
were identified through an onomastic (name 
matching) process and via referrals.

3.  Location sampling or centre-based sampling (in 

seven EU Member States).

4.  A non-probability sampling in Luxembourg 
(quota sampling) after requests for register 

data were rejected and other forms of sampling 
were not feasible.

In  the  case  of  the  most  prevalent  sampling  design 
approach No. 2 (multi-stage area sampling), screening 
is required in the second stage of the sampling to iden-
tify eligible individuals within the areas selected in the 
first sampling stage. Screening is a major component of 
survey fieldwork costs, and can be a particular burden 
to interviewers in low concentration areas where eli-
gible individuals are rare. The efficiency of the sample 
depends greatly on how the area units are selected. 
Therefore various measures were taken in EU-MIDIS II 
to increase feasibility and efficiency of screening:

(1)  Identification and exclusion of practically ‘empty’ 
primary sampling units – that is, PSUs where 
eligible individuals would be extremely rare;

(2)  Classification of the remaining areas according 
to the degree of concentration of the target 
population. After classifying the areas, it is 
possible to determine an appropriate cut-off 
point for excluding low concentration areas, along 
with an estimate of the proportion of the target 
population (see section 4.5 on coverage). Strata 
with higher concentrations can be over-sampled.

(3)  Supplementary procedures when the main area-
based design is not able to provide an adequate 
or efficient design are adaptive cluster sampling 
(ACS) or focused enumeration (FE),31 which were 
both used in EU-MIDIS II. Where neither ACS 
nor FE were feasible – or to complement other 
methods – location sampling was used.

Coverage and efficiency of design

To improve the efficiency of the screening and fieldwork 
related efforts and costs, country sampling plans were 
optimised through stratification of the frame according 
to degree of concentration of the target population (i.e. 
percentage of target population in the total population 
of the PSU), and by

 • excluding  empty  or  low  concentration  strata  from 
the sample by setting a minimum level of concen-
tration  ‘cut-off’  (see  Table  4.12  for  details  of  the 
countries  where  a  concentration  cut-off  was  set) 
and/or

 • oversampling more concentrated strata; this result-
ed in lower sample efficiencies (see Table 4.13 for 
details  of  the  countries  where  over-sampling  was 
undertaken, and efficiency of the design).

31  These two methods are described under the respective 

sample designs.

Sampling 

43

Second European Union Minorities and Discrimination Survey – Technical report

In the majority of the EU Member States, the sampling 
design targeted national coverage of the target popula-
tion. In the absence of sampling frames that allow for 
identifying the target group it was decided to exclude 
areas with low concentration of the target populations, 
as screening would become unfeasible or too expen-
sive. Screening for a population with less than 5 % con-
centration was considered to be inefficient in terms of 
costs and interviewer motivation.32

Previous analysis shows, for instance, that Roma who 
live in more segregated areas are more vulnerable 
to poverty and social exclusion.33 ‘Segregation’ was 
defined by a three-category scale of interviewer’s per-
ception. However, it is very difficult to define segrega-
tion in terms of concentration of the target population. 
Concentrations of 5–10 % can reflect a rather dispersed 
population in an urban area, or a highly segregated 
settlement within a rural area. Research in this area 
remains qualitative or based on non-probability sam-
pling. Based on the assumption that target populations 
below 5–10 % do not differ significantly from those 
just above this threshold, the survey aimed rather for 
national coverage, to represent the diversity of popu-
lations in a country, and aimed in particular to include 
rural areas.34 Given the mostly high coverage of Roma 
respondents, the assumption holds that the majority 
of Roma live in areas of higher concentration, and the 
inclusion of very low concentrated areas may not affect 
the results significantly. Further research is necessary 
on a possible bias resulting from non-coverage, and 
on possible changes in processes of self-identification 
based on ethnicity.

To achieve national coverage, the sample frames for 
each country were sorted by concentration and the low-
est concentration strata excluded irrespective of region/
city. However, national coverage was not possible in 
all countries because of restrictions on the availability 
of specially trained interviewers, and costs. In Austria, 
Denmark, Finland, Greece, Luxembourg, Poland, Portu-
gal (immigrants and descendants of immigrants from 
Sub-Saharan Africa only), the Netherlands, Slovenia 
and Sweden, the fieldwork was restricted to specified 
cities/regions. The included regions/cities were in all 
instances chosen on the basis of having the largest 
numbers of the target group populations in the coun-
try. In most of these countries, the coverage loss from 
this restriction was fairly small, no more than 20 % of 
the target population. The exceptions were Greece and 
the Netherlands, where regional restrictions resulted in 
initial coverage losses of 33 % and 39 % respectively. 
Thereafter, the sample frames in these countries were 
also partitioned and low concentrated PSUs within the 
covered cities/regions were dropped from the sample, 
following in the same approach as in other survey coun-
tries, which results in a further reduction of coverage 
(see Table 4.12 for details).

Table 4.12 presents the sampling methods used in each 
country, the number of interviews permitted per house-
hold, the concentration cut-offs set when selecting 
areas (where applicable) and the population coverage 
that was achieved after exclusion of low-concentration 
strata as well as regions/cities where the population 
size of the target groups was small. Details of any over-
sampling undertaken are provided later in this section.

32  In the case of 5 % concentration of the target population, 20 
addresses must be successfully screened before 1 eligible 
respondent can be reached. Given a successful screening 
rate of 80 % with at least three contact attempts and a 50 % 
response rate, it means that 50 households/addresses need 
to be contacted for a single interview. In practice, cut-offs 
were set at between 2 % and 20 % concentration of the 
target population (see Table 4.12).

33  See FRA (2014).
34  Further research is needed to define and identify segregated 
areas and to confirm this assumption. Furthermore, non-
coverage weights were discussed and a decision was taken 
not to apply such weights due to unavailability of relevant 
data. 

44

Table 4�12: Overview of EU-MIDIS II sampling methods, by country and target group

Country Target 
group*

Sampling method

Maximum 
number of 
interviews 
per household

Target 
sample 
size (N)

400 

Cut-off level and 
selection criteria

Areas with > 7.5 % 
concentration 

1

2

2

2

1

AT

SSAFR

TUR 

BE

NOAFR 

TUR 

BG

ROMA

CY

ASIA

CZ

ROMA

Address register 
with FEa (2a: 
indirect,b multi-
stage stratified) 
and location 
sampling (3)
Address register 
with FE (2a: 
indirect, multi-
stage stratified)
Address register 
with FE (2a: 
indirect, multi-
stage stratified)
Address register 
with FE (2a: 
indirect, multi-
stage stratified)
Random route 
with ACSd (2b: 
indirect, multi-
stage stratified)
Location sampling 
(4) + Random 
route with FE (2b: 
indirect, multi-
stage stratified)
Random route (2b: 
indirect, multi-
stage stratified)

DE

SSAFR

Onomastic, multi-
stage stratified 
+ referrals (2c)

TUR 

Onomastic, multi-
stage stratified 
+ referrals (2c)

DK

SSAFR

TUR 

Location 
sampling (3)
Register (2: 
direct, simple 
random sample)

550

700

700

1,050

400

1 (location 
sampling), 2 
(random walk)

1

2

2

1

2

850

500

880

400

450

EE

RUSMIN Random route (2b: 

1

400

indirect, multi-
stage stratified)

Sampling 

Population 
coverage after 
cut off and 
exclusion rules 

35 % of the 
Address 
register; 
with location 
sampling up 
to 69 %c
49 %

Areas with > 5 % 
concentration

Areas with > 9.5 % 
concentration

55 %

Areas with > 8.5 % 
concentration

51 %

Areas with > 10 % 
concentration and 
areas with > 20 
Roma households
Areas with > 2.7 % to 
> 7.4 % concentratione

Areas with > 139 
Roma persons (~ 30 
Roma households) 
and, among areas 
with 139–500 Roma 
persons, areas with 
> 10 % concentration
Municipalities with > 5 
onomastic addresses or 
> 40 when no statistical 
twin in the same 
stratum (BIK/Nielsen)
Municipalities with > 5 
onomastic addresses or 
> 40 when no statistical 
twin in the same 
stratum (BIK/Nielsen)
Most populous areas 
selected: Aalborg, 
Århus, Esbjerg, 
Frederiksberg, Holbæk, 
Copenhagen, Odense, 
Rødovre, Roskilde, 
Slagelse, Viborg
Areas with > 30 % 
concentration

At least 70 %

43 % for 
Random route; 
with location 
sampling up 
to 89%
79 %

75%f with 
referrals

97 % with 
referrals

Up to 63 %

60 %

73 %

45

Second European Union Minorities and Discrimination Survey – Technical report

Country Target 
group*

Sampling method

Maximum 
number of 
interviews 
per household

Target 
sample 
size (N)

Cut-off level and 
selection criteria

EL

ROMA

SASIA

ES

ROMA

NOAFR 

FI

SSAFR

Random route (2b: 
indirect, multi-
stage stratified)

Random route 
with FE (2b: 
indirect, multi-
stage stratified)
Random route (2b: 
indirect, multi-
stage stratified)
Random route 
with FE (2b: 
indirect, multi-
stage stratified)
Register (2: 
direct, simple 
random sample)

FR

SSAFR

NOAFR

HR

ROMA

HU

ROMA

IE

IT

SSAFR

SSAFR

NOAFR 

SASIA

Random route 
with ACS (2b: 
indirect, multi-
stage stratified)
Random route 
with ACS (2b: 
indirect, multi-
stage stratified)
Random route 
with ACS (2b: 
indirect, multi-
stage stratified)
Random route 
with ACS (2b: 
indirect, multi-
stage stratified)
Address register 
with FE (2a: 
indirect, multi-
stage stratified)
Random route 
with FE (2b: 
indirect, multi-
stage stratified)
Random route 
with FE (2b: 
indirect, multi-
stage stratified)
Random route 
with FE (2b: 
indirect, multi-
stage stratified)

1

2

1

2

1

2

2

1

1

2

2

2

2

500

500

700

750

500

500

1,100

500

Selection of regions 
of Attica, Western 
Greece, Central 
Macedonia, Thessaly
Largest region selected, 
Attica (covered 67 
%) and areas > 5 % 
concentrations
Areas with > 200 
Roma households and 
< 10 % concentration
Areas > 3.5 % 
concentration

Selected largest 
cities of the Helsinki 
metropolitan area, as 
well as Oulu, Tampere, 
Turku and Vaasa (total 
coverage of 86 %)
Areas with > 5 % 
concentration

Areas with > 10 % 
concentration

Areas with > 200 
Roma persons (~ 40 
Roma households)

1,050

Areas with > 10 % 
concentration

400

500

700

500

Areas with > 7.5 % 
concentration

Largest regions 
selected (covered 
98 %) and > 2.75 % 
concentration
Largest regions 
selected (covered 
98 %) and > 3.1 % 
concentration
Largest regions 
selected (covered 
98 %) and > 5.75 % 
concentration

Population 
coverage after 
cut off and 
exclusion rules 

64 %

21 %

65 %

55 %

17 %g 

63 %

57 %

68 %

61 %

32 %

38 %

41 %

38 %

46

Country Target 
group*

Sampling method

Maximum 
number of 
interviews 
per household

Target 
sample 
size (N)

Cut-off level and 
selection criteria

LT

RUSMIN Random route 

1

400

with FE (2b: 
indirect, multi-
stage stratified)

LU

SSAFR

Quota sampling (4)

1

400

LV

RUSMIN Random route (2b: 

1

550

indirect, multi-
stage stratified)

Location 
sampling (4)
Location sampling 
(4) + Address 
register with FE 
(2a: indirect, multi-
stage stratified)
Location sampling 
(4) + Address 
register with FE 
(2a: indirect, multi-
stage stratified)
Location sampling 
(4) + Address 
register (2: 
direct, simple 
random sample)
Random route (2b: 
indirect, multi-
stage stratified)
Random route 
with FE (2b: 
indirect, multi-
stage stratified)
Random route 
with ACS (2b: 
indirect, multi-
stage stratified)
Location 
sampling (3)

MT

NL

SSAFR

NOAFR 

TUR 

PL

RIMGR

PT

ROMA

SSAFR

RO

ROMA

SE

SSAFR

TUR

Location 
sampling (3)

1

1 (location 
sampling), 
2 (address 
register)

1 (location 
sampling), 
2 (address 
register)

1 (location 
sampling), 
2 (address 
register)

1

2

1

1

1

400

600

650

400

500

500

1,450

400

400

Sampling 

Population 
coverage after 
cut off and 
exclusion rules 

53 %

Up to 95 %h

56 %

Up to 100 %i 

43 % (address 
sample), 
up to 53 % 
(with location 
sampling)
43 % (address 
sample), 
up to 50 % 
(with location 
sampling)
Registry, 37 %; 
with location 
sampling, up 
to 45 (based 
on census)
70 %

41 %

64 %

Most concentrated 
areas (Visagino 
sav., Klaipėdos m. 
sav., Zarasų r. sav., 
Švenčionių r. sav., 
Vilniaus m. sav.) 
(covered 63%) & 
>10 % concentration
Largest areas selected 
(centre including 
Luxembourg City, 
North and South)
Largest cities selected 
(Riga, Dugavpils, 
Jelgava, Liepaja and 
Rezekne) (covered 
65 %) and > 30 % 
concentration
N/A

Largest areas selected 
(Amsterdam, The 
Hague, Rotterdam, 
Utrecht) (covered 
61 %) and > 7.5 %
Largest areas selected 
(Amsterdam, The 
Hague, Rotterdam, 
Utrecht) (covered 
56 %) and > 5 %
Largest areas selected 
(Gdańsk, Leśnica, 
Lublin, Poznan, 
Warsaw, Wrocław) 

Areas with > 60 
Roma households and 
> 5 % concentration
Largest region selected 
(Lisboa) (covered 
76 %) and > 10 % 
concentration
Areas with > 30 Roma 
households or > 10 % 
concentration

Largest areas selected 
(Göteborg, Malmö, 
Örebro, Jönköping, 
Stockholm, Uppsala 
and Umeå)

Up to ca. 51 % 
(based on 
figures for all 
of Africa)
Up to 62 %

47

Second European Union Minorities and Discrimination Survey – Technical report

Country Target 
group*

Sampling method

Maximum 
number of 
interviews 
per household

SI

RIMGR

Population register 
(2: indirect, 
unclustered)

2

SK

ROMA

UK

SSAFR 

SASIA

1

2

2

Random route (2b: 
indirect, multi-
stage stratified)
Address register 
with FE (2a: 
indirect, multi-
stage stratified)
Address register 
with FE (2a: 
indirect, multi-
stage stratified)

Target 
sample 
size (N)

400

1,150

700

700

Cut-off level and 
selection criteria

Largest cities (Celje, 
Koper/Capodistria, 
Kranj, Ljubljana, 
Maribor, Nova Gorica, 
Novo Mesto and 
Velenje) and rural 
areas (Celje, Koper/
Capodistria and 
Velenje) (covered 55 %)
Areas with > 30 Roma 
households or > 10 % 
concentration
Areas with > 5.75 % 
concentration

Areas with > 16 % 
concentration

Population 
coverage after 
cut off and 
exclusion rules 
30 %j

75 %

60 %

60 %

Notes:  N/A = not applicable; ACS=adaptive cluster sampling; FE=focused enumeration.  

*ASIA, immigrants and descendants of immigrants from Asia; NOAFR, immigrants and descendants of immigrants from 
North Africa; RIMGR, recent immigrants; ROMA, Roma; RUSMIN, Russian minority; SASIA, immigrants and descendants 
of immigrants from South Asia; SSAFR, immigrants and descendants of immigrants from Sub-Saharan Africa; TUR, 
immigrants and descendants of immigrants from Turkey.  
a Focused enumeration, a method of initial screening of the eligibility of households via proxy.  
b ‘Indirect’ sampling refers to the process whereby a geographic area was first selected and addresses therein sampled 
and issued to interviewers to conduct screening for the target group. ‘Direct’ means that individuals believed to be 
eligible were sampled from a population register, which contained information on eligibility.  
c The coverage figures provided for location sampling assume that all target group members living in the covered areas/
cities could have been selected at one of the locations included in the fieldwork. In practice, the coverage is likely to be 
lower, as not all target group members living in selected areas would visit any of the locations. However, we could also 
expect that people who live in other areas, not included in the coverage calculations, would visit the locations.  
d Adaptive clustering, a method of screening adjacent households if a household is eligible, assuming non-random 
clustering of target groups.  
e The sample was planned and selected based on data of South and South Eastern Asians, and a 5 % concentration cut-off 
level. However, in error, fieldwork was conducted with Asians as the target group. This meant that in reality the effective 
cut off was 7.4 %, as all areas with concentrations of Asians above this level were included in the sample frame prior to 
selection. In addition, 21 out of 60 PSUs with Asian concentrations between 2.7 % and 7.4 % were covered.  
f The coverage of the onomastic sample in Germany was particularly low as cases were sampled from the telephone 
directory following an onomastic process to identify addresses likely to contain members of the target group based 
on the name listed. Coverage losses were the result of (i) households that are mobile-only and so do not have a phone 
number listed in the telephone directory, (ii) households with landlines that choose not to list their phone numbers in the 
telephone directory and (iii) target group members not identified by the onomastic process because they do not have 
a common Sub-Saharan African/Turkish surname. This resulted in coverage of 9 % for immigrants and descendants of 
immigrants from Sub-Saharan Africa, and 16 % for immigrants and descendants of immigrants from Turkey. In addition, 
further coverage losses occurred when dropping settlements with small numbers of addresses (25 % for immigrants 
and descendants of immigrants from Sub-Saharan Africa, and 3 % for immigrants and descendants of immigrants from 
Turkey). It can be expected that final coverage was significantly higher than the numbers for onomastic process suggest 
as referrals were also conducted (which could include individuals not identified by the onomastic sample provider) with 
a wide radius of 30 km from each onomastic address. Furthermore, the addresses identified through the onomastic 
procedure covered all regions of Germany. The coverage in the table is a rough estimate of the survey company taking 
the referrals into account.  
g The coverage in Finland includes the loss of 80 % of the planned level of coverage due to losses after sample selection 
at the telephone matching stage, where it was not possible to find telephone numbers for all selected individuals. The 
areas of the country covered included 86 % of the target group.  
h For Luxembourg, where quota sampling was used, we make a similar assumption to that for location sampling groups: 
that all target group members living in the coverage areas could have been sampled. No data were available to estimate 
coverage at a more detailed geographical level.  
i Full coverage was theoretically possible in Malta because the locations selected could be reached easily from any part of 
the island. In practice, not all members of the target group will have visited one of the locations during the fieldwork. This 
may have been a particular issue for female respondents, who appear to be under-represented in the sample.  
j In addition to the coverage losses originally estimated, further loss of coverage resulted from the population register 
being able to deliver details only for those individuals who had agreed to be contacted for research purposes (54 % of 
the target population).
Source:  FRA, EU-MIDIS II 2016

48

4�5�  Direct single-stage 
sampling (design 1)

Direct  sampling  was  used  in  Denmark,  Finland  and 
Poland, where it was feasible given the regional clus-
tering of the target group (in particular in Finland). The 
addresses selected from the register are all believed 
to contain eligible members of the target group. In 
Poland, the register used did not cover the entire target 
group, because persons without a residence permit and 
naturalised immigrants were excluded to some extent 
(however, the sample was supplemented with location 
sampling, see design 3).

Furthermore, in Denmark and Finland, telephone pre-
recruitment was used, which meant that in most cases 
only one visit was required to each household after 
the appointment was made on the phone. This meant 
that the unclustered sample was still feasible, in terms 
of interviewer travel time and related costs, in com-
parison with the face-to-face recruitment survey, which 
required multiple contacts. Telephone recruitment is the 
preferred recruitment method in the majority of the 
Nordic countries. The majority of people in these coun-
tries live in apartment blocks with locked outer doors, 
which makes access difficult without prior appoint-
ments.  The  country  reports  provide  further  details 
concerning the sampling in the respective countries.

4�6�  Multi-stage area sampling 

(designs 2a, 2b, 2c)

This sampling approach was used in the majority of 
countries (21 out of 28) and involved the following broad 
stages, with some adaptations by country:

1.  The sample frame was acquired on the most 

detailed regional level available and if necessary 
aggregated to a suitable geographic level that 
could be used as PSUs.

2.  For each country and target group the sample 

design parameters were agreed with the national 
survey companies. These included sample size, 
target cluster size (number of interviews per 
PSU, set at 10 in the majority of PSUs), target 
response rate, expected number of interviews 
to be achieved on average per interviewed 
household (for target groups where two 
interviews were permitted), and – if data on first-
generation immigrants only were available – the 
estimated multiplication factor to apply to the 
population statistics of the first generation to 
estimate the size of the full target group (first- 
and second-generation respondents).

3.  For each PSU, the group concentrations were 

estimated (ideally, at household/address level, 
but if this was not possible then at individual 
adult (16+) level, and if not possible finally 
at individual (all ages) level). If necessary, 
estimates were refined based on local/expert 
information – which was required particularly 
in the case of Roma target group countries, or 
by apportioning target group numbers from 
data which was available only for higher-level 
geographic units. Specifically:
 • In  Croatia,  the  Czech  Republic,  Greece,  Portu-
gal,  Romania,  Slovakia  and  Spain  (Roma  target 
group), Estonia and Lithuania (Russian minority), 
and Cyprus (immigrants and descendants of im-
migrants from Asia), data were available at set-
tlement or municipality level only. It was there-
fore necessary to demarcate smaller areas to be 
used  as  PSUs,  usually  using  maps,  and  then  to 
estimate the target group concentrations within 
the  smaller  areas.  This  meant  apportioning  the 
target  group  population  to  these  areas  evenly, 
or using local knowledge to identify areas where 
the  target  group  lives;  see  country  reports  for 
further details.

 • In  France,  commune-level  data  were  available 
for the survey’s target groups, while data on the 
number  of  all  foreign-born  people  (i.e.  born  in 
any  country  outside  of  France)  were  available 
at  a  more  detailed  geographical  level  suitable 
for PSUs. In this instance, it was possible to set 
the  target  group  numbers  in  proportion  to  the 
foreign-born  population  to  obtain  a  reasonable 
estimate  of  target  group  concentrations  at  PSU 
level.

 • In  Latvia,  a  similar  solution  was  used  to  that 
in  France.  Target  group  data  were  available  at 
municipality  level  only.  The  numbers  in  each 
electoral  district  were  therefore  estimated  by 
apportioning  the  target  group  based  on  elec-
tion results for the proportion voting for the two 
main Russian minority parties.

 • In Slovenia, the statistical office provided a ran-
dom sample with an oversample of 20 % of the 
addresses to be eligible.

4.  If over-sampling was applied, the sample frame 
was divided into target group concentration 
strata. For each target group, the samples were 
divided into five strata, with the concentration 
levels that defined the stratum boundaries set 
at a level that divided the target population 
into groups of roughly equal size. The lowest 
concentration stratum (proportion of the target 
population ranging from zero to the concentration 
cut-off specified in Table 4.12) was excluded 
from the survey. If multiple target groups were 
to be selected in a country from the same 

Sampling 

49

Second European Union Minorities and Discrimination Survey – Technical report

frame, strata were specified across all target 
groups (interlocking cells), thereby increasing 
the number of strata. The sample design was 
then optimised to deliver the required sample 
size with maximum sample efficiency (see next 
section on oversampling)

5.  Selection of the number of PSUs specified 

by the sample design, with stratification by 
concentration strata (disproportionate, if used), 
region and urbanisation (proportionate), and 
the selection usually made with probability 
proportional to size (PPS) and sometimes with 
equal probability within each concentration 
stratum (if used). The size measure used in PPS 
was the total number of addresses in each PSU, 
or the number of persons if number of addresses 
was not available, i.e. including eligible and 
ineligible, to counter the probability of selection 
at the next stage. Twice the number of PSUs 
as the number required were selected to have 
a reserve sample of PSUs in case needed. Where 
possible, each main sample PSU had a ‘matched’ 
reserve PSU. Implementing this involved selecting 
reserve PSUs from the same stratum and of the 
nearest size to their paired main PSUs.

6.  Systematic selection of the specified 

number of addresses in each PSU, from an 
address or individual register, random route, 
or onomastic identification.

Table 4.13 provides details on the sample designs by 
country and target group, based on a multi-stage clus-
tered sampling methodology used in 21 of the 28 EU 
Member States included in EU-MIDIS II. This includes 
the method used to select the PSUs, the expected and 
actual average target group concentration across the 
PSUs selected, and the estimated sample efficiency35 
due to over-sampling higher concentration strata where 
applicable. In countries where oversampling was not 
used, the table indicates ‘N/A’, not applicable. The vari-
ation of efficiency between countries and target groups 
was highly dependent on the dispersion of a popula-
tion, and on the screening and interviewing costs in the 
respective country. Efficiency was higher when target 
groups were large and had a higher proportion in more 
concentrated areas, and lower when target groups were 
small and living rather dispersed.

35  The effective sample size is calculated as the (sum of 

weights squared / sum of squared weights)/N. The sample 
efficiency figures provided in Table 4.13 account for the 
corrective weights required to re-balance a sample that was 
disproportionally sampled by concentration with a maximum 
of 100 % efficiency in a self-weighting design. They do not 
account for further efficiency losses arising from other parts 
of the weighting (Chapter 7 provides details). 

50

As Table 4.13 shows, the estimated and actual target 
group  concentrations  differ  in  many  countries.  For 
the majority of the Roma and Russian minority target 
groups, the fairly wide discrepancies between esti-
mated and actual concentrations in some countries 
are not surprising because the expected concentrations 
were based on fairly rough estimates. Most of the actual 
field concentrations for Roma were better than those 
estimated, as identification of the selected sub-areas 
in the sample drew on local knowledge, whereas the 
concentration assumptions were based on data, which 
gave the average across a much larger area. These and 
other differences sometimes affected the number of 
interviews that were delivered, as discussed later in 
this section (see ‘fieldwork outcome’).

Oversampling concentration strata

In most countries, EU-MIDIS II sampled PSUs with larger 
numbers of members of the target group populations 
with higher probability, to increase cost efficiency and 
feasibility of the samples reaching elusive populations. 
This approach requires weighting, and decreases the 
sample efficiency. For a given resource outlay, meas-
ured as the number of addresses selected and issued to 
interviewers for screening, oversampling was optimised 
to maximise sampling efficiency and use of resources.

The sample frames were first partitioned into multiple 
target group concentration strata. A concentration cut-
off was set for the target group(s); this was the thresh-
old for exclusion from the sample. The coverage figures 
provided in Table 4.13 give the proportion of the target 
group living in areas above the threshold. For a coun-
try with a single target group, optimisation was then 
achieved by selecting the sample of addresses within 
each concentration stratum using a sampling fraction, 
calculated as a function of the square root of its target 
group concentration, given by the formula below:

r
h

×=
r

×
N
D
h
∑ ×
N

h

h

a
h
D

a
h

where: 

r1 =  
0

h =∑
r

h

r

and: rh = addresses to sample per stratum h;

r = target number of addresses to sample overall;

Nh = concentration percentage of the target group/com-
bined target groups (if multiple) in stratum h;

D h  =  number  of  addresses/households  overall 
in stratum h;

a = power adjustment; for optimal allocation a = 0.5.

Table 4�13: Details of multi-stage clustered sample designs

Sampling 

PSU selection 
method

Expected (actual) concentration in the 
sampling frame/selected areas (%)

Sample efficiency (accounting 
for oversampling) (%)

Country

AT

BE

BG

CY

CZ

DE

EE

EL

ES

FR

HR

HU

IE

IT

LT

LV

NL

PT

RO

SK

UK

Target 
group*

TUR

NOAFR 

TUR

ROMA

ASIA

ROMA

SSAFR

RUSMIN

ROMA

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

ROMA

Equal probability

NOAFR 

SSAFR

NOAFR

TUR

SASIA 

ROMA

ROMA

SSAFR

SSAFR

NOAFR 

SASIA

RUSMIN

RUSMIN

NOAFR 

TUR

ROMA

SSAFR

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

PPS

ROMA

Equal probability

ROMA

SSAFR 

SASIA

PPS

PPS

PPS

16 (14)

30 (15)

30 (15)

36 (53)

11 (10)

24 (66)

Close to 100 (93)

64 (84)

74 (67)

28 (21)

16 (9)

39 (32)

39 (32)

Close to 100 (93)

16 (14)

57 (56)

38 (69)

19 (7)

20 (16)

20 (16)

20 (16)

18 (15)

53 (66)

34 (17)

34 (17)

90 (74)

24 (31)

18 (29)

35 (58)

35 (23)

35 (23)

73

81

68

97

49

N/A

N/A

96

N/A

N/A

62

69

69

N/A

42

N/A

74

50

31

34

33

N/A

88

53

49

N/A

82

N/A

N/A

54

49

Notes:  N/A=not applicable.  

*ASIA, immigrants and descendants of immigrants from Asia; NOAFR, immigrants and descendants of immigrants from 
North Africa; PPS, probability proportional to size; ROMA, Roma; RUSMIN, Russian minority; SASIA, immigrants and 
descendants of immigrants from South Asia; SSAFR, immigrants and descendants of immigrants from Sub-Saharan Africa; 
TUR, immigrants and descendants of immigrants from Turkey.

Source:  FRA, EU-MIDIS II 2016

If required, the power adjustment was increased itera-
tively so that the sample design delivered the target 
number of interviews (given the fieldwork assump-
tions) from the target number of sampled addresses.

In countries with multiple target groups, where the 
selection of PSUs was made once and from a single 

sample  frame,  the  solution  would  deliver  numbers 
of interviews with each group in proportion to their 
relative numbers on the sample frame. Usually, these 
numbers did not meet the target sample sizes for each 
group,  necessitating  a  further  adjustment  to  force 
the sample towards PSUs with relatively more of the 
under-represented target group. This was achieved by 

51

Second European Union Minorities and Discrimination Survey – Technical report

assigning an adjustment weight to the concentration of 
each target group, for example for two target groups:

Random route

Nh = N1h × W1 + N2h × W2

where: N1 = concentration percentage of target group 1;

N2 = concentration percentage of target group 2;

W1  =  weighting  adjustment  factor  applied  to 
target group 1;

W2  =  weighting  adjustment  factor  applied  to 
target group 2.

As  with  the  single  country  solution,  the  weighting 
adjustment factors and power adjustment could be 
changed iteratively to deliver the numbers required 
(see Chapter 7 on Weighting).

Selection of addresses

The number of addresses selected and issued to inter-
viewers in each PSU was set in accordance with the 
sample design (typically an equal number of addresses 
were issued in each concentration stratum, aiming to 
deliver an average of 10 interviews across the stratum). 
Interviewers had to visit each selected address three 
times and try to make contact with somebody from the 
household who could provide details about the persons 
living there, to determine whether or not any of them 
would be eligible to take part in the survey. If several eli-
gible persons were identified, interviewers proceeded 
with randomised selection of person(s) to interview.

If an address register was available for use (the pre-
ferred approach), the addresses were selected system-
atically from the full PSU address listing, so that they 
were spread across the full area of the PSU.

Random route approach to selection of households 
was required in 15 of the 21 EU Member States, using 
multi-stage area sampling, because it was not possi-
ble to access an address register with country of birth 
information. Here, one or more random starting point(s) 
were first selected in each of the PSUs (see country 
reports for the methods used). The country coordinat-
ing team centrally determined the number of starting 
points to select, based on the size of the PSU; if the sam-
pling fraction (number of addresses in the PSU divided 
by the number of addresses to select) was less than 1 
in 20, a single starting point was selected; if 1 in 20–30, 
two starting points were selected; if 1 in 30–40, three 
starting points; and, if 1 in 40 or more, four starting 
points. Where multiple starting points were selected, 
the number of addresses to contact when proceeding 
with the random route method from each starting point 
were divided evenly between the starting points. This 
was done to ensure coverage of a wider area of larger 
PSUs by the random route.

Proceeding with the random route approach from the 
selected starting point, the addresses to contact were 
then selected as every ‘nth’ address from the speci-
fied starting point, using a standard random route pro-
cedure. A pre-determined random route interval was 
provided for each PSU. The interval n to use in select-
ing every ‘nth’ address was set at between 2 and 10, 
depending on the size of the PSU and the number of 
addresses to contact (with the objective that the con-
tacted addresses should be spread across the area of 
the PSU, or as much of it as possible). Due to the screen-
ing procedure needed to determine whether house-
holds are eligible, the random route sampling procedure 
was conducted at the same time and by the same inter-
viewer who was responsible for the survey interview.

52

Rules for random route in EU-MIDIS II

– 

– 

 Office buildings, uninhabited/abandoned housing units, schools, hospitals, public buildings, factories, workshops, 
supermarkets and shops are excluded from the count.

 Facing the street, with your back to the starting point, you should go to the left. Please ring the doorbell of every 
nth housing unit in urban PSUs (i.e. houses or apartments; rules concerning rural areas are explained below). 
When turning at the end of the street, do not stop counting housing units. In the example below, an interval 
of 5 has been used. We will use this example throughout the instructions. Your fieldwork manager will inform 
you of the interval you should use for your PSU.

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

Start

1st 
contact

2nd 
contact

3rd 
contact

4th 
contact

– 

 At crossroads, please turn left. After turning left, keep going on the left pavement till the next crossroads with 
a right turn. At this crossroad, turn right and keep going on the right pavement. At the third crossroads it is back 
to the first procedure – turning left and continuing on the left pavement:

– 

 If there is no other street (e.g. a park or construction site) at the end of the street, then return to the corner of 
the street where you changed direction and turn left if you previously turned right, or vice versa. The same 
procedure should be followed when you come to the border of the PSU.

– 

 At a square or roundabout, please follow this procedure:

Change pavement
and enter street

Right – right pavement

Sampling 

53

Second European Union Minorities and Discrimination Survey – Technical report

– 

– 

 In the case of multiple housing units (e.g. apartments), consider every apartment as a separate housing unit 
and start at the top of the building, working your way down. You should continue using the same selection 
interval in multiple apartment buildings.

 If there is no easy access to the building (e.g. there is an entryphone/housephone), count the doorbells start-
ing at the top left end and ending at the bottom right. When leaving the apartment building, continue with 
the first housing unit encountered next to the building.

– 

 Rural settlements may have a different layout. You should also use whatever interval is advised by your 
fieldwork manager in these areas. Most typically, they could be simplified as having one of these layouts:

(1)  Village next to road – Go to one end of the village and walk to the other end. Choose one pavement 

and count every three housing units. Arriving at the other end of the village, cross the street and turn 
around, following the same procedure.

(2)  Star-shaped village – There is often an obvious village centre, with several streets leading outwards 
from the village centre. Start at the central crossroads, selecting a direction and a pavement. Count 
every three housing units. At the end of the road, cross the street and walk in the opposite direction. 
At the starting point, take the next street and follow the same procedure.

Counting the interval

Beginning from the starting point address, you should count the interval and visit every nth address. From that 
address, you should count the interval and call at the next nth address. In streets with both single-occupancy 
addresses (e.g. 49 and 51 Talbot Road) and multiple-occupancy addresses (Flats 1, 2, 3, 4, 5 and 6 at 53 Talbot 
Road), the flat numbers should contribute towards counting the sampling interval.

54

For example, in an urban area, if the last visited address was 49 Talbot Road, the counting interval should include 
1) 51 Talbot Road; 2) Flat 1, 53 Talbot Road; 3) Flat 2, 53 Talbot Road; 4) Flat 3, 53 Talbot Road; 5) Flat 4, 53 Talbot 
Road. Flat 4 at 53 Talbot Road would be the next address to visit.

Usually one dwelling stands for one household. Sometimes several persons live together in one dwelling inde-
pendently. They share facilities or have separate facilities but they do not share the expenses. For example, three 
workers who live in the same apartment, paying rent to the landlord, but not sharing any other expenses; or two 
families living in the same apartment, sharing facilities, but not sharing the expenses. In such cases, you must select 
the household first. You should list, in alphabetical order, the names of the oldest household members of each 
household (surname and name). Then you should select the household whose name comes first in alphabetical order.

When you make contact with an occupant at an address, you will need to identify the correct respondent(s) 
to interview, by:

 • collecting the details of the eligible household members, and

 • using the respondent selection method to select the respondent(s) to be interviewed.

Source:  FRA, EU-MIDIS II 2016

Focused enumeration

To increase the efficiency of the screening process, 
focused enumeration was employed in PSUs with lower 
estimated concentrations of population (usually below 
25 %). This method was originally developed in the 
United Kingdom, and was already used in EU-MIDIS I 
as well as in FRA’s Roma survey 2011. Addresses are 
sampled – from address registers or by random route – 
in clusters of five neighbouring households. The middle 
address was designated the ‘core address’ and initial 
contact initiated at this address. If contact was success-
ful, screening for eligible members of the target group 
could be conducted by proxy (by asking core address 
members about country of birth of their neighbours) for 
the other four addresses in the cluster, always moving 
two addresses forward and backwards (see Figure 4.1).

If the interviewers were unable to contact anyone from 
the core address or the proxy screening was unsuccess-
ful (refused or the eligibility of neighbours not known), 

Figure 4�1

the interviewer was required to attempt contact at an FE 
address and ask the screening questions directly. Proxy 
screening, however, could be conducted at any of the 
addresses in the cluster. If eligibility was established 
by proxy, then the interviewer was to attempt contact, 
confirm eligibility and attempt an interview.

Although a final screening outcome was required for 
all addresses, core and FE, the ability to proxy screen 
meant that in most cases the interviewer would not 
need to contact all of them directly. This increased the 
time efficiency of the screening. The interviewer was 
not permitted to include any addresses outside the clus-
ters in the sample, so the address probabilities were 
known and the same as they would be without FE.

A section of the contact sheet was used to manage the 
FE addresses and record outcomes. In address register 
countries, the additional four addresses around the core 
address were pre-printed onto the core address contact 
sheet. In random route countries, the interviewer wrote 

FE
Address
(B2)

FE
Address
(B1)

CORE
Address

FE
Address
(F1)

FE
Address
(F2)

Forward direction = towards the next CORE address

Sampling 

55

Second European Union Minorities and Discrimination Survey – Technical report

down the exact core and FE addresses on the contact 
sheet before attempting contact and proxy screening.

In EU-MIDIS II, FE was usually applied in PSUs where 
the expected concentration of the target group(s) was 
below 25 %. A larger number of addresses were issued 
in these PSUs than in PSUs with higher concentration 
of  target  groups,  so  boosting  the  efficiency  of  the 
screening was helpful. Furthermore, the FE procedures 
required administering five addresses on one contact 
sheet: the core address plus two FE addresses on either 
side. FE reduced the quantity of paper interviewers had 
to carry and manage in less concentrated PSUs. For 
example, in the least concentrated stratum in Austria, 
325 addresses were sampled per PSU, so FE meant that 
in these PSUs 65 contact sheets needed to be issued 
instead of 325.

Adaptive cluster sampling

An alternative to FE to increase the efficiency of the 
screening process is Adaptive Cluster Sampling (ACS). 
ACS makes the assumption that people from the same 
target group are more likely to live close to each other 
(in neighbouring addresses), at least to some degree. 
Professor Vijay Verma developed ACS to improve the 
efficiency of screening rare populations and to fully 
implement the principle of self-identification in data 
collection. Self-identification is a particularly sensitive 
issue for Roma, so it was agreed that ACS would be 
tested for this group. The method was developed and 
first applied in study on child labour by the International 
Labour Organization (ILO). The ILO further elaborated 

Figure 4�2: Random route – adaptive cluster sampling (ACS)

and described it in a book on sampling elusive popula-
tions.36 For EU-MIDIS II, the methodology was further 
developed in close consultation with Professor Verma, 
who acted as the external senior sampling expert.

The efficiency of FE comes from allowing the inter-
viewer to screen neighbouring addresses by proxy, 
meaning that they do not need to contact all issued 
addresses. Unlike ACS, FE’s efficiency gains are not 
dependent on whether or not the target groups live 
close  to  each  other.  When  FE  was  applied  in  EU-
MIDIS II to target groups other than Roma, the screen-
ing questions were based on the country of birth and/
or country of birth of parents, and not on questions 
requiring ethnic identification.

The first testing of ACS in the EU-MIDIS II pilot among 
the Roma target group found ACS to be a potentially 
promising method for increasing the sampling effi-
ciency for this group. Therefore ACS was adopted for 
use  in  the  main  data  collection  stage  in  PSUs  with 
lower estimated concentrations (usually below 25 %) 
in several of the Roma target group countries (Bulgaria, 
Croatia, the Czech Republic, Hungary, Romania and Slo-
vakia), as well as in France. FE was not possible in France 
because legislation forbids recording the ethnicity of 
another person. Use of ACS was initially also envisaged 
for the Russian minority groups in the Baltic countries, 
but the intention to use ACS in Estonia, Latvia and Lithu-
ania was discarded after the pilot suggested ACS as 
being too complicated.

CORE
ADDRESS

X

X

ACS
ADDRESS
F4 

ACS
ADDRESS
F3 

X

X
ACS ADDRESS F2:
IF not eligible GO
TO next CORE
ADDRESS

X
ACS ADDRESS F1:
IF not eligible GO
TO next CORE
ADDRESS

X
CORE ADDRESS IF
eligible GO TO
ACS ADDRESS F1

36  See Verma, V. (2014).

56

Implementing ACS involved issuing a set number of 
‘core’ addresses for contact in PSUs with lower con-
centrations. The number of core addresses was set at 
60 % of the number of addresses that would otherwise 
be required to deliver the same number of interviews 
by direct screening. If a core address was found to 
contain eligible members of the target group(s), the 
interviewer was instructed to screen the two neigh-
bouring  addresses  on  either  side  (designated  ‘ACS’ 
addresses) of the core address for eligible respondents. 
This ‘chain’ was continued, by adding further neigh-
bouring addresses in each direction if further eligible 
households were found. This meant that, where suc-
cessful, chains of eligible addresses were added to the 
sample. Chains were ended if the interviewer reached 
a neighbouring address that did not contain anyone 
who was eligible, or if eligibility could not be estab-
lished, or if another core address was reached. However, 
the chain could continue in the opposite direction from 
that address. The rules for establishing which addresses 
were neighbours to those already in the sample were 
based on the random route rules.

Interviewers could only include addresses in the sample 
that were either (i) a core address or (ii) a neighbour 
of an eligible address that had already been included 
in the sample. Further, interviewers were required to 
continue screening at neighbouring addresses until the 
natural termination of the chain, as outlined above. 
The probability of selection of sample addresses could 
therefore be determined by the chain length (for fur-
ther details, see Chapter 7 on Weighting). Records of 
ACS addresses were kept on contact sheets started for 
each new core address.

The efficiency gains of ACS result from having to screen 
fewer addresses for the same number of identified eli-
gible addresses in a PSU as would be identified had ACS 
not been used. This can be achieved if the proportion 
of ACS addresses that contain a member of the target 
group is higher than the proportion in the set of core 
addresses initially sampled; in other words, if mem-
bers of the target group tend to live in dwellings which 
neighbour other members of the target group.

ACS  was  applied  in  PSUs  where  the  expected  con-
centration of the target group(s) was below 25 %. In 
PSUs of higher concentration, ACS was not applied. The 
rationale for this was that in more concentrated PSUs 
the fieldwork was already sufficiently efficient, so ACS 
focused on the PSUs where it was harder to find the tar-
get group and the boost to efficiency was most needed.

Difficulties  with  implementation  were  encountered 
in  a  number  of  countries,  so  interviews  had  to  be 
deleted. There was incorrect application of the rules 
for ACS – for example, finishing chains too early or 
continuing a chain even when it had come to an end. 

In Bulgaria and Romania, ACS increased efficiency of 
screening. For example, in PSUs where ACS was applied 
at core addresses (i.e. every nth address to be con-
tacted on random route), 1,669 contacts resulted in 
124 interviews, which is a rate of 7.4 %. For the 550 
ACS  addresses  (i.e.  addresses  neighbouring  a  core 
address with eligible respondents), 251 interviews were 
obtained, which is a rate of 45.6 %. The core versus ACS 
address rates in Bulgaria were 13.4 % versus 33.2 %. 
In Hungary, the rates were the same at 26.9 %. This 
means that notwithstanding challenges of implement-
ing the procedure, the method can support surveying 
ethnic minorities through more efficient screening. 
However, it demands thorough training for interview-
ers, and some development to ensure rule adherence 
and field monitoring.

Stopping and dropping rules

Additional mechanisms were put in place to prevent 
inefficient use of fieldwork resources. As the size of the 
target group population was often based on rough esti-
mates, EU-MIDIS II adopted rules to account for unex-
pected fieldwork outcomes at the PSU level. These rules 
were only applied in exceptional cases, and the decision 
to apply them was the responsibility of the fieldwork 
management and not the interviewers themselves.

A PSU could be dropped from the sample (and a replace-
ment issued) if its initial sample outcomes did not meet 
a minimum threshold of fieldwork success, indicating 
that the PSU was highly likely to be substantially less 
concentrated  than  predicted  by  the  sample  frame. 
The threshold (termed the ‘dropping rule’) was set at 
the PSU level. If the probability that the sample frame 
concentration figure was correct fell below 10 % given 
the outcomes observed in the early stages of PSU con-
tact, the PSU could be dropped. (The probability was 
assessed based on the binomial distribution.)37 The 
dropping rule was set at a level based on the expected 
concentration of each PSU. For example, with a drop-
ping rule of 16, which equates to a concentration level 
of 25 %, the rule stated that, if after 16 successfully 
screened addresses all 16 addresses were confirmed 
as not containing a target group member, then the PSU 
could be dropped and replaced. To protect against the 
risk of dropping a highly concentrated PSU too easily, 
a minimum level of 15 was set for the dropping rule in 

37  In probability theory and statistics, the binomial 
distribution with parameters n and p is the 
discrete probability distribution of the number of successes 
in a sequence of n independent yes/no experiments, each 
of which yields success with probability p. As such, the 
dropping rule was calculated as n, given an estimated PSU 
concentration of p, and a 10 % chance that an outcome 
of n screened addresses with no member of the target 
group present, keeping the PSU if p was indeed the correct 
concentration of the PSU, or dropping it if the 10 % chance 
was not met. 

Sampling 

57

Second European Union Minorities and Discrimination Survey – Technical report

all countries except France, where a level of 10 was set. 
The individual country reports indicate how often the 
dropping rule was used.

Fieldwork in a PSU could also be stopped prematurely, 
i.e. without making all the required contacts at all issued 
addresses, if a set number of interviews (the ‘stopping 
rule’) had been achieved. This was set at three times 
the expected number of interviews in a PSU (factoring 
in expected eligibility and response rates). This was 
used to avoid overly large clusters, which could reduce 
sample efficiency, given that people living in the same 
area are more likely to resemble each other in other 
respects compared with people living in different areas. 
The individual country reports indicate how often the 
stopping rule was used.

Sampling by referrals

In Germany, an additional sampling step was used to 
boost the sample coverage. Households were contacted 
using the onomastic method, the name-based screen-
ing strategy. Where an eligible member of the target 
group was interviewed, they were asked for the contact 
details of other members of the target group who lived 
within a 30 km radius. There was no limit on the number 
of referrals that could be provided, and interviewers 
recorded up to 30, but the vast majority of respondents 
provided fewer than 10 referrals when asked.

Fieldwork was completed in full at all addresses identi-
fied through the onomastic method before referral field-
work commenced. The fieldwork managers checked all 
referral contacts, removed duplicates and provided the 
contacts to the interviewers. In PSUs where too many 
referrals had been collected in proportion to the size of 
the PSU, interviewers were given a random selection 
of the contacts.

In  addition  to  boosting  sample  coverage,  referrals 
were also expected to compensate for the potential 
bias which might be introduced through a fragmented 
coverage in the initial sample frame of telephone num-
bers with onomastic screening. Gaps in the onomastic 
screening might involve, for example:

i.  households that use only mobile telephones 
and so do not have a phone number listed in 
the telephone directory;

ii.  households with landlines that choose 

not to list their telephone numbers in the 
telephone directory; and

iii.  target group members not identified by the 

onomastic process because they do not have 
a common Sub-Saharan African/Turkish surname.

The referral process worked as planned for the sample 
of immigrants and descendants of immigrants from Tur-
key, in terms of the number of interviews achieved from 
referrals. However, referrals provided by immigrants 
and  descendants  of  immigrants  from  Sub-Saharan 
Africa provided too few referrals, so additional ono-
mastic interviews had to be conducted in the same 
PSUs already selected.

Selection of dwelling units, households 
and individuals

Interviewers visited the selected addresses and made 
contact with the household living at the address. On the 
rare occasions that more than one household lived at an 
address, one of the households was selected systemati-
cally (by listing the surnames of the oldest persons in 
each household in alphabetical order and selecting the 
first on the list). Having made contact with a household, 
the interviewer asked one adult in the household the 
screening questions to establish the household’s eli-
gibility. If more than one eligible person was living in 
the household, one or two (if permitted for the target 
group) of all eligible household members were selected 
at random for the interview. Interviewers made this 
within-household selection of respondents using a Kish 
grid printed on the contact sheet.

Managing fieldwork targets

In each issued PSU, interviewers made the required 
number  of  contacts  at  the  full  number  of  selected 
addresses.  In  other  words,  the  sample  was  fully 
exhausted. The main exceptions to this were PSUs that 
met the dropping or stopping rule (see earlier in this 
section), and those deemed unsafe for interviewers 
to work in. Given the number of sample assumptions 
that had to be realised to meet the fieldwork targets, 
and the uncertainties related to some of them (for 
example, the response rate and adjustments made to 
target group concentration estimates to account for 
second generation when using population data which 
concerned  the  first  generation  only),  mechanisms 
were put in place to manage the sample to reach the 
targets. During fieldwork, the sample was monitored 
and, once the fieldwork had progressed sufficiently, 
the final numbers of interviews were predicted based 
on the emerging response and eligibility rates. It was 
sometimes  necessary  to  adjust  the  sample  at  this 
stage  to  ensure  the  target  was  reached.  Measures 
taken included issuing reserve PSUs if the sample was 
predicted to underdeliver, as happened in Ireland and 
the United Kingdom; reducing the numbers of issued 
addresses in incomplete or unstarted PSUs if the sample 
was predicted to overdeliver, as happened in France, 
Hungary, Portugal, Romania and Spain; or changing the 
sampling approach altogether, in the Netherlands. In 
some countries, a higher than expected response rate 

58

offset the effect that lower than expected target group 
concentrations had on the sample outcomes. This was 
the case in Belgium, France, Italy and Spain. The country 
reports provide details of any adjustments made to the 
sample during fieldwork.

4�7�  Location sampling 

(design 3)

Location sampling or centre based sampling was used 
in seven EU Member States (Austria, Cyprus, Denmark, 
Malta, Poland, the Netherlands and Sweden), for one 
or more target groups in each country and sometimes 
in  combination  with  one  of  the  sampling  methods 
described  earlier  in  this  chapter  (see  Table  4.12  for 
details). Location sampling can be used as a probabil-
ity-based sampling design for dispersed groups which 
have homogenous characteristics and likelihood of high 
coverage through various locations and centres. Loca-
tions can be manifold places where the target group 
meets, such as churches, associations and public places, 
but a register can also count as a location. Location 
sampling was used in EU-MIDIS II in cases where it was 
not possible to obtain a sample frame that would allow 
random sampling of the target group using one of the 
methods described in earlier sections of this chapter.

The procedures used for location sampling in EU-MIDIS II 
are based on the methodology described by Baio et al. 
in 2011,38 further developed in consultation with Profes-
sor Gian Carlo Blangiardo, a renowned expert on loca-
tion sampling. Location sampling was implemented in 
EU-MIDIS II as follows.

Regional mapping, selection of regions 
and sample allocation

The  first  step  was  to  obtain  regional  target  group 
population statistics or estimates on the size and geo-
graphical distribution of the target populations, and to 
identify regions/cities to be included in the sample. In 
the first instance, the assessment of the geographical 
distribution of the target group was based on the main 
regions/cities where the population could be found 
(largest populations, highest densities), excluding at 
the outset regions with no or very few members of 
the target group, and justifying the choice based on 
the degree of coverage achieved.

It was considered important not to include too many 
regions because covering a large number of regions in 
sufficient detail might prove unfeasible, given the pre-
paratory effort and sample sizes per location required. 
If a region/city was particularly large it might also be 
necessary to randomly sub-sample/purposively select 

38  Baio, G. et al. (2011).

lower-level units within it such as municipalities, to 
ensure that the included areas had sufficient target 
group populations and that resources were not spread 
too thinly. In practice, it was usually the case that the 
largest  regions  were  included  and  sub-sampling  of 
municipalities was not considered necessary.

Following this approach, target sample sizes were set 
for each selected region. These were:

 • proportional to the size of the target group popula-

tion in each region; and

 • of sufficient size to ensure that adequate number of 
interviews  in  each  geographically  separate  region 
could  be  included  in  the  sample  (the  target  was 
a minimum of 30 interviews).

Listing centres in each region

Next, an extensive list of centres that could be included 
in fieldwork were identified in each region, based on 
desk research, local knowledge and consulting experts 
and NGOS and other civil society organisation. The map-
ping of centres was done based on the following criteria:

 • Any locations where people from the target group 
meet/congregate  –  these  could  be  welcome  cen-
tres, marketplaces, places of worship etc.

 • The  list  should  be  sufficiently  heterogeneous  to 
permit  encountering  diverse  members  of  the  tar-
get group. For example, it should include locations 
where  women  or  older  people  from  the  target 
group congregate/meet, as well as locations where 
men do.

 • The list should cover each of the selected regions, 
incorporating  knowledge  on  the  movements  of 
populations. For example, if there are two munici-
palities and only one has a place of worship relevant 
to the target group, this could cover both areas.

 • The list of locations should be flexible and could be 
expanded during fieldwork if other important loca-
tions were discovered, e.g. by asking members of 
the  target  population  during  the  interview  which 
other  centres  they  visited.  Extension  of  the  list  of 
centres  was  completed  early  in  the  fieldwork  pe-
riod to minimise the amount of missing information 
for weighting.

 • The  objective  was  to  reflect  a  range  of  locations 
which, taken together, increase the chances of the 
interviewers  encountering  most  members  of  the 
target group living in the selected area.

Sampling 

59

Second European Union Minorities and Discrimination Survey – Technical report

In addition, the importance of each centre was esti-
mated beforehand, broadly measured as the proportion 
of the target population that visits the centre. Given 
that this could not be estimated with a high degree 
of accuracy prior to the survey, the assessment could 
be based on the known or estimated volumes of eli-
gible people visiting the location per day during the 
hours of interviewing.

Saharan African interviews were achieved from the 
conventional sample in Austria, while it was originally 
estimated that around 50 interviews could be achieved 
this way. The main reason for this was that the four 
PSUs with higher estimated target group concentrations 
turned out not to contain the expected number of immi-
grants or descendants of immigrants from Sub-Saharan 
Africa (see country report on Austria for details).

Selection of locations for inclusion 
in fieldwork

The next step was to decide how many locations were 
required in each region (set in proportion to sample size 
and to aim for a minimum of 15 or more interviews at 
each location) and then select the locations that would 
be included, out of the longer list of possible venues. 
The proposal was to carry this out by including all loca-
tions assessed as highly important, and taking a random 
selection of other identified locations. The locations 
were also selected in view of including a variety of 
different ‘types’ of centre (e.g. place of worship, shop, 
etc.) to ensure good representation of the target group. 
Having selected the locations, target sample sizes were 
set based on the assessment of the relative importance 
of each location.

When the location sampling was applied in practice in 
EU-MIDIS II, however, the plan to sub-sample locations 
other than those assessed as highly important was not 
followed through because of constraints on the number 
of locations available. Further, in most countries, many 
of the locations initially approached were not coopera-
tive, or were unproductive for interviews because they 
were not visited by members of the target groups in 
sufficient numbers. This meant that all locations that 
could be identified were typically used. Moreover, the 
minimum target of 15 interviews per location could not 
always be attained where locations were only mar-
ginally productive because few target group members 
attended. This was the case in Malta and Sweden.

In  Austria,  Cyprus  and  Poland,  the  survey  design 
included the location sampling method, to supplement 
the sample that could be achieved via one of the ‘con-
ventional’ sampling methods (see Table 4.12). This was 
because the target populations in these three countries 
were particularly dispersed or of especially low con-
centration. Relying only on the ‘conventional’ sampling 
methods would have made it difficult to achieve the 
full target sample size and coverage of the population 
would also have been very low.

For  example,  in  Austria,  setting  the  address  sam-
ple frame concentration cut-off for Sub Saharans at 
5 % meant covering only 35 % of the population – at 
this level it would require 20 successfully screened 
addresses to find one eligible household. Only six Sub 

In Denmark, a population register was available for 
sampling purposes, but immigrants and descendants 
of immigrants from Sub-Saharan Africa are thinly dis-
persed across the country. The viability of conducting 
a part of the interviews with this target group using 
a register sample was tested in the pilot, which con-
cluded that the register sampling approach would not 
be successful. Therefore, location sampling was used as 
the only sampling approach for this group in Denmark.

The fieldwork in the Netherlands was initially planned 
based on a multi–stage clustered address sample. How-
ever, this proved unviable to reach the target sample 
size when the actual field concentrations of the target 
group were half of those suggested by the sample frame 
(17 % actual concentration versus 34 % estimated; see 
Table 4.13). The lower than expected eligibility meant 
that the fieldwork got off to a poor start; interviewers 
felt discouraged by the amount of screening required, 
and a number abandoned the project. To complete the 
fieldwork in the Netherlands, the sampling methodol-
ogy was therefore changed to location sampling.

The country reports provide specific information on the 
selection of regions, numbers of locations and location 
importance estimates. Chapter 7 on Weighting explains 
how  various  sampling  approaches  were  taken  into 
account in the development of the weighting scheme 
for the survey.

Location  Sampling  was  successfully  implemented 
in all countries planned. In practice, however, some 
problems were encountered:

 • The assumption of being able to reach out to the to-
tal target population through the selected locations 
did not hold true in all countries. Women, for exam-
ple,  seem  under-represented  in  such  locations,  as 
experienced in Malta.

 • The number of interviews carried out at each loca-
tion should not be too small, to ensure robust esti-
mates of overlaps between locations, which are the 
basis for weight adjustment.

 • Standardisation of procedures across locations and 
countries is a challenge and preparatory work was 
sometimes  extensive:  contacting  organisations, 
pre-screening locations to decide on the selection 

60

process,  and  tailoring  access  and  interview  situa-
tions to each location.

 • Monitoring of fieldwork at locations was not easy, 
and it was impossible to carry out back checks with 
individual  respondents  because  addresses  were 
not linked to the respondents.

4�8�  Non-probability sampling 

(design 4)

In Luxembourg, quota sampling was used because the 
national authorities did not grant access for EU-MIDIS II 
to use the national register for sampling purposes. This 
became apparent at a very late stage, when fieldwork 
was already ongoing in all countries. Reverting to other 
random probability sampling methods was not foreseen 
in the project budget, and would have been difficult 
to implement on such short notice, so non-probability 
sampling had to be used.

Regions were selected across the country to ensure 
a good spread of different areas and to achieve as rep-
resentative a sample as possible. The target number 
of interviews for each region was set in proportion 
to country of birth statistics (which were available at 
city/commune level). No data were available to inform 
a probabilistic selection of PSUs, and so they were pur-
posively chosen from among the areas where the target 
group was known to concentrate.

Interviewers  were  permitted  to  move  around  each 
selected area, and to approach and interview anyone 
eligible for the survey provided they filled quotas set 
on gender and country of birth. The quotas were based 
on (i) the assumption of a 50/50 distribution by gender – 
although no statistics were available to assess whether 
or not this assumption is valid for the target population 
in Luxembourg – and (ii) available national-level coun-
try of birth statistics. No other quotas could be set as 
further data on the target group were not available.

Sampling 

61

5  

Piloting

This chapter presents a summary of the EU-MIDIS II pilot. 
It provides a short overview of the approach to the pilot, 
followed by the key findings, recommendations, and 
changes made to the fieldwork documents and the pro-
ject design before the main-stage fieldwork proceeded.

5�1�  Methodology

Pilot interviews were carried out in all 28 EU Member 
States in June–August 2015. Most countries started and 
finished the pilot during July except Sweden and the 
United Kingdom, which started pilot fieldwork in June, 
and Ireland, Italy and the Netherlands, which completed 
the pilot interviews in August. A total of 467 interviews 
were conducted at the pilot stage across all countries. 
This is 18 % fewer than the 570 originally specified 
in the project’s technical specifications. The difference 
was  due  to  a  change  in  the  pilot  testing  approach. 
Initially the plan was for all countries to complete 20 
full-length interviews, or 30 in Italy because there are 
three target groups in this country, using the sam-
pling approach used during the main-stage fieldwork. 
However, there was some uncertainty regarding the 
assumptions behind two sampling methods: Adaptive 
Cluster Sampling (ACS) and Focused Enumeration (FE) 
Therefore, it was agreed that a selection of countries 
covering different target groups and sample frames 
(Austria, Belgium, Bulgaria, Greece, Latvia, Spain and 
the United Kingdom) would focus on testing both ACS 
and FE, in place of testing the full questionnaire. In 
the remaining 21 countries, the full questionnaire was 
tested. This methodological pilot aimed to test the effi-
ciency of the two sampling approaches (ACS and FE), 
and therefore resulted in a need for interviewers to 
visit and screen far more addresses than originally fore-
seen. This could only be accommodated by cutting the 
overall number of interviews. In four of these countries 

(Belgium, Latvia, Spain and the United Kingdom), inter-
viewers used a short 5-minute questionnaire instead of 
the full-length version (short paper and pencil inter-
views (PAPIs) carried out only to give the appearance 
of a reason for screening respondents). Unlike the full 
questionnaire, there were no strict targets in place for 
the number of short 5 minute questionnaires completed 
per country or per target group, because the focus was 
on screening as many households as possible to test the 
sampling approach(es). In most countries, interview-
ers worked two PSUs, one of which was ACS and the 
other FE, so that they could contrast the methods. The 
countries that were to employ address-based sampling 
methods in the main stage also used ACS alongside 
their primary sampling method, and in one case FE 
(Lithuania). Countries had to achieve a minimum of 10 
interviews per target group if interviewing two or more 
target groups, or 20 interviews if they were interview-
ing just one target group. Apart from the Netherlands 
and Slovakia, all countries achieved their target number 
of interviews; however, some countries had to revert to 
using free-find methods because they exhausted their 
sample before reaching their target.

Wherever possible, countries applied in the pilot the 
same sampling methodologies as those planned for 
their  main-stage  fieldwork.  In  countries  where  the 
sampling frames were not available in time for the 
pilot – namely, Finland, Italy, Luxembourg and Slove-
nia – a free-find approach was used. This method was 
also used for a part of the pilot sample in Ireland and 
the Netherlands to achieve the target number of inter-
views when the main sampling methodology adopted 
in these two countries did not work as expected and 
interviewers were unable to reach the expected num-
ber of interviews. Although random route was planned 
in Portugal and Greece, ultimately random selection of 
addresses was not practicable in these countries. In 

63

Second European Union Minorities and Discrimination Survey – Technical report

Portugal, this was due to a Roma mediator who inter-
fered with household/respondent selection; because of 
time constraints, Portugal deliberately selected small, 
concentrated areas within PSUs using local knowledge. 
In Greece, events surrounding the country’s economic 
crises in 2015 delayed pilot fieldwork, which meant 
a delayed start to the pilot and insufficient time to apply 
the method. As a result, addresses were not selected 
following the random route procedure; however, the 
within-household selection of individuals to be inter-
viewed followed a randomised process based on the 
use of the respondent selection grid, as in other coun-
tries. Greece used local knowledge/free find to identify 
PSUs, and Italy used proxy information because sample 
frame data were not available in time.

Random route sampling was used when piloting the 
survey in all countries in which Roma were going to 
be surveyed in the main stage (Bulgaria, Croatia, the 
Czech Republic, Hungary, Romania, Slovakia and Spain). 
Exceptions to this were Portugal and Greece, as men-
tioned above. Random route was also used in countries 
where Russian minorities were to be surveyed in the 
main stage (Estonia, Latvia and Lithuania), and in some 
of the countries where immigrants and descendants 
of immigrants were to be surveyed (Belgium, Cyprus, 
France,  Greece,  Italy,  Spain  and  Portugal).  Address 
registers  for  sampling  the  pilot  respondents  were 
used in the following countries in which immigrants 
and descendants of immigrants were to be surveyed 
in the main stage: Austria, Ireland, the Netherlands 
and the United Kingdom. Onomastic sample sources 
were  used  in  Denmark  and  Germany,  and  location 
sampling was used in Austria, Cyprus, Denmark, Malta, 
Poland and Sweden.

As part of the development of the sample design and 
methodological improvements implemented, the EU-
MIDIS II pilot tested the use of Adaptive Cluster Sampling 
(ACS) in some primary sampling units (PSUs) to increase 
the efficiency of the respondent screening process. ACS 
works on the assumption that people within the same 
target group are more likely to live close to each other 
(in neighbouring addresses), at least to some degree. 
The first EU-MIDIS survey used another approach called 
Focused Enumeration (FE), which also aims to improve 
the efficiency of screening minority groups. FE’s effi-
ciency  gains  are  not  dependent  on  whether  or  not 
the target groups live close to each other, but come 
from allowing the interviewer to screen neighbouring 
addresses by proxy – that is, by asking people whether 
or not any of their neighbours fit the description of the 
survey’s target group and might be eligible to take part. 
This also means that interviewers do not need to con-
tact all issued addresses. In contrast, ACS does not rely 
on proxy identification, which for some of the targeted 
groups in the survey, e.g. Roma, could be particularly 
sensitive. In addition, ACS allows full implementation of 

self-identification without relying on information from 
proxies. Self-identification is one of the main principles 
of a human rights-based approach to data – as sug-
gested by the United Nations Human Rights Office of 
the High Commissioner (OHCHR)39 – which highlights 
the importance of respect and protection of personal 
identity as central to human dignity and human rights.

Pilot  interviewers  and  the  National  Survey  Experts 
(NSEs) provided feedback to the Central Coordination 
Team concerning the challenges faced in implementing 
the sampling methods and using survey tools such as 
the questionnaire and the contact sheet. The Central 
Coordination Team compiled these comments, which 
were instrumental for evaluating the need for final 
changes to the survey methodology and documents 
before the main-stage fieldwork.

5�2�  Key findings, 

recommendations and 
action taken as a result 
of the pilot

The remaining sections of this chapter provide selected 
key findings, recommendations and actions taken with 
regard to sampling and the survey fieldwork material, 
based on the results of the pilot.

Sampling

Key findings:  In  the  pilot,  ACS  addresses  were  not 
found to be more concentrated than core addresses. 
Five out of the seven countries (Austria, Belgium, Bul-
garia, Greece, Latvia, Spain and the United Kingdom) 
that  conducted  the  methodological  pilot  had  lower 
concentrations  among  ACS  addresses  than  among 
core addresses. In Bulgaria and Latvia, the ACS con-
centrations were higher in some PSUs and lower in 
others. Feedback from the pilot also suggested that 
FE was easier for the interviewers to implement than 
ACS. Among those testing both options, FE was seen 
as more efficient and straightforward for interviewers 
and generated less paperwork. Some of the 21 coun-
try teams piloting ACS noted only that a large number 
of mistakes were made by interviewers when imple-
menting the ACS selection rules. (This was highlighted 
in Italy, Latvia and Portugal). Some other teams, from 
the Czech Republic, France and Slovakia, gave positive 
feedback concerning the use of ACS; their assessment 
was that the method worked in finding members of the 
target group and could be applied accurately by inter-
viewers. The Central Coordination Team also examined 
the way interviewers in various countries had applied 

39  OHCHR (2016).

64

the ACS rules, and found some errors in the way the 
selection criteria had been used.

concerning  the  concentration  of  Roma  proved  to 
be significant overestimates.

Recommendations: Despite challenges related to its 
implementation, Adaptive cluster sampling was the pre-
ferred method for those countries where EU-MIDIS II 
was to survey Roma, because it lacks the element of 
third-party (proxy) identification of Roma. Focused 
enumeration was recommended for sampling of immi-
grants and descendants of immigrants in countries 
where the sample would need to cover primary sam-
pling units with a low concentration of the target group. 
In countries where ACS was to be applied, the pilot 
results recommended ensuring that this was the only 
method introduced to the interviewers, to avoid con-
fusion concerning the procedure to follow. It was also 
recommended that further material and more exten-
sive training covering possible scenarios and procedures 
needs to be provided to the interviewers. Based on 
all pilot interviews in the 28 EU Member States, the 
Central Coordination Team recommended developing 
the interviewer training sessions further, to include 
more practical exercises and examples of what to do 
in specific situations, in implementing both ACS and FE.

Key finding: The pilot suggested that the lack of accuracy 
of the sample frame – such as the available sampling 
frame not being fully up to date – could make sampling 
more difficult in some countries. Much lower concentra-
tions of the target groups were found compared with 
the sample frame data, for example in pilots carried out 
in Ireland and in the Netherlands, although the small 
number of PSUs used in the pilot limited the extent to 
which definitive conclusions could be drawn concerning 
the whole country. Furthermore, there was often a lack 
of viable alternatives to the selected sampling frames.

Recommendations: To plan for a staggered start of the 
fieldwork where possible, to closely monitor the field-
work and the validity of the initial assumptions, and to 
develop contingency plans such as having a reserve 
sample as well as stopping and dropping rules in place 
for lower than expected concentration PSUs (that is, 
rules to determine when interviewers can stop work-
ing in PSUs where, after many efforts, they have been 
unable to find eligible respondents).

Key finding: The PSUs in most of the countries survey-
ing Roma were very large, or the fieldwork teams could 
not obtain high-quality maps showing the borders of 
the PSUs prior to the pilot fieldwork. As a result, local 
knowledge was used to identify Roma settlements 
and estimate concentrations of Roma in the PSUs prior 
to fieldwork. Over the course of the pilot fieldwork, 
some teams found higher concentrations of Roma in 
the PSUs than expected based on local knowledge. 
This was the case in Bulgaria, for example. In other 
countries, e.g. Romania and Spain, initial assumptions 

Recommendations: For Roma samples, the pilot results 
suggested that concentrations of Roma should be re-
estimated after sample selection, using all available 
information on the size and location of Roma settle-
ments, so that the correct number of addresses could 
be issued for each PSU selected.

Key  finding:  Location  sampling  was  conducted  in 
Austria, Cyprus, Denmark, Sweden, Malta and Poland. 
The aim of the pilot with respect to location sampling 
was to test whether or not centres would agree with 
access and to test the interviewing processes at the 
centres (random selection of respondents, conduct-
ing interviews). Small numbers of centres were cho-
sen  purposefully  to  test  within-centre  procedures. 
Location sampling worked well in all countries where 
it was used and with all groups, except in Poland. All 
countries  found  willing  participants  for  the  survey, 
with the exception of Poland, where hardly any immi-
grants from Vietnam cooperated and immigrants from 
Ukraine were reluctant. In contrast, Sweden reported 
that the sole reason for non-participation was lack of 
time. Regarding respondent selection, specifically when 
using location sampling, interviewers were recorded 
as having some problems with following the sampling 
procedure correctly. The team in Denmark had diffi-
culties using the ‘1 in n’ rule for randomly selecting 
respondents in pre-selected locations. Austria said that 
in one (private) venue, eligible people were brought 
to the interviewers, and the latter were not free to 
select respondents randomly.

Recommendations: Using a range of closed and pub-
lic locations can maximise representativeness when 
relying on location sampling. Fieldwork teams should 
organise set-up meetings with organisations operat-
ing closed venues to obtain their full cooperation. The 
pilot also highlighted a need for the Central Coordinating 
Team and FRA to follow up with some country manag-
ers to ensure that they understood the details of the 
location sampling method, and in particular the rules 
concerning location and respondent selection.

The pilot further recommended that country teams 
discuss  and  agree  on  the  most  appropriate  way  of 
organising  interviews  in  each  location,  depending 
on circumstances at the location. This could include 
location-specific issues such as whether people ‘flow’ 
through the location during the day or would already 
be there when interviewers arrive, or both; whether 
eligible people worked there or were visitors; and what 
level of help/access the centre could provide for the 
interviewers to carry out their work. The procedures 
adopted  did  not  need  to  be  standardised  because 
the main purpose of the random selection in location 

Piloting 

65

Second European Union Minorities and Discrimination Survey – Technical report

sampling is to limit the influence of interviewers over 
respondent selection by instructing them to follow clear 
selection rules which ensure that the random selection 
process is followed.

Gaining respondent cooperation

Key finding: Country teams mentioned notable dif-
ferences  in  the  level  of  interest  in  the  survey  by 
gender, target group and education of respondents. 
Women  tended  to  be  less  interested  in  the  survey 
than men (specifically mentioned in Croatia, Finland 
and France), and in some instances female respond-
ents said that they needed to obtain permission from 
their husbands to take part in the survey (mentioned 
in Hungary and Spain).

Recommendations: Wherever possible, gender match-
ing of interviewer and respondent should be applied.

Key finding: In Croatia and Portugal, the pilot field-
work teams encountered difficulties when working 
with Roma community mediators or heads of Roma 
settlements. It was hard to convince them of the need 
to follow the random sampling rules when selecting 
the addresses to contact.

Recommendations: The pilot experiences suggested 
that there was a need for a letter which interview-
ers can hand to the mediators, and which thanks the 
mediators for their help while stressing the importance 
of random sampling. Where NGOs or local community 
leaders were to be engaged, the importance of random 
selection of respondents should be explained to avoid 
conflicts during the mainstage of the fieldwork.

Key finding: There were a number of reasons which 
respondents gave when they did not want to take part 
in the survey, such as the length of the interview, no 
interest in the subject, the fear of possible consequences 
(such as concern that their answers would be reported 
to the authorities, especially when questions concern 
social benefits or residence status), how the data will 
be used, and refusals due to religious festivals (as the 
pilot fieldwork fell in the Ramadan period). Respondents 
were more likely to refuse before the respondent selec-
tion was completed, rather than after. Some countries 
recruited interviewers with the same background as 
the target groups and confirmed that having a good 
language match between interviewers and respondents 
encouraged survey participation.

Recommendations: As far as possible it was recom-
mended that local subcontractors use interviewers with 
the same background as the target groups. Interview-
ers should be provided with a list of potential reasons 
respondents might give for refusal, to help them respond 
to these arguments and overcome initial refusals.

Key finding: Having the survey material (questionnaire 
and introductory letter) available in the language of 
the respondent facilitated engagement and encouraged 
participation. However, some teams felt that the lan-
guage of the introductory letter was too complex for 
some respondents to understand.

Recommendations: It was agreed that countries could 
tailor the letter to reflect local circumstances, and all 
tailored versions were to be signed off by FRA before 
being used in the field in the main-stage fieldwork. 
A  postcard/information  leaflet  was  produced  in  all 
national  languages  to  supplement  the  information 
provided in the introductory letter.

Contact sheets

Key finding: The pilot indicated the need to redesign the 
address-based contact sheet to make it easier for inter-
viewers to use, and to ensure that the data collected 
were complete and accurate. The length and number 
of contact sheets that had to be completed resulted in 
some cases in partial and incorrect information being 
recorded. The need to complete a contact sheet for 
every screened address (even if ineligible) means that 
interviewers have to carry a large amount of paper 
with them at all times. The pilot contact sheets were up 
to nine pages long. The structure of the contact sheet 
meant that interviewers had to go back and forth on 
the sheet to complete sections.

Recommendations: There was a need to reduce the 
length of the contact sheet, and to make it easier to 
record non-contacts and non-eligible households, to 
revise the number of outcome codes, and to reduce 
ambiguity to make sure that interviewers are able to 
easily  select  the  correct  code.  When  implementing 
adaptive cluster sampling, interviewers should open 
a new contact sheet and add the relevant anonymised 
household  identification  number  (key),  rather  than 
screening ACS addresses on the same contact sheet 
as the core address. In this way, the original core + ACS 
contact sheet could be reduced to three pages. The pilot 
results also called for the Central Coordinating Team and 
FRA to reconsider the extent to which some informa-
tion, (e.g. concerning neighbourhood characteristics) 
is needed. The structure of the contact sheet could be 
further simplified by moving the respondent selection 
grid so that it is next to the list of eligible respondents, 
to make respondent selection easier.

Questionnaire

Key finding: The pilot questionnaire took slightly longer 
to  administer  than  expected.  The  average  length 
of an interview was 58 minutes, compared with the 
expected average length of 50 minutes. Most coun-
try teams reported that it was difficult to maintain the 

66

Recommendations: The National Survey Experts should 
ensure  that  interviewers  understand  how  to  code 
respondents’ various employment situations, and how 
to probe respondents to ensure that they have cap-
tured enough details about their employment situation 
to perform the coding correctly.

Key finding: A large majority of the country teams (18) 
reported that asking respondents about their income 
was  very  difficult.  Respondents  were  reluctant  to 
answer the questions on income because 1) they felt 
this question was intrusive, particularly among Roma; 
2) they struggled to estimate their income, which might 
come  from  a  number  of  different  sources;  3)  their 
income was irregular and therefore difficult to esti-
mate on a weekly, monthly or annual basis; 4) they felt 
embarrassed to say that they were reliant on various 
benefits as a source of income; or 5) they did not want 
to include irregular/illicit income.

Recommendations: Interviewers should be reminded 
to  reassure  interviewers  that  information  col-
lected  in  the  survey  will  remain  anonymous  and 
be treated confidentially.

Key  finding:  The  teams  in  four  countries  (the 
Czech Republic, Denmark, France and Slovakia) reported 
that respondents were confused by the term ‘married 
traditionally’. This was among the answer categories 
when respondents were asked whether they were mar-
ried, single, etc. They asked if it referred specifically to 
a marriage in their country of birth.

Recommendations: Interviewers were instructed to 
clarify what ‘traditional marriage’ meant. This answer 
category could also be removed if considered unlikely 
to provide many responses given the country and the 
target group in question, while it was kept in the main 
stage of the survey where relevant.

respondents’ motivation and interest towards the end 
of the interview, and in some cases respondents were 
getting visibly bored or irritated, particularly as parts of 
the questionnaire were considered very repetitive. As 
a result, the teams expressed concern that the length 
and repetitive nature of the questionnaire could have 
a detrimental impact on data quality and that, once 
respondents get used to the flow of the survey, they 
were more likely to deliberately answer ‘no’ to the dis-
crimination questions to bypass the follow-up ques-
tions. A number of questions asked respondents about 
their experiences in the last five years in their current 
country of residence. This was a problem for those who 
had not yet been in the country that long.

Recommendations: Following the pilot, the question-
naire  should  be  shortened  by  combining  repetitive 
questions to improve the flow of the interview. A poten-
tial option was to simplify the timeframes used in vari-
ous survey questions and to ask only about experiences 
since respondents had moved to the country, if this had 
been less than five years ago.

Key finding: Interviewers stated that some respondents 
became frustrated because, although the survey was 
introduced as asking about discrimination, the inter-
view started with a lengthy section concerning house-
hold composition, which some respondents considered 
a sensitive subject.

Recommendations: The interviewer training sessions 
should pay more attention to the sensitive nature of the 
questions, so that interviewers are prepared to reas-
sure respondents and explain why they were asking 
certain questions. Interviewers should also be familiar 
with the definition of a household used in the survey 
and explain this to the respondent. The introduction 
at the beginning of the household information section 
needed  revision  to  ensure  that  respondents  would 
expect to be asked questions about their household 
and themselves, but that the main part of the question-
naire would focus on discrimination and other issues. 
The flow of the questionnaire would be better if some 
of the questions related to housing and living standards 
were at the end.

Key  finding:  Interviewers  in  Estonia,  Germany  and 
Greece reported that the questions concerning employ-
ment were sensitive and difficult to answer because 
respondents were either unsure of their employment 
status or reluctant to reveal their working situation 
for fear of reprisal from the authorities. Interviewers 
in Romania and Slovenia reported that many Roma 
respondents who were working under irregular con-
ditions were afraid to give an honest answer about 
their employment situation.

Piloting 

67

6  

Fieldwork

This chapter provides an overview of the main stage of 
fieldwork, including a description of fieldwork progress, 
procedures and outcomes.

6�1�  Fieldwork dates 
and progress

Table 6.1 shows the fieldwork dates, which varied across 
countries and target groups. All in all, data collection 
for EU-MIDIS II lasted for almost one year. It started 
with fieldwork in the United Kingdom at the end of 
September 2015. Most other countries started fieldwork 
in October–November 2015. The reasons for different 
starting times include differences in the amount of time 
needed to prepare sampling designs and obtain access 
to existing sampling frames. Countries also differed in 
terms of the time needed to adapt survey materials 
in the countries, such as translations, and to prepare 
the CAPI scripts following the pilot. In some countries, 
particular events that would have made fieldwork more 
difficult delayed the start of fieldwork slightly. These 
included elections in Bulgaria and Croatia, and a high-
level summit in Malta. In Luxembourg, the fieldwork did 
not start until 16 March 2016, after lengthy negotiations 
with the National Statistical Institute (NSI) concerning 
access to its register data. Since full access to the reg-
ister in Luxembourg was not granted, an alternative 
sample design had to be developed and implemented.

The average time spent in the field to complete the 
interviews for one target group was 19 weeks. Field-
work duration varied across countries and ranged from 
six weeks (Roma in Greece) to 37 weeks (in France). The 
terrorist attacks in Paris on 13 November 2015 and in 
Brussels on 22 March 2016 brought a temporary halt to 
fieldwork in Belgium and France. In some other coun-
tries, fieldwork was interrupted because of the need 
to adjust the sampling approach due to either over- or 

under-achievement of fieldwork targets. In Finland, for 
example, interviewers exhausted the issued number of 
addresses (gross sample) more quickly than expected, 
and had to obtain an additional sample.

During the course of fieldwork, Ipsos MORI’s Central 
Coordination Team (CCT) provided FRA with weekly 
updates on the number of completed interviews and 
other final outcomes. The local project teams were also 
required to provide fortnightly/periodical updates on 
response rates achieved by each interviewer, as well 
as fieldwork outcomes at PSU level. This was done to 
allow FRA and the CCT to evaluate progress, and to 
provide early warning of any potential issues with par-
ticular interviewers or with the sample design. Each 
local contractor collected the necessary information 
independently and presented it to the CCT for review 
and comment, and for discussion with FRA as necessary.

In addition to these fieldwork progress reports, a tel-
econference was held with each local contractor, FRA 
and the CCT during the early stages of fieldwork, to 
share first experiences of interviewing in the relevant 
country and to identify potential issues at an early 
stage. The teleconference covered the following issues:

 • check on understanding of the sampling/fieldwork 
rules and how these were being implemented;

 • experience  with  fieldwork  to  that  point,  covering 
feedback  from  the  interviewers,  including  with 
screening  respondents  and  any  language,  ethical 
and cultural considerations;

 • any challenges faced and discussion on how these 

could be overcome;

 • sharing any best practice tips/examples from other 

fieldwork agencies.

69

Second European Union Minorities and Discrimination Survey – Technical report

Table 6�1: Fieldwork dates by country

Country

Fieldwork dates

Dates fieldwork paused (where necessary)

Start

End

Start

End

AT

BE

BG

CY

CZ

DE

DK (SSAFR)

DK (TUR)

EE

EL (ROMA)

EL (SASIA)

ES

FI

FR

HR

HU

IE

IT

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

21/10/2015

03/11/2015

11/11/2015

22/10/2015

25/11/2015

24/10/2015

16/10/2015

15/01/2016

07/12/2015

07/11/2015

21/11/2015

02/11/2015

02/11/2015

06/11/2015

18/11/2015

28/10/2015

26/10/2015

30/10/2015

28/11/2015

16/03/2016

31/10/2015

18/11/2015

13/01/2016

28/11/2015

10/11/2015

22/10/2015

02/11/2015

26/11/2015

27/11/2015

24/09/2015

14/04/2016

14/09/2016

20/02/2016

05/03/2016

06/03/2016

14/03/2016

26/04/2016

31/05/2016

26/04/2016

21/12/2015

04/02/2016

07/04/2016

09/08/2016

21/11/2016

29/02/2016

13/03/2016

09/05/2016

26/03/2016

11/04/2016

09/05/2016

05/02/2016

26/01/2016

07/08/2016

31/03/2016

03/03/2016

18/03/2016

30/03/2016

04/03/2016

29/02/2016

24/04/2016

Source:  FRA, EU-MIDIS II 2016

13/06/2016

18/08/2016

11/01/2016

24/02/2016

12/01/2016

31/03/2016

21/07/2016

23/12/2015

08/02/2016

30/01/2016

25/02/2016

12/07/2016

14/11/2016

27/01/2016

14/02/2016

13/03/2016

23/12/2015

04/01/2016

70

6�2�  Contacting respondents 

– recruitment 
and interview format

Interviewers working in EU-MIDIS II were required to 
record the outcome of contacting each sampled address 
on a contact sheet. Different types of contact sheets 
were used across the 28 EU Member States. The contact 
sheets were used as a record of fieldwork progress, and 
to screen for and select eligible respondents.

Overall, screening based on country of birth – for those 
interviewing immigrants and descendants of immi-
grants, including recent immigrants – and on self-iden-
tification – for those interviewing Roma or members of 
Russian minorities – worked well. Respondents were 
able to understand and easily answer the screening 
questions. Some countries, such as Austria, reported 
using a map of Africa when screening for immigrants 
and  descendants  of  immigrants  from  Sub-Saharan 
Africa, as this made it easier for respondents to iden-
tify whether or not they are from one of the eligible 
countries. Croatia and Lithuania were the only two 
countries that reported respondents being reluctant to 
self-identify as Roma or Russian minority, respectively.

For the majority of countries where FE was used, there 
were no reports of difficulties in proxy screening based 
on country of birth (for those interviewing immigrants 
and descendants of immigrants) or on Russian minority 
identity. Only very few interviewers, for example in 
Greece, reported that some respondents were reluctant 
to provide information about their neighbours.

Arriving at the selected address, interviewers were 
required to attempt to make contact with respondents 
face to face, except in Denmark and Finland, where 
eligibility screening was by telephone. At least three 
contact attempts had to be made at all core and ACS 
addresses before a case could be regarded as a non-
contact. In those sampling points where FE was imple-
mented, at least three contact attempts had to be made 
at each cluster of five addresses – one core and four FE 
addresses – before any of the cluster of addresses could 
be regarded as a non-contact.

In Denmark (for immigrants and descendants of immi-
grants from Turkey) and in Finland, initial contact was 
by telephone because individual sample frames with 
telephone contact details were available. At least 10 
contact attempts by telephone were required before 
a case could be regarded as a non-contact.

Contact attempts, face to face or by telephone, had 
to take place at different times of day and on differ-
ent days of the week, wherever possible. On mak-
ing contact, interviewers had to establish whether or 
not there were any eligible people within the house-
hold, randomly select one or two (depending on tar-
get group) eligible respondents, and then interview 
them. Table 6.2 shows the average number of contact 
attempts in each country.

In location sampling, interviewers made initial contact 
face to face at the locations. The rules for screening var-
ied depending on the country/location, but in general, 
interviewers’ instructions were to approach every nth 
person as they worked their way through the venue 
or crossed a person’s path in a particular direction. The 
number of people present determined the sampling 
interval n for each location. If fieldwork took place on 
days when a location was busier than on other days, 
the interval was adapted (i.e. shorter interval).

Contact  sheets  were  used  to  record  all  screening 
attempts at both addresses and locations. The location 
sampling contact sheets were relatively straightforward 
to complete. However, the contact sheets for addresses 
were more demanding because of the more complex 
way of screening for eligibility and recording all neces-
sary information. Few countries included the contact 
sheets in the CAPI script or scripted the contact sheet 
on a smartphone app, which was much easier for the 
interviewers than using paper contact sheets.

All interviews in EU-MIDIS II were administered using 
CAPI. The only exception from this are some interviews 
in Portugal, which were conducted using paper ques-
tionnaires (PAPI) due to safety concerns over carrying 
a laptop in certain areas. Six countries – Austria, Den-
mark, Estonia, Latvia, Lithuania and the Netherlands 
– also used the CAPI scripts in a language that was 
tailored to the target group, besides having the CAPI 
tool scripted in the main national language. In each of 
these six countries, a minimum of 15 % of productive 
interviews were carried out in languages tailored to 
the target group, which indicates that the additional 
CAPI script translations were a valuable resource (In 
Estonia, all interviews were conducted in Russian). In 
other countries, translated paper versions of the ques-
tionnaire were used instead to help overcome language 
problems during an interview. Interviewers used trans-
lations most often in Belgium, Germany and Poland. In 
each of those countries, at least 10 % of the interviews 
used translated paper questionnaires.

Fieldwork 

71

Second European Union Minorities and Discrimination Survey – Technical report

Table 6�2: Contact attempts per issued address by country

Country

Mean

Median

Minimum

Maximum

AT

BE

BG

CY*

CZ

DE

DK**

EE

EL

ES

FI**

FR

HR 

HU

IE

IT

LI

LV

NL

PL

PT

RO

SI

SK

UK

Notes: 

2

1.5

1.1

1

2.2

2

6.1

2.2

1.3

1.4

4.4

2.3

1.1

1.2

1.6

1.3

1.5

1.7

1.8

1.5

1.8

1.4

1.6

1.9

2.8

2

1

1

1

3

2

5

2

1

1

3

2

1

1

1

1

1

1

2

1

2

1

1

1

2

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

4

8

4

1

4

8

16

4

4

7

15

7

7

5

5

5

4

4

8

8

7

4

8

4

8

Information calculated from core and ACS contact sheets, excluding cases with the following outcomes: invalid address, 
unable to screen – contact not attempted, unprocessed addresses. Productive cases that did not meet quality control 
requirements were later excluded. Luxembourg, Malta and Sweden are excluded from this table because they did not 
conduct any address-based interviews; quota sampling was used in Luxembourg, while in Malta and Sweden the survey 
used location sampling only.  
* Interviewers found that they were always able to make contact with someone from the household to complete the 
screening.  
** These include a combination of telephone calls and face-to-face visits.

Source:  FRA, EU-MIDIS II 2016

72

Fieldwork 

6�3�  Interview length

The average interview length was 48 minutes and the 
median length was 44 minutes. The distribution of 
interview length shows that interviews varied around 
the median time, with a tendency for few longer inter-
views (Figure 6.1). Considering only the first respondent 
for countries where up to two persons per household 
could be interviewed, the average interview length 

increases to 49 minutes. In other words, interviewers 
were typically able to complete the second interview 
in a shorter time than the first. The longer duration 
of the first interview reflects the time needed to ask 
a set of questions concerning each household mem-
ber, which only the first respondent in a household 
needed to answer.

Figure 6�1: Distribution of interview length in EU-MIDIS II, EU-28

Median length: 44 minutes

2000

i

s
w
e
v
r
e
t
n

i

f
o
r
e
b
m
u
N

1000

0

15

30

45

60

75
120
Interview length in minutes

105

90

135

150

165

180

Notes: 

Statistics concerning the interview length are based on 24,478 interviews. For a small number of interviews, information 
concerning interview length was not recorded or is otherwise missing/incorrectly recorded. In all countries, some 
particularly short and particularly long interview times were recorded. These were often due to issues with the 
timestamp functionality within the CAPI software. For this reason, those interviews that were shorter than 15 minutes or 
longer than three hours are excluded from the calculations.

Source:  FRA, EU-MIDIS II 2016

Figure 6.2 shows the average and median length of 
interviews  by  country  and  target  group.  Interview 
length varied quite considerably across the countries. 
Cyprus recorded the highest average at 92 minutes per 
interview, followed by interviews with immigrants and 
descendants of immigrants from Sub-Saharan Africa in 
Germany (72 minutes) and in Sweden (70 minutes). In 
contrast, the average interview was much shorter in 

Portugal (Roma), Austria (SSAFR) and Lithuania; around 
30 minutes. Reasons for these differences include lan-
guage difficulties, recorded levels of discrimination 
(respondents who had felt discriminated against were 
asked follow-up questions concerning their experi-
ences) and household size (due to more information to 
be provided about household members).

73

 
 
Second European Union Minorities and Discrimination Survey – Technical report

Figure 6�2: Average and median duration of interviews, by country and target group

p
u
o
r
g
t
e
g
r
a
t
−
y
r
t
n
u
o
C

CY − ASIA
DE − SSAFR
SE − SSAFR
SE − TUR
LU − SSAFR
NL − NOAFR
FI − SSAFR
DE − TUR
SK − ROMA
CZ − ROMA
NL − TUR
RO − ROMA
EL − ROMA
DK − SSAFR
FR − SSAFR
DK − TUR
EL − SASIA
MT − SSAFR
FR − NOAFR
LV − RUSMIN
BE − TUR
IT − SSAFR
IT − NOAFR
BE − NOAFR
PL − RIMGR
IT − SASIA
HU − ROMA
HR − ROMA
ES − NOAFR
BG − ROMA
AT − TUR
UK − SASIA
SI − RIMGR
UK − SSAFR
ES − ROMA
EE − RUSMIN
PT − SSAFR
IE − SSAFR
LT − RUSMIN
AT − SSAFR

average

median

30

40

50

60
Duration in minutes

70

80

90

100

Notes: 

 Statistics concerning the interview length are based on 24,478 interviews. For a small number of interviews, information 
concerning the interview length was not recorded or is otherwise missing/incorrectly recorded. In all countries, some 
particularly short and particularly long interview times were recorded. These were often due to issues with the 
timestamp functionality within the CAPI software. For this reason, those interviews that were shorter than 15 minutes 
or longer than three hours are excluded from the calculations. In Portugal, 200 interviews with Roma used paper 
questionnaires instead of CAPI, because of safety concerns, and data entry staff later transferred the data from the paper 
questionnaires into the CAPI script. As a result, their interview length is much shorter and this group was taken out for 
calculating interview length.  
ASIA, immigrants and descendants of immigrants from Asia; NOAFR, immigrants and descendants of immigrants from 
North Africa; RIMGR, recent immigrants; ROMA, Roma; RUSMIN, Russian minority; SASIA, immigrants and descendants 
of immigrants from South Asia; SSAFR, immigrants and descendants of immigrants from Sub-Saharan Africa; TUR, 
immigrants and descendants of immigrants from Turkey.

Source:  FRA, EU-MIDIS II 2016

74

 
 
 
Fieldwork 

6�4�  Fieldwork support 

material

A number of fieldwork support material were used as 
a part of EU-MIDIS II fieldwork, including an introduc-
tory letter and a project postcard, respondent-friendly 
questionnaires and incentives. The introductory letter 
and postcard contained basic information about the 
survey, and they were used in almost all 28 countries. 
Having the letters in alternative languages in addition 
to the national languages of each country was also 
useful, although the letters were not available in all 
target group languages, for example in all Asian and 
African languages. In several countries, the respond-
ent-friendly paper questionnaires helped the inter-
view process when a respondent’s command of the 

Table 6�3: Incentives used in EU-MIDIS II

Country

Incentive

national language was limited, although in some cases 
using them reportedly increased the interview length 
because interviewers frequently had to check that the 
respondent was looking at the correct page. Table 2.2 
lists the languages these materials were translated into.

Many surveys use small incentives as a token of appre-
ciation for respondents, recognising the time and effort 
taken to complete the survey. In some cases, infor-
mation about incentives can also encourage hesitant 
respondents to participate. In EU-MIDIS II, incentives in 
the form of low value non-monetary gifts, gift vouchers 
and cash were used in the majority of countries as sum-
marised in Table 6.3. Local teams determined the value 
of the incentives used in each country, based on their 
extensive experience. In some countries, the fieldwork 
teams did not consider incentives necessary.

AT

BE

BG

CY

CZ

DE

DK

EE

EL

ES

FI

FR

HR

HU

IE

IT

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

Money: € 15

Money: € 10

Double coffee package: value € 2

None

Money: CZK 35 (€ 1.30)

Money: € 20

Money: € 27

Light reflectors: value € 1

Supermarket voucher: value € 7

Supermarket voucher: value € 10

Voucher: value € 10

None

Gift: value € 3

None

Money: € 10 for one person in household, € 25 for two people in household

Shopping voucher: value € 5

Pens/reflectors: value less than € 1

Money: € 20

Light reflector: value € 1

Money: € 10

Money: € 15

Gifts such as coffee, tea, sweets: value PLN 7 (€ 1.60)

None

Gifts (sweets, coffee): value € 1.65 

Supermarket voucher: value € 25

None

Money: € 2

None

Source:  FRA, EU-MIDIS II 2016

75

Second European Union Minorities and Discrimination Survey – Technical report

6�5�  Fieldwork outcomes

Table 6.4 provides a summary of fieldwork outcomes 
by country and the associated response rate. This is at 
household level and only for address-based samples. 
This section later describes the exact definitions used 
to calculate the rates.

The eligibility rates range from only 7 % in Ireland to 
100 % in Denmark and Finland. The wide range of eli-
gibility rates and differences between countries and 
target groups are due to differences in the sampling 
designs, which targeted different levels of concentra-
tion of the target groups. In Ireland, for example, a lot of 
screening was needed in areas with low concentration 
of the target group. In Denmark and Finland, on the 
other hand, the samples used information on the target 
population from population registers, which resulted 
in 100 % eligibility of respondents. The response rates 
vary from 15 % in Denmark and Finland to 90 % in 
Cyprus. The lower response rate in Denmark and Finland 
is in part due to the two-phase recruitment approach 
used, in which respondents were first contacted and 
screened by telephone to make an appointment for 
a face-to-face interview.

In addition, feedback from the local contractor in Den-
mark suggests that there was a higher level of suspi-
cion among immigrants and descendants of immigrants 
from Turkey contacted about taking part in the survey. 
Some women were not given permission to take part 
by their husbands, in particular where it was not pos-
sible to arrange for a female interviewer to conduct the 
interview. Gender matching was used as much as pos-
sible, but it was not always possible given the available 
pool of interviewers and the location of respondents.

However, it was also reported in Finland that there 
was extensive media coverage of the refugee crisis 
and related issues, such as immigration, discrimination 
and racism, before the start of fieldwork and during its 
first phase, from November 2015 to March 2016. Far-
right groups became more active during this time, e.g. 
patrolling streets and committing arson attacks against 
reception centres where asylum seekers were living. 
The fieldwork agency in Finland reported that these 
events and the general political climate had a negative 
impact on the willingness of immigrants and descend-
ants of immigrants from Sub-Saharan Africa to take part 
in the survey. This may also have contributed to a lower 
response rate in Finland.

The number of eligible addresses is based on addresses 
that were given a final outcome code of:

 • Completed interview

 • Partial interview

 • Eligible but respondent selection refused

 • Refusal by contacting field office

 • Refusal by selected person before interview

 • Refusal by proxy before interview

 • No contact with selected person after three visits

 • Proxy refusal by someone else at the address

 • Broken appointment – no re-contact

 • Mentally/physically  unable  to  be  interviewed/too 

ill/sick (during main-stage fieldwork)

 • Other  unproductive  results  at  known  eligible 

households

 • Eligible household – excluded interview (for exam-

ple because of high item non-response).

The  number  of  ineligible  addresses  is  based  on 
addresses that were given a final outcome code of:

 • Screened – no eligible household members40

The number of addresses with an unknown eligibil-
ity  is  based  on  addresses  that  were  given  a  final 
outcome code of:

 • Unable to screen because of language difficulties

 • Unable  to  screen  because  of  physical/mental 

disability

 • Refused all information

 • No contact made after three visits (includes unable 

to establish via proxy screening)

 • Address issued but contact not attempted (for ex-

ample where the PSU was stopped).

40  This includes FE addresses that were found to be ineligible 
(either by direct contact, or when screened by proxy). 

76

Table 6�4: Fieldwork outcomes at household level for address-based samples, by country

Country

Ineligible 
households

Households 
where 
eligibility is 
unknown

Eligible 
households

Interviewed 
households*

Eligibility 
rate (%)

Response 
rate (%)

Fieldwork 

AT

BE

BG

CY

CZ

DE

DK

EE

EL – ROMA

EL – SASIA

ES – NOAFR

ES – ROMA

FI

FR

HR 

HU

IE

IT

LT

LV

NL

PL

PT – ROMA

PT – SSAFR 

RO

SI

SK

UK

Notes: 

4,082

6,902

1,206

2,212

1,022

113

0

92

386

3,081

8,562

4,882

0

4,729

461

560

5,484

10,485

3,148

510

1,182

206

233

1,587

4,244

2,533

1,552

5,763

6,133

2,133

483

35

1,964

1,041

1,547

211

156

502

429

543

1,037

6,118

81

1,200

826

1,503

646

527

2,049

382

154

284

1,639

439

1,656

1,114

675

1,173

1,377

255

2,020

1,473

436

468

791

498

798

1,282

2,263

2,185

591

1,245

425

1,989

572

973

249

440

674

713

1,727

389

2,147

1,767

486

987

1,078

233

817

1,085

289

401

508

399

540

776

502

1,414

538

1,171

298

1,145

404

614

209

272

553

449

1,408

285

1,098

908

14

15

53

10

66

93

100

84

67

14

9

21

100

32

56

69

7

16

15

66

17

68

74

31

29

13

58

23

31

67

66

90

25

44

15

62

57

70

65

56

15

34

85

56

62

51

60

47

35

39

70

56

64

64

35

45

* Includes only accepted interviews. Interviews excluded because of high item non-response are not counted as accepted 
interviews, but are included in the number of eligible households.  
Luxembourg, Malta and Sweden are excluded from this table because they did not conduct any address-based 
interviews. Data for Austria, Cyprus, Denmark, the Netherlands and Poland are provided for the address-based part of the 
sample, while in these countries the remaining part of the sample relied on sampling methods other than address-based 
sampling.  
ROMA, Roma; SASIA, immigrants and descendants of immigrants from South Asia; SSAFR, immigrants and descendants of 
immigrants from Sub-Saharan Africa.

Source:  FRA, EU-MIDIS II 2016

77

Second European Union Minorities and Discrimination Survey – Technical report

Invalid addresses are those assigned the codes below. 
They are not included in the response rate calculation:

 • Address not valid (does not exist/cannot locate/de-

molished/under construction/not accessible)

 • Address not private (institution/business)

 • Address  not  occupied 

(empty/second/holiday 

home).

Interviewed households include only those where at 
least one interview was completed.

The household eligibility rate e is calculated as follows:

e = 

CEH

CEH + CIH 

where:  CEH  =  confirmed  eligible  households  and 
CIH = confirmed ineligible households.

The  household  response  rate  is  calculated  as  fol-
lows,  in  accordance  with  the  RR3  definition  of 
response rates by the American Association for Public 
Opinion Research (AAPOR):41

Response rate = 

I

CEH + e(UE) 

where I = interviewed households and UE = households 
where eligibility is unknown.

6�6�  Field force and fieldwork 

control

Table 6.5 provides details on the number of interviewers 
who worked on EU-MIDIS II in each country, including 
information on the percentage of interviews conducted 
per interviewer and the number of interviewers with 
the same ethnic background as the target groups. Alto-
gether 1,105 interviewers worked on the project and 
conducted at least one interview. The number of inter-
viewers per country ranged from 15 to 78. The contrac-
tor’s aim was that each interviewer should complete 
not more than 5 % of the total number of interviews. 
Table 6.5 shows that in most countries it was not pos-
sible to achieve this, and at least one interviewer did 
more interviews. For all interviewers who did more than 
10 % of interviews in a country, a justification had to 
be provided by the contractor. Reasons for interview-
ers conducting more than 10 % of the total sample 
are mainly related to difficulties, in some countries, 
in finding enough experienced and skilled interview-
ers  ready  to  work  on  a  challenging  project.  Where 
interviewers carried out more than 10 % of the total 
interviews, additional back checks and quality control 
measures were put in place to guarantee the quality of 
the data collected.

It was not possible in all countries to find sufficient num-
bers of interviewers with the same ethnic background 
as the target group, because of the level of experi-
ence required of interviewers working on the survey. 

Table 6�5: Number of interviewers and average percentage of interviews conducted

Country

Number of 
interviews

Number of 
interviewers that 
completed at least 
one interview

Interviews in the total sample 
conducted per interviewer

Mean  
(%)

Median 
(%)

Minimum 
(%)

Maximum 
(%)

AT

BE

BG

CY

CZ

DE

1,054

1,339

1,078

436

817

1,419

31

52

59

19

49

76

3

2

2

5

2

1

3

1

1

4

2

1

1

< 0.5

< 0.5

1

< 0.5

< 0.5

7

15

5

9

5

5

Number of 
interviewers 
with same 
background as 
target groups

13 Turkish 
background

1 Turkish 
background
13 North African 
background

3 Roma 
background

2 Asian 
background

5 Roma 
background

None

41  AAPOR (2011).

78

Fieldwork 

Country

Number of 
interviews

Number of 
interviewers that 
completed at least 
one interview

Interviews in the total sample 
conducted per interviewer

Mean  
(%)

Median 
(%)

Minimum 
(%)

Maximum 
(%)

DK

851

EE

FI

FR

EL

ES

HU

HR

IE

IT

LV

LT

LU

MT

NL

PL

PT

RO

SE

SI

SK

UK

401

502

1,640

1,023

1,563

1,171

538

425

1,722

614

404

402

411

1,270

429

1,078

1,408

802

404

1,098

1,216

51

18

23

78

36

50

50

25

15

61

16

30

18

15

34

16

19

75

24

24

75

66

2

6

4

1

3

2

2

4

7

2

6

3

6

7

3

6

5

1

4

4

1

2

1

5

3

1

2

1

2

3

8

1

6

3

4

8

3

4

3

1

3

4

1

1

< 0.5

8

< 0.5

< 0.5

< 0.5

< 0.5

< 0.5

< 0.5

< 0.5

1

< 0.5

1

1

< 0.5

1

< 0.5

< 0.5

< 0.5

< 0.5

< 0.5

1

< 0.5

< 0.5

12

12

7

9

7

6

11

15

6

12

9

14

10

6

22

21

6

20

11

5

9

Number of 
interviewers 
with same 
background as 
target groups

21 SSAFR 
background; 
20 Turkish 
background

12 Russian 
background

None

Unknown as 
ethnicity not 
captured

None

None

None

None

2 SSAFR 
background

None

7 Russian 
background

2 Russian 
background

4 SSAFR 
background

3 SSAFR 
background

11 Turkish 
background
14 NOAFR 
background
1 Asian 
background
2 SSAFR 
background

2 Ukrainian 
background

None

None

9 SSAFR 
background

None

10 Roma 
background

4 SSAFR 
background
10 SASIA 
background

Note: 

SASIA, immigrants and descendants of immigrants from South Asia; SSAFR, immigrants and descendants of immigrants 
from Sub-Saharan Africa.

Source:  FRA, EU-MIDIS II 2016

79

Second European Union Minorities and Discrimination Survey – Technical report

In a number of countries, the local survey companies 
already had in their roster trained interviewers with 
the same background as the relevant EU-MIDIS II target 
group, and with appropriate language skills. In some 
countries (Belgium, Denmark and Estonia), the local 
teams went to the effort of recruiting new interviewers 
and training them specifically for this project. However, 
many of the newly recruited interviewers struggled 
with the requirements of the project and resigned. In 
the Netherlands, the vast majority of the interviewing 
team was composed of experienced interviewers with 
Turkish or North African background.

To ensure the quality of the fieldwork, every national 
agency  was  required  to  re-contact  a  minimum  of 
10 % of respondents to validate interviewers’ work. 
This process provided the opportunity to confirm that 
data had been gathered from genuine respondents, 
that interviews had been conducted correctly, and to 
assess whether or not the respondent had been left 
with a good impression of the interviewer and the sur-
vey company: Ipsos MORI or the local contractor. Local 
contractors were required to use one of three methods 
to carry out these call backs:

 • By  telephone:  respondents  who  had  provided  at 
the  end  of  the  interview  a  telephone  number  for 
quality control purposes were contacted by a mem-
ber of the validation team and asked to confirm se-
lected details of their interviews.

 • By post: cards with return postage paid were sent 
to the respondents, asking them to confirm select-
ed demographic and survey details.

 • By personal visit: a supervisor visited the respond-
ent’s address and asked them to confirm selected 
details of the interview. The interviewer was asked 
to  accompany  the  supervisor  in  case  there  were 
queries.

Countries using random route were also required to 
check that the procedures had been followed correctly 
and  that  the  correct  addresses  had  been  selected. 
If problems were detected, national agencies were 
required to increase the number of checks carried out. 
Where serious problems were identified with the quality 
of an interviewer’s work, the interviewer in question 
was removed from the project, and their interviews 
were excluded from the final data. In less serious cases, 
interviewers were re-briefed to prevent future errors.

6�7�  Data processing and data 

cleaning

Nine countries used Dimensions software for their ques-
tionnaire script, which was programmed centrally by the 

Central Coordination Team (CCT) of the survey contrac-
tor – specifically their data processing team in the United 
Kingdom – while other countries programmed the script 
using other software. Countries using software other 
than Dimensions were required to submit their data 
in ASCII format, based on a standardised data map. In 
preparation for the main data delivery, interim data 
were requested after 10 %, 60 % and 100 % of inter-
views had been completed. After each interim delivery, 
the data were checked for routing errors. Where errors 
were detected, countries were asked to amend their 
scripts and re-contact any respondents that might not 
have been asked a question that they should have had 
the instructions of the questionnaire been followed cor-
rectly. The local contractors were also required to check 
the quality of their data on a regular basis. To ensure 
the quality of the data, the CCT wrote the checking 
syntax centrally for the statistical programme SPSS, and 
distributed it to countries to run on their data before 
submission. The syntax converted the ASCII data into 
SPSS format and then checked that:

 • the data matched the data map;

 • the ID numbers were unique and there were no du-
plicate records (unless there were two interviews in 
the same household);

 • the routing had been adhered to;

 • only permitted values had been entered;

 • the  household  grid  and  demographic  information 
did not contain any inconsistencies (‘sense check’).

All countries were required to check their data using 
this syntax and amend accordingly. Firstly, countries 
were required to go back to their raw data files to check 
if any mistakes were processing errors which might 
have occurred when data were transferred from one 
file or format to another. If this was not the source 
of  the  detected  error,  countries  were  instructed  to 
raise the issue with the interviewer who had admin-
istered  the  questionnaire.  If  any  data  were  miss-
ing, the respondent would be re-contacted to collect 
the missing information.

Once the data were submitted to the CCT, the checks 
were re-run to ensure that there were no outstand-
ing errors. In addition to the checking syntax, any very 
short or very long interviews, as well as interviews with 
high levels of item non-response were investigated. The 
CCT also ensured that every interview could be matched 
to the associated contact sheet.

The questionnaire contained a number of questions with 
an ‘other – specify’ response code. Where a respond-
ent selected this option, their verbatim response was 

80

Fieldwork 

captured (open text). During data processing, these 
responses were translated into English, and these were 
back coded into the existing code frame where possible. 
In some instances, the back coding prompted follow-up 
questions that had been skipped during the interview 
on the basis of the initial coding of the response. In 
these cases, the data in the follow-up questions are 
recorded as missing.

Data from all 28 EU Member States were combined into 
a single data file. In this process, the country-specific 
questions needed to be harmonised into a single vari-
able to allow for comparison. This included all education 
and income questions. To minimise the size of the data-
set, it was also agreed that country-specific questions 
regarding the Equality Bodies would not be included. 
Instead, these are harmonised, and the codebook can 
be used as a reference to determine which Equality 
Body is being referred to in each country. The same 
approach was also taken for recording a respondent’s 
level of comprehension of the country’s national lan-
guage and the frequency of attending selected loca-
tions for countries using location sampling.

Interviews with a high level of item non-response were 
excluded from the final dataset. For EU-MIDIS II, a high 

level of item non-response was defined as interviews 
where more than 50 % of the questions asked were 
answered with ‘Do not know’, ‘Refused’ or ‘Does not 
understand the question’. In addition, any interview 
which had between 25 % and 50 % item non-response, 
and did not have an answer to a key variable which 
would be used to develop important indicators, was 
also excluded from the final dataset. The list of impor-
tant indicators was defined separately for the target 
groups ‘Roma’, ‘Russian minority’ and ‘immigrants and 
descendants of immigrants’.

Data from the contact sheets were checked as well for 
correctness and if they matched correctly the cases in 
the main dataset. For countries using Focused Enumera-
tion and Adaptive Cluster Sampling, additional checks of 
the contact sheet data were made by the CCT and FRA.

Due to the higher levels of missing data on income and 
the importance of having information on household 
income as well as age and gender for all household 
members  for  calculating  social  inclusion  indicators, 
missing data for the exact household income have been 
imputed using standard methods of imputation.42

42  Data imputation means that values were inserted for missing 

data.

81

7  

Weighting

This  chapter  describes  the  weighting  approaches 
applied to the EU-MIDIS II dataset. Weighting is neces-
sary to adjust the statistics produced in accordance with 
the sample selection process, and to correct for poten-
tial deficiencies due to non-response. The weighting 
approaches for EU-MIDIS II took account of the sample 
design for each country. See Chapter 4 on Sampling, for 
a description of the sampling designs.

7�1�  Weighting procedures: 

clustered and 
unclustered samples 
(sample designs 1 and 2)
This weighting approach was applied in the major-
ity of countries. Three types of weights were used to 
weight these samples:

1) design weights;

2) non-response weights; and

3) post-stratification weights.

Design weights

Design weights were required to equalise differential 
selection probabilities arising from the sample design. 
On the EU-MIDIS II samples, this could occur at any of 
the following stages:

1.  PSU selection;

2.  address or individual selection;

3.  dwelling unit and/or household selection;

4.  respondent selection; and

5.  respondent referral, in Germany only, where 

respondents were also recruited through referrals.

The probabilities calculated at each stage were mul-
tiplied to provide the overall probability of inclusion 
for each case in the sample, and the design weights 
were calculated as the inverse of this probability. In 
countries with multiple target groups sampled together, 
the  target  group  membership  was  ignored  at  the 
design weighting stage.

PSU selection (clustered samples)

In most countries, differential selection probabilities 
applied across target group concentration strata, as 
strata with higher concentration of the target groups 
were oversampled. The probability of selection was 
recorded at the PSU selection stage so that it could be 
corrected. Table 4.13 reports the impact on precision 
from this stage of weighting.

In most countries, PSUs were selected with PPS. See 
Table 4.13 for the method used, by country. This resulted 
in  addresses  being  selected  with  equal  probability 
within concentration strata, when equal numbers of 
addresses were selected at the next stage.

For samples based on addresses, the PPS size meas-
ure would ideally be the total number of addresses 
belonging to the members of the target group within 
each PSU. In some countries, it would be the number of 
households or families. Where this was not available, 
the total population size was used instead. Samples 
drawn from individual registers used the total target 
population size as the PPS size measure. The weight 
adjustment was based on the PSU size measure used 
for selection, i.e. the actual probability of selection, 

83

Second European Union Minorities and Discrimination Survey – Technical report

irrespective  of  whether  or  not  this  was  the  ideal 
measure (e.g. addresses).

The probability of selection of a PSU at this stage was:

PSU selection probability = 

NPSUh × size PSU
∑ size PSUh

where  N  PSU   h  =  number  of  PSUs  selected  in  con-
centration  stratum  h,  size  PSU  =  size  of  the  PSU 
selected and Σ size PSUh = sum of sizes of all PSUs in 
concentration stratum h.

The PSU selection weight (DWPSU) was the reciprocal 
of its probability.

Address or individual selection within PSUs 
(clustered samples)

In the clustered samples, addresses had different selec-
tion probabilities across PSUs, given that the PSUs are 
of different sizes. The numbers of addresses selected 
were designed to deliver similar numbers of interviews 
in each PSU. The probability of selection of each address 
(conditional on selection of its PSU) was:

Address or individual selection probability = 

Number of addresses selected 

Number of addresses in the PSU

Where  the  number  of  addresses  (or  households  or 
families) in the PSU was not available the number was 
estimated. This was typically based on survey data 
on the size of households at an appropriate level of 
aggregation (concentration strata, regions, rurality), 
checked against population data. Survey data were 
considered (and not just population statistics) because 
most of the target groups live in households larger than 
the country average.

The address or individual selection weight (DWUNIT) was 
the reciprocal of its probability.

Further adjustments for focused 
enumeration (FE) and adaptive cluster 
sampling (ACS)

No additional adjustment was required in PSUs where 
focused enumeration was used as a sampling method; 
the  number  of  addresses  selected  and  used  in  the 
probability calculation included both core (centre) and 
focused enumeration (two neighbours either side of the 
core) addresses. Hence, the sample address selection 
weight, DWUNIT, was based on all addresses in FE PSUs.

For ACS further adjustments were required. Firstly, at 
the address-selection stage, fewer addresses were 
selected (than would be needed for the same number of 
interviews without ACS) and additional addresses were 
added in accordance with the ACS procedures. The prob-
ability of selection of the initial set of (core) addresses 
was therefore based on the number initially selected.

Additionally, in theory any address in an ACS chain of 
neighbouring eligible addresses would be included in 
the sample if any one of the addresses in the chain 
was initially selected, as the addresses neighbouring 
an eligible address are added to the sample. Therefore, 
the probability of selection of all addresses in ACS PSUs, 
whether selected at the start or through association 
with other eligible addresses, could be considered to be 
in proportion to the number of addresses in the chain. 
For example, an address which neighboured one other 
eligible address had twice the chance of selection as 
an eligible address on its own; addresses in chains of 
three eligible addresses had triple the single eligible 
address selection probability; and so on.

This conclusion, however, depends on two assump-
tions: 1) that all addresses are successfully screened 
for eligibility and 2) that the chain size is not capped. In 
reality, many addresses were not successfully screened 
for eligibility, and the approach meant that a chain with 
a sequence of outcomes of 0011100 received the same 
weight as one of XX111XX (where X = no information 
available; 0 = ineligible household; 1 = eligible house-
hold). Furthermore, the full length of the chains was not 
always established if the fieldwork was stopped in the 
PSU upon reaching the set maximum number of inter-
views (see stopping rule description in Section 4.4). 
In practice, the second issue, capping, occurred very 
rarely, so it is unlikely to have had any impact on the 
weighted survey estimates.

Following on from this discussion, the probability of 
selection of an address in ACS PSUs was:

ACS address selection probability = 

∙

Number of addresses initially selected in PSU

Number of addresses in the PSU

∙

× Number of eligible addresses in its chain

The ACS PSU address selection weight (DWUNIT_ACS) was 
the reciprocal of its probability.

Note that further capping of ACS chain lengths was con-
sidered at the weighting stage, to limit the impact on 
the efficiency of the sample, but the final chain lengths 
were not long enough to warrant it.

84

Household selection (individual register 
samples)

The  dwelling  unit  (DWDU)  selection  weight  was  the 
reciprocal of its probability.

In the countries using individual-level registers one of 
two selection approaches was used in the field:

Weight for referrals (used in Germany)

Weighting 

1.  The individual named on the register sample was 
treated as the sampled unit and only this person 
was interviewed (if eligible); or

2.  The household (that the named individual resided 
in) was treated as the sampled unit and up to 
two individuals living in that household were 
randomly selected for the interview out of all 
eligible individuals, even if this resulted in not 
selecting the named individual.

The  first  procedure  required  no  further  weighting 
at this stage.

For  the  second  procedure,  because  it  could  not  be 
ascertained in advance which individuals on the sam-
ple belonged to the same households, the probability of 
selection of households varied according to the number 
of eligible individuals within them and required further 
weighting. This was equal to (conditional on selection 
of a household member):

Number of eligible people in the household.

The household selection weight for individual regis-
ter samples (DWUNIT_IND) was the reciprocal of its prob-
ability. In samples that were both clustered and from 
an individual register (as in Slovenia), this weight was 
additional to the DWUNIT weight.

Dwelling unit (clustered samples)

If an interviewer discovered in the field that an issued 
address was made up of multiple dwelling units, they 
selected one dwelling unit randomly. The probability 
of selection of a dwelling unit (conditional on selection 
of its address) was:

Dwelling unit selection probability = 

1

Number of dwelling units or house-
holds at the address

Random selection of dwelling units was only required 
for address register samples (because the random route 
procedures identify unique addresses at the outset). 
Similarly, random household selection was not required 
for individual registers because the selected individual 
could belong to only one household.

In Germany, an additional weight was added to account 
for the probability of selection of referral addresses. 
The probability of selection of each referral address 
was based on the total network size (number of refer-
rals provided by each initial sampled address) and the 
number of referral cases selected:

Probability of selection of each referral address = 

Referral cases selected

Referral cases provided (size of network)

The referral selection weight (DWIND_R) was the recipro-
cal of its probability. This weight was applied to refer-
ral cases after application of the full set of weights 
up to this stage. This included PSU selection, address 
selection – see above – and household non-response 
weight, described below.

Individual respondent selection

Whenever a household contained more eligible individ-
uals than the survey protocols permitted for interview, 
the interviewer selected the allowed number of individ-
uals randomly (i.e. one for Roma and Russian minority 
target groups and two for all other target groups). The 
probability of a respondent being selected (conditional 
on selection of their household) was:

Probability of respondent being selected = 

1 (Roma/Russian minority) or 2 (all others)

Number of eligible people in the household

The individual selection weight (DWIND) was the recipro-
cal of its probability.

Non-response weights

Non-response weighting relies on having information at 
respondent or address level that covers both responders 
and non-responders. This is usually obtained from the 
sample frame, and in some countries where popula-
tion registers were used to make contact with sam-
pled individuals directly, this approach was considered. 
In Finland, however, demographic variables were not 
obtained from the register and therefore an additional 
adjustment was not possible. In the absence of sam-
ple frame information the details can be collected by 
interviewers in the field, which was the approach used 
for most of the EU-MIDIS II samples. On EU-MIDIS II, 
this approach was employed due to the lack of demo-
graphic population data covering the target group that 

85

Second European Union Minorities and Discrimination Survey – Technical report

could be used for post-stratification weighting (the final 
weighting step) – and so estimating and correcting for 
non-response from the sample frame was important.

Non-response weighting involved the following steps:

1.  Neighbourhood characteristics (recorded on the 

contact sheet) were used to correct non-response 
by households, conditional on household target 
group eligibility; and

2.  selected respondent characteristics (recorded 
on the contact sheet) were used to correct for 
nonresponse by individuals, conditional on their 
selection for an interview.

The  weights  were  calculated  in  separate  stages, 
first  household  nonresponse  weights  (NRW HH), 
where  the  household-level  achieved  sample  was 
weighted  to  the  neighbourhood  characteristics  of 
the  overall  issued  and  confirmed  eligible  sample, 
after applying the design weights up to the house-
hold  selection  stage  (DWPSU  ×  [DWUNIT  or  DWUNIT_ACS] 
[× DWUNIT_IND] × DWDU × DWHH).

Following this, individual nonresponse weights were 
calculated (NRWIND), by weighting the individual-level 
data to the respondent characteristics of all selected 
individuals, after applying the full sequence of applicable 
weights up to that stage (DWPSU × [DWUNIT or DWUNIT_ACS] 
[× DWUNIT_IND] × DWDU × DWHH × NRWHH × DWIND).

Both forms of weights used logistic regression to fit 
a model with response to the survey (as a binary vari-
able) as the dependent variable. The weight was cal-
culated as the reciprocal of the probability of response 
generated by the model.

Post-stratification or calibration 
(individual and household) and final 
weights

The last stage of in-country weighting was to weight 
the individual-level data to the population profile of the 
covered target group. This stage aimed to correct for 
remaining deficiencies due to non-response not already 
corrected by the non-response weighting.

Post-stratification or calibration weighting adjustments 
depend on reliable population data covering the sur-
vey population. The challenge for EU-MIDIS II was that, 
because the target groups typically made up a small 
proportion of the overall country populations, profile 
data from other surveys could not be used (due to small 
sample sizes). Moreover, in most countries EU-MIDIS II 
did not cover the full target group population in the 
country. That complicated it further.

Therefore, the variables that were used in the post-
stratification were taken from the EU-MIDIS II country 
sample frames, specifically region and urbanisation. 
These were available for all countries, including the val-
ues of interlocking cells. The weighting profiles were 
based on the covered target group, rather than also 
attempting to use the weighting to correct for cover-
age. This approach was taken because weights are not 
able to ‘correct’ for gaps in the sample where there are 
no similar cases that can be used to represent those 
that are missing.

Post-stratification weights were calculated for each 
target group separately, after applying the full set of 
applicable  design/non-response  weights  described 
above. The calculation used cell weighting. The weights 
that were applied at this stage were gross population 
weights, based on the covered target group population 
of all ages,43 so that weighted sample estimates took 
the same total value as the total size of the covered 
target group population in each country. This means 
that a single weight can be used for any within- or 
cross-country analyses that may be required. This final 
individual weight was applied to the interviewed cases.

Given that other information potentially used for post-
stratification, such as gender and age structures, was 
not (easily) available for the covered target popula-
tion, no adjustments for gender and age were made 
in this phase for most countries (but in the early non-
response adjustment). The exceptions were Finland 
and the Netherlands. There, additional adjustments 
were  made  using  available  demographic  variables. 
Evaluations were undertaken for these samples due 
to concerns with the sampling and because there were 
statistics available to make comparisons. Specifically, 
in Finland there were coverage losses because most 
individuals sampled could not be matched with a tel-
ephone number. In the Netherlands the sample design 
was changed to location sampling part-way through 
fieldwork and the full address sample was not realised.

These samples were adjusted to population charac-
teristics taken from the population register and based 

43  Estimates of the size of the covered population were taken 

directly from the sample frames; see Table 4.11 in Section 4.3 
for a list of data sources. The post-stratification population 
targets were based on the population of all ages, rather 
than adults, because data were available at this level for 
the majority of countries. In a handful of countries, the 
population figures available were for adults only; in these 
instances, the figures have been inflated based on the 
survey data (number of children in household). A similar 
procedure was used to inflate first generation target group 
population figures to include second generation where this 
was necessary. Particularly in Cyprus, the proportion of 
citizens of Cyprus in the survey data was used to inflate the 
citizenship figures provided in the sample frame. Similarly, 
in Poland the sample frame figures were inflated by the 
proportion of non-registered immigrants in the survey data.

86

on  the  covered  population  (Finland)  or  a  combina-
tion of the covered and total population depending 
on data availability (the Netherlands). The following 
variables were used:

 • Finland: NUTS3, age (in four bands) and gender (all 

to covered population totals);

 • The  post-stratification  weighting  adjustment  cells 
were capped to the next-nearest weight if outliers 
were observed.

 • The  final  individual  weight  was  trimmed  at  the 
99th  percentile  (after  applying  post-stratification 
weights).

 • Netherlands: NUTS3 (to covered population totals); 
and generation (first or second), country of origin 
for  immigrants  and  descendants  of  immigrants 
from North Africa (Morocco or other), age (in five 
bands) and gender (demographics to 16 + total tar-
get group population proportions).

The raking calibration weighting procedure was applied 
using  the  statistical  software  package  Quantum. 
For both countries the adjustment was made to the 
full design-weighted samples.

A further weight was calculated, the final household 
weight, to be used for analyses based on all cases in 
each household (which were provided in the final data-
set). This weight was calculated as the final individual 
weight divided by the number of people (of all ages) 
in each household. This meant that the weight was 
consistent with the individual weight, and weighted 
analyses based on either weight add up to the covered 
target group population of each country. This weight 
was applied to each case in the dataset (one case per 
household member, whether interviewed or not).

Trimming weights

The weighting procedures resulted in large variations 
between weights. Large weights can result in substan-
tial losses in sample efficiency, so it is common practice 
to trim weights. Weights were therefore trimmed at 
several stages during the weighting. Broadly, the fol-
lowing trimming was considered:

 • The  full  address  selection  weight  (PSU  and  ad-
dress unit selection weights combined) was usually 
trimmed at the 5th and 95th percentiles of the dis-
tribution. This was the maximum level of trimming 
applied. Less trimming was applied if the difference 
between the smallest and largest weight was with-
in  10,  or  could  be  brought  within  10  by  a  smaller 
amount of trimming, or if no trimming was required 
to reach this level.

 • The 

individual  selection  weight  was  always 

trimmed at the 97.5th percentile.

 • The address referral weight (for the sample in Ger-

many) was trimmed at the 95th percentile.

 • Location sampling weights were trimmed if outliers 
were observed (by setting the weight for the loca-
tion to the size of the next largest weight).

This iterative process was used as it means that the 
effect of later stages of weighting is retained in the 
sample. If all trimming were at the end, this would dis-
proportionally affect certain cases, e.g. those that were 
initially sampled at a lower rate because they were in 
less concentrated strata.

7�2�  Weighting procedures 
– location sampling 
(sample design 3)

The weighting strategy for the location sample fol-
lowed the approach described in the paper by Baio et 
al. (2011).44 For countries that included more than one 
city or region (all countries where location sampling 
was used, except Malta), the weights were generated 
separately for each region and then adjusted to be in 
proportion to the size of the target population when 
the regions were combined. This is a requirement of 
the method; if several cities were included in a sin-
gle weighting scheme, the estimates of importance 
of the locations would be too low in some regions/
locations, as it would be uncommon for members of 
the population to visit locations across multiple cities. 
The method requires a certain amount of overlap of 
members visiting different locations, which warranted 
treating each region separately.

In weighting, the locations were considered not as sin-
gle specific locations, but rather types of location. For 
example, rather than considering each religious centre 
in a region separately, they were treated as a single 
entity. This is consistent with the approach described in 
the Baio et al. (2011) paper,45 and has the advantage of 
increasing the sample sizes and so gives more precise 
estimates of attendance to be used for the weighting.

For  each  region/country,  the  baseline  location  was 
defined to be the location that was visited by the high-
est proportion of the eligible population. In most cases, 
one location dominated, so this decision was clear cut. 

44  Gianluca Baio, G. et al. (2011), pp. 451–465.
45  Ibid.

Weighting 

87

Second European Union Minorities and Discrimination Survey – Technical report

For others, the location that appeared most important 
was selected initially and then a measure of relative 
importance was calculated and checked for each loca-
tion (rk; see calculations for locations) – this usually 
confirmed the choice of the baseline location. In a few 
cases, the measure of relative importance indicated 
that an alternative location should be defined as the 
baseline, and in these cases the baseline location was 
changed to that identified as the most important.

In countries where a register or clustered address sam-
ple was used to select some of the sample (and there-
fore a combined approach used), the register/address 
sample was defined to be the baseline location, and 
participants sampled at locations were asked where 
they lived so that it could be determined whether they 
could have been sampled from the register. In Poland, 
an additional question on whether respondents have 
a permit to stay in the country was used to identify 
participants sampled at locations who could not have 
been sampled from the register which does not include 
persons without a permit. The full set of weights were 
applied  to  the  register/address  component  of  the 
samples before the location sampling weighting steps 
described below; this applied in Cyprus and Poland.

In the Netherlands this approach was not used, because 
location sampling was added partway through field-
work and, in spite of additional efforts to re-contact 
sample members, location visit information was not 
available for enough of the register sample. Instead, the 
location sampling and address sampling components of 
the Netherlands sample were weighted separately and 
combined in proportion to their respective sample sizes 
before the final calibration weighting stage.

Calculations for locations

Using the same notation as in Baio et al. (2011),46 the 
following were estimated for each location k:

1. The proportion of respondents at the baseline 

location that also visited location k:

3. The ratio (rk) of Ak to Bk:

rk = Ak/Bk

This is the survey-based estimate of the relative preva-
lence of visits to location k compared with the baseline. 
It can be considered to be a measure of the relative 
importance of each location k compared with the base-
line location. A measure of rk greater than 1 implied that 
location k was more important than the baseline loca-
tion. The estimates of rk were used to test that the most 
important location had been selected as the baseline. If 
another location was identified as the most important 
(rk > 1), then it was re-defined as the baseline location 
and the steps above were repeated.

4. The proportion of interviews carried out 

at location k:

θk = nk/nall interviews

5. A measure of how over- or under-represented 

each location was:

δk = θk/rk

If a location has a value of δk that is greater than that 
of the baseline location, it implies that more interviews 
were carried out at that location than would be required 
based solely on its relative importance. Conversely, if 
the value of δk is less than that of the baseline loca-
tion, it implies that fewer interviews were carried out 
at that location than would be required based solely on 
its relative importance.

For some locations, due to small sample sizes or there 
being little or no overlap with the baseline location, 
there was an extreme value of δ (e.g. zero if no overlap). 
If left unamended, this would have resulted in extreme 
weights, which would have reduced the efficiency of 
the sample. To avoid this, extreme values of δ were 
trimmed to the next largest estimate for high values or 
to the next lowest estimate for small values.

Ak = n baseline that also visited k/nbaseline

Individual weight

(Note that, when a register was the baseline location, 
then Ak was estimated based on a weighted register 
sample to obtain unbiased estimates.)

2. The proportion of respondents at each location that 

also visited the baseline location:

Bk = nk that also visited baseline/nk

If a participant only visited one location (e.g. location s), 
then their weight is calculated to be:

weight = 1/δs

That means that a participant who visited a location 
that was under-represented compared with the base-
line location would be given a higher weight relative to 
the baseline location. This is consistent with what could 
be expected for the weights.

46  Ibid.

88

The weight for a participant who visited more than one 
location (e.g. locations s, t and u) is calculated from the 
corresponding values of δ as:

weight = 1/(δs + δt + δu)

This means that a participant who visited more than 
one location is given a lower weight than a participant 
who visited only one of them. This again is consist-
ent with what can be expected for the weights and 
acknowledges the increased likelihood of participation 
of that participant.

In general, the formula for the weights for each partici-
pant was calculated as:

weight = 1/∑ (Ci × δi)

where  Ci  =  1  if  the  participant  visited  location  i, 
and otherwise 0.

Before finalising the weights they were checked to 
make sure the weights had no extreme values, which 
would reduce the efficiency of the sample. All small 
weights were trimmed to the next highest value and 
any large weights to the next smallest value, so that 
the largest weight was not more than approximately 
10 times larger than the smallest one.

Post-stratification weight

Post-stratification weights were calculated for each 
target group separately, after applying the location 
sampling  weight  described  above.  The  calculation 
used cell weighting. As with the other samples, the 
post-stratification weights were to the size of the target 
group population at region level, and were based on the 
covered target group population of all ages (i.e. if some 
but not all of the regions of a country were included 
in the sample, the control totals included the target 
group population sizes of covered regions only). This 
final individual weight was applied to the interviewed 
cases. The combined location/address sample in the 
Netherlands was weighted using calibration weighting 
to available demographic variables.

As with the other samples, a final household weight 
was also calculated to be used for analyses based on 
all individuals in each household (which were provided 
in the final dataset). This weight was calculated as the 
final individual weight divided by the number of people 
(of all ages) in each household. This weight was applied 
to each case in the dataset (one case per household 
member, whether interviewed or not).

7�3�  Weighting procedures – 
quota sampling (sample 
design 4)

In Luxembourg, no population figures could be obtained 
on which to base the weighting. Therefore, a uniform 
weight was calculated to gross the weighted sample to 
the total estimated population size. For sample frames 
for the source used, see Table 4.11.

7�4�  Weighting efficiency

Table 7.1 provides the range and ratio (largest to small-
est) of the final individual and household weights for 
each country and target group (standardised to a mean 
of 1 for each sample). It also provides the weighting 
efficiency of the samples. This is calculated as the 
ratio  of  the  effective  sample  size  to  total  sample 
size.  The  effective  sample  size  is  calculated  using 
the standard Kish formula (sum of weights squared 
divided  by  sum  of  squared  weights).  The  efficien-
cies therefore do not take account of the impact of 
clustering or stratification.

In the majority of countries, the total weighting effi-
ciencies shown here are similar to the levels predicted 
at the design stage (see Table 4.12 and 4.13), when 
efficiencies were estimated based on whether or not, 
and to what extent, the designs oversampled areas 
of higher target group concentration (made for non-
location sampling samples). Of these, the majority of 
samples are within 50 % of the design-based levels 
presented in Table 4.13, meaning that the dispropor-
tionate sample design contributed at least two thirds 
of the efficiency loss in these countries. Further reduc-
tions in efficiency can broadly be explained by the 
impact of the additional planned weighting stages 
(random selection of individuals for interview; house-
hold and individual level non-response weighting; and 
post-stratification weighting regional adjustments). 
Six samples are above this level (the Czech Repub-
lic, Hungary, Portugal, Romania, Slovakia and Spain 
(Roma)), i.e. all except three of the Roma samples. 
The Roma samples were further impacted by (i) the 
selection of one individual per household (instead of 
two) which requires larger weights; (ii) reducing the 
size of the sample during fieldwork to avoid delivering 
too many interviews which increased weight varia-
tion (this was required in Hungary, Portugal, Romania 
and Spain); and (iii) the use of ACS which requires 
additional weighting adjustments.

Weighting 

89

Second European Union Minorities and Discrimination Survey – Technical report

Table 7�1: Range and efficiency of final individual (ind) and household (hh) weights

Country

AT

BE

BG

CY

CZ

DE

DK

EE

EL

ES

FI

FR

HU

HR

IE

IT

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

Target 
group

SSAFR

TUR

NOAFR 

TUR

ROMA

ASIA

ROMA

SSAFR

TUR

SSAFR

TUR

RUSMIN

ROMA

SASIA 

ROMA

NOAFR 

SASIA

SASIA

NOAFR

ROMA

ROMA

SSAFR

SSAFR

NOAFR 

SASIA

RUSMIN

SSAFR

RUSMIN

SSAFR

NOAFR 

TUR

RIMGR

ROMA

SSAFR

ROMA

SSAFR

TUR

RIMGR

ROMA

SSAFR 

SASIA

Range (ind) Ratio (ind)

Efficiency 
(ind) (%)

Range (hh)

Ratio (hh)

Efficiency 
(hh) (%)

0.14–1.42

0.18–4.99

0.05–4.6

0.14–5.06

0.11–3.49

0.26–7.08

0.06–4

0.35–3.57

0.01–4.14

0.3–3.63

0.23–2.42

0.15–2.79

0.19–2.92

0.2–5.11

0.08–7.71

0.1–5.82

0.3–1.21

0.07–5.66

0.12–5.47

0.05–7.16

0.26–2.85

0.11–6.22

0.07–8.52

0.02–6.74

0.04–7.2

0.25–3.51

1–1

0.2–4.18

0.13–3.94

0.18–3.83

0.13–4.71

0.17–2.83

0.11–5.47

0.24–4.36

0.08–3.67

0.12–2.04

0.01–2.24

0.23–2.22

0.09–3.74

0.07–4.6

0.08–7.06

10.1

27.7

92.0

36.1

31.7

27.2

66.7

10.2

414.0

12.1

10.5

18.6

15.4

25.6

96.4

58.2

4.0

80.9

45.6

143.2

11.0

56.5

121.7

337.0

180.0

14.0

1.0

20.9

30.3

21.3

36.2

16.6

49.7

18.2

45.9

17.0

224.0

9.7

41.6

65.7

88.3

84

54

55

52

69

53

66

75

60

71

79

76

76

46

53

51

90

53

51

43

78

36

31

32

35

71

100

63

66

63

56

78

52

63

64

73

73

86

66

58

42

0.05–2.83

0.18–12.95

0.04–7.85

0.06–13.03

0.15–3.68

0.13–5.83

0.05–4.5

0.18–6.21

0.01–7.84

0.13–11.43

0.17–4.95

0.09–2.85

0.2–3.39

0.07–5.73

0.06–5.53

0.1–7.59

0.1–3.7

0.05–15.52

0.06–15

0.06–11.08

0.23–10.18

0.07–8.76

0.06–11.48

0.02–12.51

0.06–13.51

0.13–2.79

0.38–3

0.14–3.71

0.09–7.58

0.08–12.48

0.06–13.86

0.08–5.52

0.13–6.28

0.15–10.56

0.1–6.14

0.04–5.5

0.01–6

0.1–4.62

0.11–7.22

0.03–10.96

0.04–12.06

56.6

71.9

196.3

217.2

24.5

44.8

90.0

34.5

784.0

87.9

29.1

31.7

17.0

81.9

92.2

75.9

37.0

310.4

250.0

184.7

44.3

125.1

191.3

625.5

225.2

21.5

7.9

26.5

84.2

156.0

231.0

69.0

48.3

70.4

61.4

137.5

600.0

46.2

65.6

365.3

301.5

62

44

49

38

75

50

69

59

55

46

69

83

81

43

58

47

57

42

41

45

76

34

30

35

36

76

69

69

52

44

36

65

54

51

65

46

45

58

67

48

34

Notes:  ASIA, immigrants and descendants of immigrants from Asia; NOAFR, immigrants and descendants of immigrants from 
North Africa; RIMGR, recent immigrants; ROMA, Roma; RUSMIN, Russian minority; SASIA, immigrants and descendants 
of immigrants from South Asia; SSAFR, immigrants and descendants of immigrants from Sub-Saharan Africa; TUR, 
immigrants and descendants of immigrants from Turkey.

Source:  FRA, EU-MIDIS II 2016

90

8  

Lessons learned

FRA’s  Founding  Regulation  requires  the  agency 
(Article 4.1b) to develop methods and standards to 
improve the comparability, objectivity and reliability of 
data at the European level. This technical report outlines 
in detail how the data for the EU-MIDIS II survey were 
collected. Information on how the data collection was 
implemented provides insight in terms of interpreting 
the results, but details of the data collection process 
also contribute to the iterative improvement of survey 
methodologies. EU-MIDIS II is an indispensable and 
unique source of information for policy development 
and evaluation in the area of fundamental rights. Its 
methodology is based on approaches developed for 
EU-MIDIS I in 2008 – the first ever survey on immigrants 
and ethnic minorities in all EU Member States – and the 
Roma pilot survey in 2011.

In  the  past  decade,  the  demand  for  data  on  immi-
grants and ethnic minorities has increased. This is also 
reflected in the European Commission’s efforts to pre-
pare, evaluate and present indicators on immigrant 
integration (the ‘Zaragoza indicators’),47 and related 
studies.48 For example, while the data on immigrant 
integration that Eurostat prepared49 represent a key 
source for these indicators, they cannot satisfy all data 
needs. The data on immigrant integration are mainly 
based on EU’s high quality data collection tools, includ-
ing the EU Labour Force Survey (EU-LFS), EU Statistics on 
Income and Living Conditions (EU-SILC) and Eurostat’s 
migration statistics, as well as OECD’s Programme for 
International Student Assessment (PISA). Many of the 
existing sources of survey data are general popula-
tion surveys. Although these are the best sources of 
comparable data in their respective areas, data are 

47  Council of the European Union (2010).
48  Eurostat (2011); Huddleston, T. et al. (2013); OECD (2015).
49  See the Commission’s webpage on migrant integration 

statistics. 

still lacking for some countries, for some target groups 
and on some topics, given that surveys of the general 
population are severely limited when it comes to pro-
viding data on immigrants. This lack of data is a result 
of low samples of immigrants in general population 
surveys in some countries, so data and indicators are 
unavailable and/or do not allow additional breakdowns 
for particular groups. For example, it is difficult to col-
lect data from a high-quality sample of immigrants in 
countries where they constitute a low share of the total 
population, for example in some Central and Eastern 
European countries, such as Bulgaria, Poland or Slo-
vakia. It is also not possible to obtain information on 
immigrants and descendants of immigrants from Sub-
Saharan Africa in most EU Member States, because 
they make a small proportion of the total population. 
At the same time, certain ethnic minorities and immi-
grants and their descendants are particularly vulner-
able to various disadvantages and violations of their 
fundamental rights. Some of the information needed 
to assess and evaluate existing policies is not avail-
able from existing data sources, most notably data on 
discrimination against and victimisation of vulnerable 
groups. Furthermore, existing general population sur-
veys do not always ask questions that would enable 
identification of ethnic minorities and immigrants and 
descendants of immigrants in the sample, or use of the 
survey to analyse their experiences.

To obtain information and data about particular vulner-
able groups, efforts such as EU-MIDIS II are necessary to 
provide information that helps to design policies better. 
EU-MIDIS II fills a gap in data availability and contrib-
utes to the further development of methods to gather 
information about groups that are considered hard to 
reach or difficult to survey.

This  technical  report  is  a  source  of  information  for 
promising  practices  in  collecting  data  about  ethnic 

91

Second European Union Minorities and Discrimination Survey – Technical report

minorities and immigrants, at the international and 
national levels. It not only provides a benchmark in 
terms  of  full  transparency  concerning  data  collec-
tion methods used, but should also help researchers 
to improve the design of future studies and surveys 
covering hard-to-reach groups.

The following points were observed in the course of 
the project and deemed important to take into account 
when designing a survey on ethnic minorities, or immi-
grants and descendants of immigrants, in the future.

Before developing and deciding on any survey design, 
collecting background data and information on compo-
sition and concentration of the groups to be surveyed is 
an important task for every data collection. An informed 
decision on the best possible sampling design can only 
be taken after a comprehensive mapping of information 
on the target population in each country to be surveyed. 
The extensive background data collection carried out 
in EU-MIDIS II was crucial to the success of the survey.

The results of the cognitive pre-test study of EU-MIDIS II 
proved indispensable for a sound understanding of 
whether or not respondents with diverse backgrounds 
understood  the  question  wording,  and  the  specific 
terms used, as the research team intended. The out-
come of the pre-test informed the final revision of the 
questionnaire for the pilot and the full-scale survey. 
Future surveys could also consider the need to develop 
an easy-to-read version of the questionnaire for per-
sons with learning difficulties, as a simplified version of 
the questionnaire could also help in overcoming cultural, 
language and cognitive barriers.

Translation procedures play a central role in multina-
tional and multicultural surveys. FRA used the so-called 
TRAPD (translation, review, adjudication, pre-testing, 
and documentation) team translation model, a team-
based approach to survey translation. It proved very 
successful and can be recommended for future EU-wide 
surveys of persons with ethnic minority or immigrant 
background. As already recommended in existing sur-
vey literature,50 the documentation of each step of the 
translation process (for example in an Excel format) 
was used as a quality assurance and monitoring tool. 
For complex projects with diverse target groups and 
different languages – such as EU-MIDIS II – the plan-
ning, scheduling and documentation of the translation 
process should be done very carefully, and sufficient 
time needs to be reserved for this step.

Cooperation with National Statistical Institutes (NSIs) is 
of major importance for the success of surveys that aim 
to achieve a properly designed probability sample. The 
NSIs often have access to high quality sampling frames 

50  Survey Research Center (2016).

of the general population, and in some countries these 
sampling frames can also be used for surveying immi-
grants and ethnic minorities. When data collection is 
not carried out by the NSIs themselves, however, there 
may be restrictions concerning access to certain sam-
pling frames. The experience in EU-MIDIS II showed that 
requests for cooperation with NSIs cannot be made on 
a ‘one size fits all’ basis, because of the survey’s com-
plex sampling approach. Instead, requests for data had 
to be bilaterally negotiated with the NSIs and tailored to 
the needs of each country and to the specific require-
ments of the survey. It was therefore important to start 
well ahead of fieldwork to contact the NSIs and explore 
opportunities for cooperation and request relevant 
population data. Some of the delays or interruptions of 
fieldwork experienced in EU-MIDIS II were due to delays 
in getting access to the relevant sampling frames.

Meetings with local survey companies that undertake 
fieldwork  should  take  place  immediately  after  the 
inception meeting with the contractor and before the 
background research is completed, to better inform 
work related to sampling frames.

EU-MIDIS II used a relatively complex screening proce-
dure to identify potential respondents: In many cases 
questions on country of birth and country of birth of 
parents or questions on self-identification with a spe-
cific minority group had to be made at the doorstep to 
determine if anyone in the household was eligible to 
participate in the survey. Based on experiences and dif-
ficulties reported from the pilot phase, the interviewer 
training prepared for the full-scale survey included 
a sufficient number of practical how-to examples for 
dealing with various situations that might arise during 
the screening. This included showcards with maps, to 
explain the countries included in the definition of the 
target group interviewed, as well as lists of countries 
or groups to identify with. As a result, screening based 
on the country of birth and parents’ country of birth 
worked quite well.

Selecting the most qualified interviewers is very impor-
tant because it affects data quality. For that reason, 
when  surveying  dispersed  or  hard-to-reach  groups 
using a complex screening procedure, such as the one 
used in EU-MIDIS II, arrangements for the interview-
ers’ pay should be checked to make sure that they are 
compensated not only for the interview but also for 
the screening process.

Cultural and ethical training of interviewers should be 
part of every survey on ethnic minorities and other 
minority  groups.  Country-specific  stereotypes  and 
prejudices can be addressed in trainings, which helps 
interviewers  to  respect  the  guidelines  and  rules  in 
the field. The training sessions for interviewers in EU-
MIDIS II addressed cultural and ethical aspects, but 

92

Lessons learned 

field controls showed – in particular at the start of the 
survey – that additional training would have benefited 
some interviewers. Gender matching – female respond-
ents interviewed by women and male respondents by 
men – is essential in some communities, particularly 
Muslim ones. In addition, matching interviewers and 
respondents on the main language of the respondent 
should be made whenever possible. That may improve 
cooperation and willingness to participate, and increase 
the quality of the data collected.

Traditional and well-tested survey methods should be 
preferred over new methods that are not fully tested 
in the field. Alternatively the new methods should be 
piloted before implementing them on a larger scale. 
For example, random route procedures are well estab-
lished as a method of probability sampling, compared 
with random selection procedures at locations such as 
open places. However, if traditional methods are not 
feasible, as was the case in several countries for EU-
MIDIS II, alternative methods must be implemented to 
obtain the data. The implementation of such methods, 
which are theoretically well founded, can be challeng-
ing in practice – for example, if interviewers are not 
used to stopping people on the street systematically 
to ask for an interview. This can result in interview-
ers applying selection procedures differently. Well-
designed and comprehensive training of interviewers 
minimises this problem, so they can avoid mistakes and 
guarantee to collect high-quality data. Methods applied 
and experiences gained during a survey should always 
be reported transparently. That helps to improve such 
methods and will increase the quality of any future 
data collection efforts.

Surveying groups without sampling frames or without 
fully accurate sampling frames is challenging and there-
fore different assumptions – for example, on the levels 
of concentration, eligibility, or response rates –have to 
be made before starting the fieldwork. In EU-MIDIS II, 
the allocation of sample sizes and number of inter-
viewers engaged was based on several assumptions 
related to concentrations of the target population and 
the expected response rates in the Primary Sampling 
Units (PSUs). To minimise the risk to the survey in case 
the initial assumptions would be challenged, as data 
collection progresses, dropping and stopping rules were 
defined before commencing data collection. The rules 
applied in the EU-MIDIS II worked well and were applied 
only in a few cases. However, in some cases actual con-
centrations and response rates deviated considerably 
from early assumptions, which produced either insuf-
ficient number of interviews or more interviews than 
expected and planned. Therefore, implementing a stag-
gered sampling approach is recommended. That means 
releasing the sample in stages, to be able to correct the 
number of addresses over time while simultaneously 
ensuring a proper random sampling.

In EU-MIDIS II, the national survey companies often 
underestimated the response rates, mostly because 
these were based on experiences with surveys of the 
general population. Interest in the survey was high 
among the target group populations, which resulted in 
unexpectedly high participation rates in some countries. 
When the topic of the study is of direct interest to the 
respondents, as with discrimination experiences in EU-
MIDIS II, participation rates can increase considerably.

Field control for a total of 41 target groups in 28 coun-
tries  was  done  through  the  local  contractors  who 
entered  information  into  a  central  reporting  tool, 
informing on different parameters of progress of the 
fieldwork on a weekly basis, combined with monthly 
progress reports. It is important to ensure that such 
a reporting tool is completed on time and in detail by 
all country teams.

To handle the monitoring of the fieldwork in an EU-
wide comparative setting, a centralised computer sys-
tem is recommended, operating on a single platform 
for  collecting  and  recording  data  from  the  screen-
ing process and interviews. This would also help to 
minimise paper administration of the screening pro-
cess, and enable accurate and real-time monitoring 
and timely interventions.

EU-MIDIS  II  also  developed  and  applied  sampling 
methodologies  that  had  not  yet  been  employed  in 
cross-country comparative settings.

 • Adaptive Cluster Sampling (ACS) was developed to 
increase  the  efficiency  in  terms  of  screening  and 
related costs and to fully comply with the principle 
of self-identification in data collection on ethnicity. 
The plan was to use ACS in all countries where EU-
MIDIS II interviewed Roma, as well as in France51 in 
PSUs with less than 25 % concentration of the tar-
get population. Difficulties with the implementation 
of ACS were encountered in some of the countries 
where ACS was implemented. Interviews had to be 
deleted from the dataset where there were doubts 
about whether or not the ACS procedure had been 
followed correctly. The incorrect use of ACS in some 
instances  included  finishing  ACS  chains  too  early 
or continuing chains when they should have been 
brought  to  an  end.  In  Romania  and  Bulgaria,  ACS 
significantly increased efficiency of screening. De-
spite  the  challenges  related  to  the  correct  imple-
mentation of the procedure, the method is deemed 
suitable for surveying ethnic minorities. Neverthe-
less, when applying ACS it is necessary to pay close 

51  Due to legal restrictions related to data collection on 

ethnicity in France, it was preferable to use ACS in France 
as compared with FE, where neighbours are asked about 
persons’ country of birth.

93

Second European Union Minorities and Discrimination Survey – Technical report

attention to thorough training of the interviewers, 
and to find further ways of simplifying the ACS ap-
plication rules, as well as ensuring that fieldwork is 
carefully monitored.

 • Focused  enumeration  (FE)  was  clearly  easier 
to  implement  in  the  field  than  ACS,  and  could  be 
used  when  it  is  deemed  acceptable  for  persons 
to  be  asked  about  their  neighbours’  ethnicity  or 
origin.  Monitoring  of  the  fieldwork  and  screening 
seemed  easier  with  FE  than  with  ACS,  in  particu-
lar  when  address  lists  were  available  and  all  ad-
dresses could be pre-printed on the contact sheets 
that the interviewers used. However, it is important 
to  highlight  that  the  final  decision  on  respondent 
eligibility should always be left to the respondents 
themselves.

 • Location sampling proved to be a feasible alterna-
tive to the standard sampling procedures when the 
target population is highly dispersed and/or cannot 
be  reached  with  standard  survey  methodologies. 
A  prerequisite  for  this  method  is  the  assumption 
that most of the target population can be found at 
pre-selected locations, centres and areas. It is im-
portant to spend enough time to create a compre-
hensive list of all possible locations where the target 

group might congregate, as the sample will be only 
as good as the list of locations. Some locations, such 
as  counselling  offices,  churches  or  associations, 
are not public places. A further challenge involves 
convincing  the  gatekeepers  in  these  locations  to 
cooperate and allow interviewers to carry out the 
fieldwork autonomously at the location. Moreover, 
it is essential to have good estimates in advance of 
the size of the target population at each of the loca-
tions  (relative  to  each  other)  as  this  increases  the 
efficiency of the sample. Interviewing at a location, 
once it is selected and access is granted, is easier 
than with a random route approach. However, mak-
ing estimates precise, gaining access to the location 
and  applying  the  selection  rules  properly  requires 
a large amount of preparatory work, which can be 
time  intensive  and  costly.  Implementing  location 
sampling  in  EU-MIDIS  II  showed  that  standardisa-
tion of procedures across locations and countries is 
difficult,  and  that  respondent  selection  and  steps 
taken  to  ensure  access  to  the  location  have  to  be 
tailored to the specificities of locations. Further de-
velopment  is  necessary  related  to  quality  control 
of the selection procedure at locations, as it is dif-
ficult  to  check  after  fieldwork  whether  or  not  the 
selection  procedures  were  correctly  implemented 
at a location.

94

Annex 1:  Questionnaire flowchart

The full questionnaire can be accessed on FRA’s website. The chart below gives an overview of the topics covered 
by the questionnaire.

Screening

Contact sheet

Household grid

Age, education,
employment

Household grid

Housing and living
standards

Rights awareness,
perceptions
and attitudes

Discrimination

At work

Discrimination

Health

Discrimination

Education
(children)

Corruption
and experiences
with the police

Societal participation

Participation
and group relations

Discrimination

Other services

(Hate-) Crime
victimisation

Harassment
and violence

Socio-ecnomic
background

Discrimination

Looking
for work

Discrimination

Housing

Discrimination

Awareness of other
people's experiences
of discrimination

Societal participation

Residence
and status,
citizenship

Interviewer
questionnaire

Source:  FRA, EU-MIDIS II 2016

95

Annex 2:  Local contractors for fieldwork

Country

Austria

Belgium

Bulgaria

Croatia

Cyprus

Partner agency

IFES

Ipsos Belgium

Ipsos Bulgaria

Ipsos Croatia

CMR Cypronetwork

Czech Republic

Median

Denmark

Estonia

Finland

France

Germany

Greece

Hungary

Ireland*

Ireland**

Italy

Latvia

Lithuania

Luxembourg

Malta

DMA Research

Faktum & Ariko

Taloustutkimus

Ipsos France

Ipsos Germany

Ipsos Opinion

Ipsos Hungary

Ipsos MRBI

Red C

Ipsos Italy

SKDS

RAIT

TNS ILRES

Ipsos Misco

Netherlands***

PMR

Netherlands****

Labyrinth

Poland

Portugal

Romania

Slovakia

Slovenia

Spain

Sweden

Ipsos Poland

Ipsos Portugal

Ipsos Romania

Median

Ipsos Slovenia

Ipsos Spain

Ipsos Sweden

United Kingdom

Ipsos MORI

Notes: 

* Ipsos MRBI was involved until May 2016.  
** Red C was appointed in June 2015.  
*** PMR was involved until August 2015.  
**** Labyrinth was appointed in October 2015.

Source:  FRA, EU-MIDIS II 2016

96

Annex 3:  Quality assurance plan

Objective 

Quality criteria

Achievement of 
a (random) probability 
sample representative 
for the target groups 
within each country

PSUs drawn from suitable up-to-date registers and data sources where available. 

Development of sampling based on statistical grounds 
and optimising between precision, feasibility and costs, by 
collecting data for each target group in each country.

Comprehensive mapping exercise with statistical provision of distribution 
of target groups on the level of enumeration districts (if not available on 
the most detailed regional level), covering indicators for each country.

 • The ‘rarity’ of the population (for each target group) overall and on regional level 
(by reporting the size of the target group overall and in each region, and its per-
centage of the total population).

 • Geographical concentration of the target group. This should be based on data at 
the  smallest  territorial  level  available/feasible  (PSUs),  considering  all  available 
data  sources  such  as  registers  and  Census  2011  data.  This  statistical  mapping 
should cover the entire country, including PSUs with low or no density of the tar-
get population. Reporting the target population coverage which can be achieved 
at different levels of minimum concentration, at a national level.

 • In countries where centre location sampling is to be considered, further research 
to be undertaken to identify centres within selected concentrated areas (gener-
ally to be drawn from the areas included in the address sampling). This should 
not be defined too narrowly but will need to factor in what centres can be ac-
cessed by interviewers. Include the order of magnitude, with estimates if neces-
sary (to be undertaken after developing address-sampling strategies).

 • Assessment of difficulty of surveying the group and any social network aspect 

of that particular group.

 • Qualitative assessment: How reclusive is that population?

 • Establishing if telephone screening lists are available, collecting information on 

source(s) and assessing their possible use.

 • Identification of information primarily for first generation, and for second gen-
eration if available. Census 2011 includes ‘country of birth’ for all countries. In LFS 
as a proxy nationality (distinguishing three categories: country of residence, an-
other EU country and non-EU country), years of residence in the country, country 
of birth (again distinguishing only three categories) and region (NUTS2, NUTS3 
on  national  request).  Years  of  residence  in  the  country  is  a  useful  variable  for 
distinguishing ‘recent migrants’ in all the migrant groups.

Documentation of coverage per target group on the level of PSU and nationwide.

Documentation of optimised statistical cut-off point 
with regard to costs and precision.  

Use of an appropriate sample design reflecting the country 
requirements: single or two-stage stratified sampling.

Minimising and estimating design effect for sampling strategies.

Stratification of the sample by region and urbanisation level 
using the most recently available population figures.

Specification of targets for response rates on a local level, taking into consideration 
different sampling strategies and fieldwork conditions. To be reported per 
interviewer and local unit (to be specified after developing sampling strategies). 

Minimum number of interviews per target group in each country (to be 
specified after developing sampling strategies) to be completed.

Enumeration process documented on the PSU level. Two-
phase enumeration is required if no other measures can be put 
in place to quality assure random walk procedures.

Development of a reliable statistical procedure to replace unsuccessful PSUs.

97

Second European Union Minorities and Discrimination Survey – Technical report

Objective 

Quality criteria

Sampling plan is 
documented and 
reported, including 
measurement and 
systematic documentation 
of sampling errors

Submission of a sampling report containing source of population statistics, 
information on sample size (gross and net), information on the stratification of 
the population, sampling method, sampling design and description of sampling 
stages, sampling frame and its coverage and date, definition of PSU (primary 
sampling unit), number of sampling points and maximum cluster size.

Random probability 
and random sampling 
is ensured

Correction for unequal 
selection probability within 
each stage of sampling

The questionnaire is 
systematically tested prior 
to the data collection

Where required, enumeration by trained enumerators who will not 
be carrying out interviewing in the PSU they enumerate.

Enumeration checking containing the percentage of routes with full 
compliance, small errors and systematic deviation, but carried out randomly 
(e.g. random procedure always to the left instead of to the right).

At minimum 10 % of the routes must be checked against the selection rule.

 • In case of non-compliance, all routes/PSUs of the enumerator will be rechecked 
and  if  necessary  replaced.  In  that  case  it  will  be  necessary  to  enumerate  the 
route/PSU again.

Documentation of remuneration schemes for enumeration and 
for interviews to ensure incentive to deliver high quality.

Following the sampling scheme, weighting schemes are developed 
under the lead of the senior sampling and weighting expert.

All design weights are controlled for, reflecting accurately the sampling 
approach. Post-stratification shall be based on the best available 
source and controlled for between country comparability.

Where data are available, non-response adjustment 
weights are included in the weighting scheme.

All weighting steps are documented – design weights, non-response 
weights, post-stratification weights, cross-national weights, personal and 
household weights – and included as variables in the final data set and 
documented in the final technical and quality report with core parameters.

A pilot is carried out with randomly selected respondents from each 
target group over a period of two weeks in each EU Member State.

 • All participating countries are covered, with at least 20 to 30 interviews per coun-
try (depending on the number of groups to be surveyed), and including all target 
groups in each country, as far as possible.

 • CAPI  is  tested  for  comparability  between  countries  and  CAPI  systems.  CAPI 

checks are developed after the pilot.

Submission of pilot dataset and final pilot report (including country-specific 
reports). The report presents an analysis of a) adequacy and feasibility 
of the proposed sampling approaches; b) usability and functioning of the 
questionnaire, the question routing and the instrument’s technical design; c) 
equivalence of the survey instrument across languages (focus on quality of 
questionnaire translation); d) collection of meta- and paradata; e) different 
practices of interviewing. On the basis of the pilot report, revisions and 
recommendations are made before commencement of the fieldwork.

Debriefing of pilot interviewers.

98

Annex 3: Quality assurance plan

Objective 

Quality criteria

Statistics are compiled 
on the basis of common 
standards with respect 
to scope, definitions, 
units and classifications 
in the various surveys 
and sources

The translated questions 
are comparable and 
consistent over time, 
across countries and 
across populations 
(language groups)

Questions and fieldwork 
material are understood 
by all respondents (in 
the same way) and can 
be answered correctly to 
reduce non-response bias

Replication of standards from previous cross-national studies 
on education, qualifications and occupation, with a focus on 
core variables of the European Statistical System.

The comparability of Translated questions is ensured; with regard to: 
same meaning; same question format; same measurement properties 
(range of response options); the original stimulus is maintained.

Translation procedure: team based, TRAPD procedure.

Translations are validated by FRA (including the 
submission of a translatability report).

Use of glossary of terms to translate key terms in the same 
way in each country (terms to be taken partly from the existing 
FRA glossaries, InterActive Terminology for Europe).

Translation of the information letter into 10 languages (in addition to 
26 core languages) according to sampling mapping exercise.

Relevant foreign-language paper versions of the questionnaire for respondents 
having difficulties answering questions in the national language (to be 
specified based on the results of the mapping exercise for sampling).

All fieldwork documents checked for correctness (glossary, show cards, 
contact sheet, introduction letter, national briefing documents, etc.).

Accuracy of Fieldwork 
documents and 
CAPI system

CAPI testing is through technical and content checks (including 
routing of the questionnaire), via a central dummy data 
file (in particular, filters and answer options).

Experienced interviewers 
facilitate successful 
interview implementation

If the application of automated survey systems is not feasible in some countries, 
interviews are carried out using paper questionnaires upon prior approval by FRA.

Interviewers have at least 3 months’ experience working as market and 
social research interviewers, and most will have substantially more.

No interviewer to conduct more than 5 % of the total sample. If there are 
specific reasons for an interviewer to exceed this (such as language/cultural 
sensitivities, etc.), the CCT will review this before giving approval.

Recruitment of 10–15 % more than the number of interviewers needed in each 
Member State to allow for attrition and possible dismissal of interviewers.

Recruitment of a pool of interviewers that, where possible, allows:
1) matching of respondents and interviewers by specific characteristics (gender, age, 
language skills, ethnic minority/immigrant background); and 
2) using pairs of interviewers.

Information on the interviewers is to be recorded and provided in an interviewer 
register, with a unique identifier to be linked with each interview: gender, age 
and ethnicity (with consent); dates of training received; and language skills. 

99

Second European Union Minorities and Discrimination Survey – Technical report

Objective 

Quality criteria

Uniform and detailed 
information are 
provided on all aspects 
of the fieldwork (e.g. 
contact procedure, 
interviewing, data entry, 
quality control, etc.)

Data collection, data 
entry and coding are 
routinely monitored and 
revised as required

Consistency and accuracy 
in data collection 
between countries

Each national agency receives a briefing document, including a detailed 
project timetable, a list of the national agency’s responsibilities during each 
phase of the project and an outline of the lines of communication between 
the CCT and the national agencies (including contact information).

Central briefing in Berlin for all NSEs, and briefing materials 
to conduct their national interviewer briefings.

Standardised training materials are developed on the basis of the 
pre-test study and feedback from expert consultation (including 
information on sampling strategy, selection of respondent, structure of 
questionnaire, ensuring confidentiality and privacy, dealing with sensitive 
and distressed respondents, and use of fieldwork materials).

Participation by all interviewers in a two-day in-person local interviewer 
training session held by the NSE in each country, and based on the interviewer 
training manual. Any interviewers who are recruited to work on the project 
after fieldwork commences will be required to complete a separate briefing 
session with the NSE, lasting 1 day. (If PAPI mode is used, additional time 
training days focusing on PAPI interviewing need to be allocated.)

Regular quality checks of CAPI data conducted by national agencies throughout the 
fieldwork phase (data checking guidelines are provided), and by the CCT, checking for:
 • high  levels  of  item  non-response;  do  not  know/refusal  response;  conspicuous 
consistency  in  response  items;  interview  length;  outliers;  interventions  in  the 
interview; completeness and consistency in contact sheet and household grid; 
reliability of metadata; and start/end times of interviews.

CAPI data uploaded at several points (after completion of 10 %, midway 
through phase 1, end of phase 1 and 100 % of interviews) during the 
fieldwork, and quality control checks carried out by the CCT.

Interviews with more than 25 % of item non-response are not considered complete.

Interviews with missing household grid not considered complete.

Basic core variables are checked for completeness.

Incomplete, inconsistent and erroneous interviews to 
be checked back and completed or replaced.

A standardised questionnaire used in all countries (with 
the exception of items tailored to country).

Use of standardised interviewing technique: carrying out face-to-face 
interviews in each country with use of automated survey systems (CAPI).

Minimum net sample size of 25,200 respondents across the 
EU (15,200 Phase 1; 10,000 Phase 2) with optimal allocation 
of sample among target groups and countries.

 • The entire gross sample shall be drawn in Phase 1.

Coding of regional and address information so that survey data can be linked 
to external contextual/auxiliary data (e.g. NUTS3, LAU2, DEGURBA).

 • All metadata and paradata are linked to the survey data via a unique identifier.

Aim of a minimum overall response rate of 50 % within each EU Member State 
and a minimum response rate of 40 % within each PSU and each target group. 

Contact procedure is detailed in the interviewer manual and interviewer 
instructions, and addressed during central briefings and local interviewer training.

Where possible, working together with local mediators or facilitators, 
such as representatives of local communities or civil society 
organisations who can help establish cooperative relationships 
within certain groups or communities (especially with Roma).

To enable non-response analysis, information recorded on contacts and non-
contacts is collected (relevant meta-/paradata, e.g. stratification characteristics, 
survey process characteristics, results of various stages of the process).

100

Annex 3: Quality assurance plan

Objective 

Quality criteria

Each interviewer uploads their interviews at the end of each 
working day to the head office of each country.

The contact sheet information will be captured electronically 
so that data can be linked to each dataset and interviewer. This 
will be carried out via a unique identifier at data entry.

Each week, completed paper contact sheets are sent back.

Weekly updates by national agencies in the fieldwork 
phase are sent to the CCT, addressing:

 • changes in resources and personnel; number of interviews and random walks 
back checked; number of CAPI data checking rounds completed; response rate; 
basic demographic profile of respondents; outcomes of non-response and non-
contacts analysis; qualitative information collected during fieldwork).

Provision of an interim report to FRA in the middle of the fieldwork phase. The report 
is divided into information on each country and outlines fieldwork developments 
with respect to 1) progress, 2) challenges and 3) solutions implemented. (Addressing: 
a) selection and application of population sampling frames and approaches; b) 
selection of interviewers and enumerators; c) respondent screening; d) reaching 
respondents; e) interview success rate/(non-)contact and (non-)response rate.)

Final quality control report to be provided by national 
agencies at the end of the fieldwork.

Submission of field progress reports to FRA on a weekly basis. These field reports 
are set up so that progress can be monitored easily, and should include:

 • information on how the data collection and the response rates develop based on

a) target group, region and interviewer (fortnightly); 
b) completed interviews, contacts, type of non-response, eligible and non-

eligible households; and 
c) PSUs dropped or exchanged;

 • information that can be easily viewed and compiled over several time points;
 • highlighting of critical areas and developments;
 • issues that emerged in the local fields and how these were addressed;
 • general and specific measures that were taken to speed up fields that are behind;
 • number of back checks carried out, and their outcome.

Back checking interviews: 10 % of interviews in each country 
will be back checked by the fieldwork management team in 
each country throughout the duration of fieldwork.

Back checks control for initial contact (contact procedure), use of fieldwork materials, 
completed interview, household size and core dimensions of the questionnaire.

If quality is poor, all interviews of the interviewer concerned 
will be rechecked and if necessary replaced.

Guidelines on formatting rules and data checks on country level are 
provided to national agencies, including instructions to ensure that:

 • all variables are included in each dataset; the base size of each question is cor-
rect and reflects the rules of routing; there are no missing values; and the label-
ling of variables and value labels is clear and correct.

Fieldwork is documented 
and monitored 
continuously with respect 
to quality criteria

High quality of the work of 
the interviewers and the 
response data gathered 
and optimised quality 
throughout the field work

Interviewers are prevented 
from working incorrectly 
or inaccurately

Consistency and 
transparency of data

101

Second European Union Minorities and Discrimination Survey – Technical report

Objective 

Quality criteria

Each country to convert its data and upload it to a central location.

Frequency outputs for the country files to be provided to 
check merged data against the national files.

Collection and transmission of data with national coding, and 
documentation of all recoding steps to achieve the harmonised version 
(harmonisation of variables such as level of education and income).

Merging of data collected in Phase 1 and Phase 2 and submission in SPSS format.

Merged data to be checked for consistency and validity by the CCT:

 • high  levels  of  item  non-response,  do  not  know/refusal  response,  conspicuous 
consistency  in  response  items,  interview  length  outliers,  start/end  times  of 
interviews;

 • correctness of base sizes at each question and reflection of the rules of the rout-
ing in the questionnaire, and correctness against the frequency tables provided 
by  countries,  ensuring  that  there  are  no  missing  data  and  that  all  labelling  of 
variables and values is clear;

 • error rates.

The quantitative data collected from the interview and interviewer 
debriefing forms are submitted in an SPSS-compatible computer file.

A second data file is submitted containing meta- and paradata on the gross sample 
(information on sampling, address information, contact information, PSU, GPS 
coordinates, regional information, degree of urbanisation, incentives, mode).

The data file is anonymised and sensitive information on the address or 
GPS coordinates are handled in compliance with data protection rules.

Files are fully documented in a codebook (variable names and value labels) 
including metadata, paradata, interviewer data and sampling information.

Provision of a report on the calculation of selected indicators (including SPSS code).

Provision of a data editing and cleaning report.

The Code (SPSS syntax or similar) related to data file 
construction, and used to produce the quality checks, data 
coding and labelling, is submitted in electronic format.

Tasks are carried out and deliverables are submitted 
according to the agreed timetable.

All tasks are completed within 16 months from the contract 
signature unless there are agreed changes to the timetable.

Monthly updates on the progress of the project in all countries are 
provided to FRA by the project coordinator, using feedback from the 
National Survey Experts on the first working day of each month:

 • covering all elements of the quality control strategy;
 • including a record of decisions made in consultation with FRA regarding sampling 

and fieldwork in each country.

Weekly progress reports on overall project progress to be 
submitted by the project coordinator to FRA.

If quality criteria are not fully met, FRA to be informed immediately.

Organisational and quality standards are set and 
communicated (e.g. The Quality Assurance Plan).

A central management and coordination team oversees all aspects of 
work relating to the survey undertaken in all 28 EU Member States.

Consistency and 
accuracy of National 
and merged data file

Complete and anonymised 
delivery of the data 

Data editing and cleaning 
is documented with 
respect to quality criteria

Deadlines are kept as 
agreed in the inception 
meeting, in accordance 
with the agreed timetable 
and the time schedule of 
the overall deliverables 
of the project 

Regular updates to FRA 
are provided as an ‘early-
warning’ system regarding 
overall project progress

Quality standards are 
consistently applied 
by all participating 
survey organisations

102

Annex 3: Quality assurance plan

Objective 

Quality criteria

All aspects of data 
collection, survey research 
methodology and quality 
assurance are thoroughly 
documented and reported

A final technical and quality report is submitted after 
completion of the fieldwork, including:

 • information on each country describing the fieldwork with respect to:

(1) assumptions/plans at the start of fieldwork,
(2) outcomes,
(3) challenges,
(4) quality control measures, and
(5) solutions implemented concerning each of the following themes: (a) 

selection, application and assessment of population sampling designs; 
(b) geographical coverage and distribution; (c) interviewer selection and 
training; (d) ethical issues throughout fieldwork; (e) interview success/
(non-)contact and (non-) response rate; (f)interviewer feedback; (g)
weighting; (h) coherence and consistency of results;

 • highlighting of any difficulties that arose in the course of implementing the survey. 

Notes: 

CAPI, computer-assisted personal interviewing; CCT, central coordination team; DEGURBA, degree of urbanisation; LFS, 
Labour Force Survey; NSE, national survey expert; PAPI, paper and pencil interview; PSU, primary sampling unit; TRAPD, 
translation, review, adjudication, pre-testing, and documentation.

Source:  FRA, EU-MIDIS II 2016

103

References

AAPOR  (American  Association  for  Public  Opinion 
Research) (2011), Standard definitions: Final dispositiions 
of case codes and outcome rates for surveys, revised 
2011, Oakbrook Terrace, IL, AAPOR.

Baio, G., Blangiardo, G., Blangiardo, M. (2011), ‘Cen-
tre sampling technique in foreign migration surveys: 
A methodological note’, Journal of Official Statistics, Vol. 
27, No. 3, 2011, pp. 451–465.

Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, 
L., Bryant, J., Brozek, I., Hughes, C. (2014), ‘Reaching the 
hard-to-reach: A systematic review of strategies for 
improving health and medical research with socially 
disadvantaged groups’, BMC Medical Research Meth-
odology, Vol. 14, No. 42, pp. 1-29.

Council of Europe (2012), ‘Descriptive glossary of terms 
relating to Roma issues’.

Council of the European Union (2000), Council Directive 
2000/43/EC of 19 July 2000 implementing the principle 
of equal treatment between persons irrespective of 
racial or ethnic origin, OJ 2000 L 180, p. 22–26.

Council of the European Union (2010), Declaration of 
the European Ministerial Conference on Integration, 
Zaragoza, 15–16 April 2010.

Eurofound  (2010),  ‘5th  European  Sur vey  on 
Working Conditions questionnaire’.

FR A  (20 13),  Ro ma  Pi lot  Su r vey  –  tech nical 
report:  Methodology,  sampling  and  fieldwork, 
Luxembourg, Publications Office.

FRA (2014), Roma Survey – data in focus: Poverty and 
employment: The situation of Roma in 11 EU Member 
States, Luxembourg, Publications Office.

FRA  (2016),  European  Union  Minorities  and  Dis-
crimination  Survey:  Roma  –  selected  findings, 
Luxembourg, Publications Office.

FRA and United Nations (2016), Surveying hard-to-reach 
groups from a comparative cross-country perspective: 
The second European Union Minorities and Discrimi-
nation Survey (EU-MIDIS II): Note by European Union 
Agency for Fundamental Rights (FRA), Geneva, Eco-
nomic Commission for Europe.

Groves, R. M., Fowler, F. J., Couper, M., Lepkowski, J. 
M., Singer, E., Tourangeau, R. (2009), Survey meth-
odology,  Wiley  Series  in  Survey  Methodology, 
Hoboken, N.J., Wiley.

Huddleston,  T.,  Niessen,  J.  and  Tjaden,  J.  (2013), 
Using  EU  indicators  of  immigrant  integration, 
Brussels, European Commision.

OECD  (Organisation  for  Economic  Co-operation  and 
Development) (2015), Indicators of immigrant integra-
tion 2015: Settling in, Paris, OECD.

European Commission (2014), Report on the imple-
mentation  of  the  EU  Framework  for  National  Roma 
Integration Strategies, COM (2014) 209 final, Brussels, 
2 April 2014.

OHCHR (United Nations Human Rights Office of the High 
Commissioner) (2016), A human rights-based approach 
to data: Leaving no one behind in the 2030 Development 
Agenda, Geneva, OHCHR.

European Court of Auditors (2016), EU policy initiatives 
and  financial  support  for  Roma  integration:  Signifi-
cant progress made over the last decade, but addi-
tional efforts needed on the ground, Special report, 
Luxembourg,  Publications  Office  of  the  European 
Union (Publications Office).

Eurostat (2011), Indicators of immigrant integration: 
A pilot study, Luxembourg, Publications Office.

Eurostat  (2016),  ‘People  at  risk  of  pover t y 
or social exclusion’.

E u r o s t a t  
Interview Survey (EHIS)’.

( 2 0 1 7 a ) ,  

‘ E u r o p e a n   H e a l t h 

Eurostat (2017b), ‘European Union Statistics on Income 
and Living Conditions (EU-SILC)’.

FR A  (European  Union  Agency  for  Fundamen-
tal  Rights)  (2009),  EU-MIDIS:  Main  results  report, 
Luxembourg, Publications Office.

Reichel, D. and Morales, L. (2017), ‘Surveying immi-
grants without sampling frames: Evaluating the success 
of alternative field methods’, Comparative Migration 
Studies, Vol. 5, No. 1, pp. 1-22.

Schnell,  R.,  Trappmann,  M.,  Gramlich,  T.  (2014),  ‘A 
study  of  assimilation  bias  in  name-based  sampling 
of migrants’, Journal of Official Statistics, Vol. 30, No. 
2, pp. 231–249.

Survey Research Center (2016), Guidelines for best prac-
tice in cross-cultural surveys: Full guidelines, Ann Arbor, 
MI, Institute for Social Research, University of Michigan.

Verma,  V.  (2014),  Sampling  elusive  populations: 
Applications  to  studies  of  child  labour,  Geneva, 
International Labour Office.

105

The EU-MIDIS II Main results report presents the main findings for all groups surveyed� 
In addition, two Selected findings reports have been published – one focusing on Roma and one on Muslims� 

EU-MIDIS II

F
R
A

EU-MIDIS II

F
R
A

EU-MIDIS II

i

S
e
c
o
n
d
E
u
r
o
p
e
a
n
U
n
o
n
M
n
o
r
i
t
i
e
s
a
n
d
D
i
s
c
r
i

i

i

m
n
a
t
i
o
n
S
u
r
v
e
y

|

R
o
m
a
–

l

S
e
e
c
t
e
d
fi
n
d
n
g
s

i

Second European Union Minorities 
and Discrimination Survey

Roma – Selected findings

Second European Union Minorities 
and Discrimination Survey

Main results

i

S
e
c
o
n
d
E
u
r
o
p
e
a
n
U
n
o
n
M
n
o
r
i
t
i
e
s
a
n
d
D
i
s
c
r
i

i

i

m
n
a
t
i
o
n
S
u
r
v
e
y
–

M
u
s
l
i

m
s

–

l

S
e
e
c
t
e
d
fi
n
d
n
g
s

i

Second European Union Minorities 
and Discrimination Survey

Muslims – Selected findings

FRA’s online data explorer tool will allow for quick access to the full survey data, see  
http://fra.europa.eu/en/publications-and-resources/data-and-maps/.

HOW TO OBTAIN EU PUBLICATIONS

Free publications:
•  one copy: 

via EU Bookshop (http://bookshop.europa.eu);

•  more than one copy or posters/maps: 

from the European Union’s representations (http://ec.europa.eu/represent_en.htm); 
from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm); 
by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or 
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*).

(*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

Priced publications:
•  via EU Bookshop (http://bookshop.europa.eu).

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Second European Union Minorities and Discrimination Survey – Technical report

HELPING TO MAKE FUNDAMENTAL RIGHTS A REALITY FOR EVERYONE IN THE EUROPEAN UNION

FRA’s Second European Union Minorities and Discrimination Survey (EU-MIDIS II) surveys around 26,000 individuals 
from different ethnic minority and immigrant backgrounds across all 28 EU Member States. This technical report 
presents a detailed overview of the research methods used by FRA when collecting the survey data.

FRA - EUROPEAN UNION AGENCY FOR FUNDAMENTAL RIGHTS
Schwarzenbergplatz 11 – 1040 Vienna – Austria
Tel. +43 1580 30-0 – Fax +43 1580 30-699
fra.europa.eu – info@fra.europa.eu
facebook.com/fundamentalrights
linkedin.com/company/eu-fundamental-rights-agency
twitter.com/EURightsAgency