COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Report on the evaluation of the EU Framework for National Roma Integration Strategies up to 2020
Date: 2018-12-04
Institutions: Directorate-General for Justice and Consumers, European Commission
Subjects: Roma, access to education, aid to disadvantaged groups, equal treatment, ethnic discrimination, health policy, housing, marginalisation, rights of minorities, social integration

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 4.12.2018 COM(2018) 785 final  COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Report on the evaluation of the EU Framework for National Roma Integration Strategies up to 2020 {SWD(2018) 480 final}  
 
 
1. INTRODUCTION 
The  EU  framework  for  national  Roma  integration  strategies  up  to  20201  is  the  first  EU 
initiative targeting Roma that included a follow-up mechanism. Its main objectives are to 
tackle socioeconomic exclusion of and discrimination against Roma by promoting equal 
access to education, employment, health and housing. It invited Member States to design 
national Roma integration strategies (NRIS)2 and nominate national Roma contact points 
(NRCP)  to  coordinate  the  planning,  implementation  and  monitoring  of  these  strategies 
with a view to meeting EU Roma integration goals. For each Roma integration goal, the 
EU  framework  listed  a  set  of  measures  that  Member  States  could  take.  In  enlargement 
countries3  a  fifth  goal,  access  to  civic  documentation  was  added,  and  the  EU  aims  to 
improve delivery of pre-accession assistance, strengthen involvement of civil society and 
enhance monitoring. 

Box 1: EU Roma integration goals 
1. Ensure that all Roma children complete at least primary school 
2. Cut the employment gap between Roma and the rest of the population 
3. Cut the gap in health status 
4. Cut the gap in access to housing and public utilities 

An evaluation covering the 2011-2017 period was carried out in response to the Council 
Conclusions on ‘Accelerating the Process of Roma Integration.’4 This report sums up the 
findings of the open public consultation (OPC), the results of the evaluation, and the main 
lessons learnt from the evaluation. The accompanying staff working document annexed to 
this Report contains the full evaluation.   

2. PUBLIC CONSULTATION 
The consultation enquired about the causes of exclusion, ways and actors to tackle them, 
and  the  priorities  for  action.  It  collected  opinions  on  achievements,  challenges  and 
progress  regarding  both  policy  development  and  the  situation  of  Roma  in  education, 
employment,  health,  housing  and  regarding  discrimination/antigypsyism5.  The  findings 
cover  perception  of  stakeholders  such  as  National  Roma  Contact  points,  civil  society 
organizations and individuals from Member States and enlargement countries. 

An  overwhelming  majority  of  the  240  respondents  think  that  the  situation  of  Roma  is 
worse than that of non-Roma in  the five  fields,  particularly regarding  discrimination,  in 
employment  and  housing.  Most  also  think  that  public  interventions  are  needed  both  by 
EU institutions and national authorities in all these areas to improve the situation. 

In terms of change since 2011 most people see an improvement in education. While many 
see improvement in health, opinions regarding employment are divided. Regarding both 
housing and discrimination those who believe the situation is worsening outnumber those 
who see improvement.  

1 COM(2011) 173 final 
2 NRIS also stands for integrated sets of policy measures. 
3 Enlargement countries include Albania, Bosnia and Herzegovina, former Yugoslav Republic of Macedonia, 
Kosovo*, Montenegro, Serbia and Turkey. 
* This designation is without prejudice to positions on status, and is in line with UNSCR 1244/1999 and the ICJ 
Opinion on the Kosovo declaration of independence. 
4 EPSCO, 8/12/2016. 
5 https://ec.europa.eu/info/consultations/public-consultation-evaluation-eu-framework-national-roma-integration-
strategies-2020_en.  

1 

 
 
                                                           
Chart 1: Perceived changes in the situation of Roma since 2011 

OPC 

Regarding  policy  development  at  EU  level  most  progress  is  seen  in  education  and  in 
fighting  discrimination.  Also  for  health  and  employment  more  people  see  improvement 
rather than deterioration. Regarding housing negative opinions outnumber positive ones. 
On  policy  development  at  the  national  level  the  balance  tilts  towards  the  positive  for 
education  and  healthcare,  and  to  the  negative  for  employment,  discrimination  and 
especially housing. 

Replies suggest that successful Roma integration strategies at both European and national 
level need to be comprehensive, covering at least the key fields of the EU framework  as 
well  as  fighting  antigypsyism.  From  a  list  of  20  possible  priority  areas6  education 
emerged  as  a  clear  priority  (67 %  at  European  and  76 %  at  national  level),  followed  by 
employment (49 % at European and 57 % at national level), while more than one third of 
respondents at both European and national levels selected the fight against discrimination 
and antigypsyism, as well as access to housing and healthcare as priorities. 

The  need  for  a  comprehensive  approach  is  also  suggested  by  the  finding  that  there  are 
multiple  causes  of  exclusion  that  integration  strategies  must  address  simultaneously. 
These  include  primarily  discrimination  and  antigypsyism,  limited  political  commitment, 
lack of Roma participation, limited capacities of institutions and insufficient funding.  

According  to  60 %  of  the  respondents  national,  regional  and  local  authorities  need  EU-
support to improve the situation of Roma. Stakeholders see a stronger role for the EU than 
for  national  authorities  in  monitoring  and  enforcing  European  non-discrimination  and 
anti-racism legislation and in making the development and implementation of ambitious 
Roma  inclusion  policies  a  condition  for  access  to  funding.  National  authorities  are 
expected  to  play  a  bigger  role  in  measures  fighting  antigypsyism  (such  as  community 
building between Roma and non-Roma, non-discrimination and Roma inclusion training 
for public officials, or making Roma history and culture part of school curricula) and in 
increasing  Roma  participation  (political  participation,  national  fora  of  cooperation, 
capacity building, involvement in policy development and monitoring). 

6  Access  to  education,  employment,  health  care,  housing  and  essential  services,  documentation;  promoting  Roma 
culture  and  arts,  teaching  of  Romani  language,  teaching  of  Roma  history;  fighting  discrimination,  addressing 
antigypsyism, Roma empowerment, specific measures for Roma children,  women, youth; poverty reduction; capacity 
building  for  national,  regional  and  local  authorities,  for  civil  society;  coordination  and  cooperation  of  stakeholders; 
funding; monitoring and evaluation. 

2 

 
 
 
                                                           
Achievements  (putting  Roma  inclusion  high  on  the  policy  agenda,  EU  funding, 
recognition of antigypsyism as a specific form of intolerance) are linked to the EU, rather 
than the national level. Regarding challenges to the framework and NRIS, the overarching 
challenge highlighted by most stakeholders participating at the public consultations is the 
lack  of  effective  mainstreaming  (i.e.  sensitivity  of  mainstream  public  policies  to  Roma 
needs). 

3. RESULTS BY EVALUATION CRITERION 

The  evaluation  assessed  five  basic  criteria  (relevance,  coherence,  effectiveness, 
efficiency,  and  the  EU  added  value)  and  three  additional  ones  (equity,  coordination, 
sustainability). The additional criteria are reviewed below linked to  the basic evaluation 
criteria  to  which  they  are  most  closely  connected  (equity  under  relevance,  coordination 
under effectiveness, and sustainability under EU added value). 

3.1. Relevance 

The  analysis  of  relevance  looks  at  whether  the  objectives  set  were  appropriate  to  the 
needs  at  the  time  and  continue  to  be  relevant  today.  Relevance  is  assessed  as  overall 
positive  with  limitations.  The  evaluation  confirmed  the  priority  areas  as  key  to  Roma 
inclusion and they continue to be valid today.  

The evaluation also identified shortcomings in the initial design of the framework: 

  While relevant, the goal in the area of education (that all Roma  children complete at 

least primary education) was not ambitious. 

  Fighting  discrimination  was  a  general  objective,  a  cross-cutting  theme  without  a 
specific  non-discrimination  goal.  Action  targeting  antigypsyism  was  found  to  be 
insufficient,  although  the  framework  identified  measures  tackling  discrimination  in 
each policy area and subsequent EU policy documents recognised antigypsyism as a 
specific  form  of  intolerance,7  as  well  as  steps  at  EU  and  national  level  to  fight 
discrimination, racism and xenophobia in line with EU legislation8. 

  While the framework highlighted diversity under the broad umbrella term of Roma9, it 
proved to have limited capacity to deal with diversity within the Roma population. It 
did not pay sufficient attention to targeting specific groups among Roma, addressing 
the gender dimension or multiple discrimination (equity). 

To  respect  Member  State  competences  the  EU  framework  provides  flexibility  for  the 
adaptation  of  its  objectives  and  selection  of  target  groups  to  specific  national  contexts. 
This  flexibility 
the 
appropriateness and ambition of the objectives and measures set out in NRIS. While such 
an approach has the potential to strengthen the relevance of actions, the evaluation found 
that overall it contributed to fragmented implementation reducing effectiveness. 

the  framework’s  relevance  hinges  closely  on 

implies 

that 

7  The  Council  of  Europe  (ECRI)  defines  antigypsyism  as  a  ‘specific  form  of  racism,  an  ideology  founded  on  racial 
superiority, a form of dehumanisation and institutional racism nurtured by historical discrimination, which is expressed, 
among  others,  by  violence,  hate  speech,  exploitation,  stigmatisation  and  the  most  blatant  kind  of  discrimination.’  
COM(2017) 458 final, 2008/913/JHA. 
8  Council  Directive  2000/43/EC  of  29  June  2000  implementing  the  principle  of  equal  treatment  between  persons 
irrespective  of  racial  or  ethnic  origin  and  Council  Framework  Decision  2008/913/JHA  of  28  November  2008  on 
combating certain forms and expressions of racism and xenophobia by means of criminal law. 
9 The term ‘Roma’ is used in line with the terminology of European institutions and international organisations to refer 
to  a  number  of  different  groups  (such  as  Roma,  Sinti,  Kale,  Gypsies,  Romanichels,  Boyash,  Ashkali,  Egyptians, 
Yenish, Dom, Lom, Rom, Abdal) and also includes Travellers, without denying these groups’ special characteristics. 

3 

 
 
                                                           
3.2. Effectiveness 

The  analysis  of  effectiveness  considers  the  extent  to  which  the  framework  has  made 
progress  so  far  towards  its  objectives,  both  in  terms  of  Roma  integration  goals  and  the 
setting  up  of  instruments  and  structures  for  Roma  integration  at  EU  and  national  level 
(coordination). 

Effectiveness  in  progress  towards  Roma  integration  goals  is  assessed  as  overall 
limited with significant differences across areas and countries10. Education is found as the 
area with most progress (improvements in early school-leaving, early childhood education 
and  compulsory  schooling,  but  deterioration  in  segregation).  The  self-perceived  health 
status  of  Roma  has  improved  but  medical  coverage  continues  to  be  limited.  No 
improvement has been observed in access to employment, and the share of young Roma 
not  in  employment,  education  or  training  (NEET)  has  even  increased.  The  housing 
situation  remains  difficult.  Some  progress  has  been  observed  in  respect  to  the  general 
objective  of  fighting  poverty.  Antigypsyism  and  hate  crime  continue  to  be  a  matter  of 
high concern, despite evidence of some reduction in discrimination experiences of Roma 
when accessing services in some areas. 

Chart 2: 2011-2016 changes in the situation of Roma in the policy areas, poverty; Discrimination 
by area and antigypsyism 

FRA 2011, 2016, 2017 

10 Roma integration indicators scoreboard, SWD (2017) 286 final/2 accompanying COM (2017) 458 final. 

4 

 
 
 
 
 
                                                           
Effectiveness in terms of coordination is assessed as positive at EU level and limited 
at  national  level.  According  to  the  evaluation  findings,  the  framework  succeeded  in 
setting  up  or  strengthening  instruments  and  governance  structures  and  building 
cooperation  among  and  capacity  of  actors  working  on  Roma  inclusion.  The  framework 
further improved stakeholder cooperation at EU and national level, but participation does 
not  yet  sufficiently  impact  policy-making  and  service  design.  At  national  level,  the 
framework  led  to  the  development  of  instruments  –  national  Roma  strategies,  Contact 
Points (NRCP) and platforms – but the alignment of NRIS with Roma integration goals is 
incomplete.  According  to  the  evaluation  findings,  although  the  establishment  of  NRCP 
has led to improvements in cooperation among public administration entities and between 
public administration and other stakeholders, several NRCP still lack institutional weight, 
resources and capacity. 

3.3. Coherence 

The  analysis  of  coherence  examines  how  well  the  EU  framework  works  together  with 
other EU and national instruments. Coherence has been assessed as positive at EU level 
and limited at national level. 

The  evaluation  shows  that  at  EU  level  progress  has  been  made  in  mobilising  policy, 
funding  and  legal  instruments  for  Roma  inclusion.  This  is  particularly  the  case  for  the 
Europe  2020  strategy  (through  synergies  between  Europe  2020  targets  and  Roma 
integration  goals  and  country-specific  recommendations  on  Roma)  and  European 
Structural  and  Investment  Funds  (ESIF  2014-2020:  Roma-specific  investment  priority11 
and ex ante conditionality). Moreover, a link between Europe 2020 and ESIF was created: 
Member States that received country-specific recommendations on Roma were to include 
the Roma-specific investment priority and use ESIF for inclusive policy  reform. This in 
turn triggered the applicability of the Roma-related thematic ex-ante conditionality and its 
fulfilment criteria, closely linked to the framework’s objectives.  

The  evaluation  suggests  that  the  framework  also  succeeded  in  mainstreaming  Roma 
inclusion  goals  into  areas  such  as  enlargement12,  Urban  Agenda13,  EU  anti-trafficking 
policies14  or  proposals  such  as  the  recast  Drinking  Water  Directive15.  However,  the 
evaluation  notes  that  Roma  are  not  explicitly  referred  to  in  the  Youth  Guarantee  or  the 
European Pillar of Social Rights. The EU framework and EU legal instruments are highly 
complementary  in  terms  of  aims  and  scope.  The  enforcement  of  EU  legislation  by  the 
Commission  strengthens  the  framework’s  policy  objectives,  while  the  monitoring  NRIS 
implementation informs the Commission on the state of play of the EU legislation. 

The evaluation found that the framework contributed to mainstreaming Roma integration 
into legal, policy and financial instruments at national level to a more limited extent. Most 
NRIS  do  not  connect  well  with  public  policies.  A  major  obstacle  is  NRCPs'  limited 

11 For the 2014-2020 period  €1.5 bio were  made  available to 12 Member States under ESF investment priority 9(ii) 
‘Integration of marginalised communities such as the Roma’.  
12  Roma  integration  was  fully  mainstreamed  into  enlargement  negotiations.  It  was  included  in  enlargement  key 
priorities, visa liberalisation road map, negotiations on chapter 23 "Judiciary and Fundamental rights", IPA assistance. 
Progress was assessed  in annual Commission reports. 
13Eurocities2017:Roma 
http://nws.eurocities.eu/MediaShell/media/Mapping_of_the_situation_of_Roma_in_cities_FINAL_REPORT.pdf . 
14 See deliverables under the 2012-2016 EU strategy on eradicating trafficking in human beings, where the situation of 
Roma  and  their  particular  vulnerability  and  risk  of  being  trafficked  are  referred  to,  This  is  in  line  with  the 
comprehensive  EU  anti-trafficking  legal  (Directive  2011/36/EU)  and  related  EU  policy  framework,  including  the 
Commission  Communication  reporting on  the  follow  up to  the  EU  Strategy  towards  the  eradication of  trafficking  in 
human beings and identifying further concrete actions (COM(2017)728final). 
15 COM(2017) 753 final. 

Brussels. 

Europe. 

Cities 

in 

in 

5 

 
 
                                                           
 
influence  on  the  design,  implementation  and  decision-making  processes  of  mainstream 
policies.  

3.4. Efficiency 

The  analysis  of  efficiency  assesses  the  relationship  between  the  resources  used  in  an 
intervention  and  the  changes  generated.  The  evaluation  focused  on  the  efficiency  of 
monitoring and reporting and attempted to assess the costs and benefits in the context of 
the EU framework and NRIS.  

On  monitoring  and  reporting  systems  efficiency  has  been  assessed  as  limited. 
Monitoring  instruments  developed  gradually.  The  2013  Council  Recommendation 
introduced  comprehensive  reporting  by  Member  States,  albeit  only  from  2016.16  This 
reporting,  in  line  with  Member  States’  commitments  primarily  focuses  on  integration 
measures,  rather  than  outcome  and  result  indicators  that  would  allow  the  overall 
effectiveness  of  the  interventions  to  be  measured.  The  evaluation  finds  that  the  online 
reporting  tool  provided  by  the  Commission  for  reporting  from  the  national  to  the 
European  level  is  a  positive  step  towards  policy  learning  and  transparency,  but  still  has 
weaknesses to be addressed (See Section 4.5.). Independent monitoring has been ensured 
through  FRA  and  IPA-funded  surveys  making  possible  monitoring  progress  over  time. 
Since 2017 the Commission has piloted a project to monitor the implementation of NRIS 
by coordinated civil society coalitions17. 

The  evaluation  states  that  costs  and  benefits  could  not  be  conclusively  quantified, 
evaluated and attributed to the framework. While costs are short-term, the majority of 
the potential benefits  are long-term  and  cannot  be realised until  2020.  In the long term, 
progress  made  in  education,  employment,  housing  and  health  would  not  only  reduce 
poverty  among  Roma,  but  also  bring  fiscal  benefits  such  as  contributions  to  the 
government  budget  or  impact  on  the  use  of  public  goods  and  services  (reduction  in  the 
take-up  of  social  benefits).  The  integration  of  Roma  could  positively  impact  the  labour 
market  (improved  efficiency  through  diminished  labour  and  skill  shortages)  and  the 
economy, including GDP. 

3.5. EU added value 

The analysis of EU added value looks for changes triggered by the framework over  and 
above  what  could  reasonably  have  been  expected  from  stakeholders  alone  or  from  no 
action  at  all.  EU  added  value  has  been  assessed  as  positive.  The  evaluation  observes 
that  the  EU  action  has  provided  added  value  to  national  Roma  policies  and  their 
implementation through political, governance and financial dimensions. 

Box 2: EU added value 
Political 
  Roma inclusion high on the EU and national agendas 
  Attention to Roma issues even in countries with smaller Roma populations 
Governance 
  Development of structures for Roma inclusion 
  Stability thanks to the multiannual character of EU action 
  Policy guidance, monitoring and reporting framework for accountability and transparency 
  Opportunities for mutual exchange and cooperation 
  Comprehensive approach  
Financial 

16 2017 in the enlargement region 
17 https://cps.ceu.edu/roma-civil-monitor A similar process is also funded by IPA. 

6 

 
 
                                                           
  ESIF and IPA support to implement NRIS 

The  evaluation  finds  that  without  the  EU  framework,  Roma  issues  would  receive  less 
attention in the EU policy agenda. In some countries NRIS may cease to exist; in others 
they  may  be  weakened  further  reducing  political  commitment  to  Roma  inclusion.  The 
ending of the framework would result in less and looser monitoring and reporting. Some 
current  national  policies  and  targeted  structures  would  stop  or  become  less  operational 
and  more  symbolic.  More  time  is  needed  to  consolidate  working  structures,  ensure 
sustainability  and  the  long-term  impact  of  policies,  according  to  the  evaluation.  It  is 
essential that NRIS guided by a common framework are continued and improved. While 
the  initial  phase  has  achieved  some  tangible,  albeit  insufficient  results,  the  evaluation 
submits  that  the  overall  process  be  strengthened  and  become  better  focused,  with  an 
emphasis  on  improved  political  commitment,  the  introduction  of  specific  measurable 
targets  and  rigorous  monitoring,  and  more  effective  implementation  supported  by 
sufficient funding and participatory governance systems. 

4. EVALUATION: CONCLUSIONS AND LESSONS LEARNT 

The evaluation showed that the framework has been key for the development of and EU 
and  national  instruments  and  structures  aiming  to  promote  Roma  inclusion,  but  the 
ambition of "putting an end to the exclusion of Roma" has not been achieved. Both, the 
evaluation findings and the earlier annual assessmentsi carried by the Commission on the 
implementation of the NRIS point to the key priorities to be addressed; these include the 
need  for  better  mainstreaming,  clear  focus  on  fighting  antigypsyism,  improving 
partnership  and  Roma  participation,  addressing  diversity  among  Roma  (with  focus  on 
Roma women, youth and children) and better target setting, data collection and reporting 
to promote policy learning.18  

4. 1. Combining effective mainstreaming and Roma targeting to ensure sustainability 

According  to  the  findings  of  the  evaluation,  the  EU  framework  and  NRIS  should  be 
mobilised  more  effectively  to  ensure  that  Roma  needs  are  reflected  in  mainstream 
policies.  Inclusion  of  Roma  happens  when  mainstream  policies  are  responsive  to  their 
specific  needs.  Most  current  policies  aimed  at  Roma  inclusion  lack  a  systemic 
perspective. The evaluation finds that national authorities should follow a twin strategy 
of  making  mainstream  services  inclusive  and  providing  targeted  programmes 
towards  the  most  vulnerable.  NRIS  should  detail  how  to  further  include  Roma  in 
education,  employment,  health  and  housing  policies  and  what  explicit  measures  will  be 
developed  to  overcome  specific  disadvantages  and  ensure  effective  equal  access  in  the 
different areas. To this end Member States should make full use of the investment priority 
on  integrating  marginalised  communities  such  as  Roma  and  other  relevant  investment 
priorities.  

18 See Annex for a table  with a summary of assessment by evaluation criterion. 

7 

 
 
 
 
                                                           
In relation to this point, for the post-2020 ESIF regulations the Commission proposed to 
strengthen  the  link  between  NRIS  and  EU  funding,  maintaining  a  specific  objective  on 
socio-economic  integration  of  marginalised  communities  such  as  Roma  and  allowing 
funds  to  be  used  only  if  they  are  linked  to  strengthened  thematic  enabling  conditions 
which include requirements for NRIS19: 

taking into account the gender dimension and situation of young Roma; 

  measures to accelerate Roma integration, and prevent and eliminate segregation; 
 
  setting baselines, measurable milestones and targets; 
  arrangements for monitoring, evaluation and review; 
  arrangements for mainstreaming Roma inclusion at regional and local levels; 
  arrangements for ensuring that the design, implementation, monitoring and review of 
NRIS  are  conducted  in  close  cooperation  with  Roma  civil  society  and  all  other 
relevant stakeholders, including those at the regional and local levels. 

Roma  inclusion  is  also  among  the  thematic  priorities  for  future  assistance  to  the 
enlargement region under IPA III.. 

Also  at  EU  level,  the  evaluation  suggests  that  consistency  between  the  framework  and 
other policies should be strengthened. Linking policy priorities (both promoting inclusive 
structural reform under the Semester and enlargement policy, and targeted policies under 
NRIS) to EU funding should be stepped up. The Commission could for example explore 
further  possibilities  to  condition  funding  on  the  fight  against  discrimination  and 
antigypsyism, and fully exploit synergies between Roma integration goals and of key 
EU and international policy efforts, such as the European Pillar of Social Rights and the 
UN 2030 Agenda on Sustainable Development is another mechanism to be considered to 
be used to the full. 

4.2. Increased focus on fighting discrimination and antigypsyism 

The  evaluation  found  that  the  lack  of  a  specific  non-discrimination  goal  and  targeted 
strategies  and  action  to  fight  antigypsyism  were  among  the  key  weaknesses.  While 
discrimination  and  social  exclusion  reinforce  each  other,  any  Roma  can  be  exposed  to 
discrimination, but not all are socially excluded. 

Some efforts to rectify this weakness have already been made: Antigypsyism is monitored 
under the Racial Equality Directive20 and the Council Framework Decision on combating 
racism  and  xenophobia21;  monitoring  hate  speech  online  covers  antigypsyism  and  has 
involved  Roma  NGOs22  ;  the  fight  against  discrimination  and  antigypsyism  were 
prioritised  in  calls  under  the  Rights,  Citizenship  and  Equality  programme,  under  the 
Roma  Civil  Monitor  project,  in  stakeholder  discussions,  such  as  the  European  Platform 
for Roma Inclusion or the High Level Group on combatting racism and xenophobia23 and 
in the FRA’s research work24. 

19  https://ec.europa.eu/commission/publications/investing-people_en  for  Commission  ESF+  proposal  of  30/05/  2018; 
https://ec.europa.eu/commission/publications/regional-development-and-cohesion_en for Commission CPR proposal of 
29/05/ 2018. 
20 Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment 
between persons irrespective of racial or ethnic origin, OJ L 180, 19.7.2000, p. 22–26. 
21 Council Framework Decision 2008/913/JHA of 28 November 2008 on combating certain forms and 
expressions of racism and xenophobia by means of criminal law; OJ L 328, 6.12.2008, p. 55–58. 
22 http://ec.europa.eu/newsroom/just/item-detail.cfm?item_id=54300  
23 http://ec.europa.eu/newsroom/just/item-detail.cfm?&item_id=51025  
24 FRA 2018, A persisting concern: anti-Gypsyism as a barrier to Roma inclusion. 

8 

 
 
                                                           
The evaluation suggests that Member States should be encouraged to implement more 
measures  tackling  discrimination  and  antigypsyism,  e.g.  according  to  the  evaluation, 
NRIS  should  include  targeted  measures  aimed  at  preventing  and  countering  bias 
motivated  hate  crime,  hate  speech  and  stigmatisation  caused  by  antigypsyism  (such  as 
updating  education  curricula,  inter-ethnic  community  building,  trainings  sensitising 
employers, educational, health and housing authorities, as well as police, prosecutors and 
judges).  Systematic  efforts  are  needed 
to  prevent  EU  funds  from  supporting 
discriminatory  practices.  Mutual  learning  and  cooperation  between  public  authorities, 
civil  society,  the  media  and  equality  bodies  should  cover  monitoring  and  fighting 
antigypsyism.  NRCP  should  involve  equality  bodies  when  drawing  up  action  plans  on 
fighting antigypsyism, racism and discrimination under NRIS. 

The  evaluation  puts  forward  that  fighting  discrimination  and  antigypsyism  should  be  a 
separate priority area of the framework with a specific non-discrimination goal alongside 
the four Roma integration goals. At the same time, it should also remain a cross-cutting 
priority  with  specific  objectives  in  each  of  the  four  policy  areas.  A  clearer  focus  on 
fighting  antigypsyism  and  discrimination  should  complement,  not  replace  the 
inclusion  approach.  This  could  both  strengthen  the  enforcement  of  anti-discrimination 
and hate crime legislation and increase effectiveness of social inclusion policies. 

4.3. Partnership 
The  evaluation  found  that  governance  mechanisms  are  in  place  at  the  EU  and  national 
level,  but  their  function  is  still  limited.  In  general,  NRIS  do  not  connect  effectively  to 
mainstream policies, as they do not involve key sectors and  stakeholders in a consistent 
way. Civil society participation at all stages of European multilevel governance has been 
fostered. However Roma still have only limited opportunities to participate effectively in 
political  life  and  in  all  stages  of  the  policy  process.  Strengthening  the  framework  and 
NRIS  requires  empowerment  and  capacity  building  of  Roma,  civil  society,  national 
authorities, in particular NRCP and local governments. 

To  ensure  Roma  empowerment  and  participation  dedicated  measures  needs  to  be 
encouraged to strengthen the Roma voice in the policy process. These include:  

  political participation through community action; 
  promoting Roma participation in professions in which they are underrepresented; 
  promoting good practices for cooperation between Roma and non-Roma;  
  supporting Roma engagement with local authorities and civil society.  
Gender equality needs to be taken into account when promoting more effective Roma 
engagement at all levels. 
The  evaluation  shows  that  empowerment  and  capacity  building  of  Roma  and  pro-
Roma civil  society is  crucial  not  only in  providing social  services to  Roma, but  also  in 
policy  development  and  as  independent  watchdogs.  Continued  and  coordinated  civil 
involvement in  monitoring implementation  should be ensured.  Their involvement  in  the 
designing, implementing and monitoring mainstream policies should also be supported at 
the  national  level,  to  ensure  that  policies  reach  out  to  Roma.  To  this  end  synergies  and 
cooperation between Roma and mainstream NGOs needs to be supported. 

NRCP  should  be  strengthened  in  terms  of  mandate,  institutional  capacity,  human 
resources and budget. They have to become efficient advocates of mainstreaming Roma 
inclusion into all relevant policy areas and engage key departments at national level. To 
this end they should: 
  ensure a permanent dialogue with all departments relevant for Roma inclusion; 

9 

 
 
  work in close alignment with managing authorities; 
  work with regional and local actors; 
 

involve  all  relevant  actors  in  overseeing  NRIS  and  relevant  policies  for  Roma 
inclusion and anti-discrimination; 

  strengthen  the  system  of  consultation  and  dialogue  with  Roma  civil  society  and 
their involvement in monitoring, evaluating and reporting on implementation. 

Local governments often do not possess sufficient resources and expertise when it comes 
to fighting discrimination and promoting social inclusion. EU funds should be used to a 
larger extent to build local capacities and promote the development, implementation and 
monitoring of local strategies. Engaging Roma communities in initiatives targeting them 
and  building  their  capacity  to  take  the  leadership  in  such  projects  improves  the 
effectiveness  and  efficiency  of  EU  funds  for  Roma  inclusion  and  contributes  to 
empowering the people usually referred to as “target group”.25 

4.4. Addressing diversity among Roma 

The evaluation found that the EU framework had limited capacity to deal with diversity 
among Roma. The gender dimension was found to be weak, with only some NRIS taking 
into  account  the  specific  vulnerabilities  of  women.  A  child-sensitive  approach  would 
have required more comprehensive strategies tackling children’s needs simultaneously in 
education, health and housing policies as well as in child protection. Fighting trafficking 
in human beings with child sensitivity and addressing the gender dimension is still a need. 
Multiple and intersectional discrimination is rarely addressed.  

Although a key trigger for the launch of the EU  framework has been intra-EU mobility 
after enlargement, the framework and most NRIS lack consideration of the needs of EU-
mobile Roma, non-EU nationals or stateless Roma. Addressing these needs would require 
explicit measures and human rights-based practices. 

The evaluation shows that NRIS should better reflect the needs of specific groups and the 
diversity of conditions across the Roma population. NRIS should include explicit targets 
and indicators to tackle the specific needs of vulnerable groups in each of the key areas. 
The  target  group  among  Roma  of  key  inclusion  measures  should  be  specified  in  NRIS 
while taking into account the Citizens’ Rights Directive.26 

4.5. Target setting, data collection, monitoring and reporting to enable policy learning 

According  to  the  evaluation,  complementing  the  four  priority  areas  with  a  focus  on 
fighting  discrimination  and  antigypsyism  as  both  a  cross-cutting  requirement  and  a 
separate  area  and  specifying  a  portfolio  of  individually  adaptable  Roma  integration 
objectives with accompanying targets and indicators would improve the framework’s 
effectiveness and relevance. Country-specific targets could be selected from a detailed list 
of  optional  targets  and  indicators  per  area  (differentiated  targets  in  a  common 
framework). Governments should update their objectives and prioritise according to their 
national  circumstances  on  the  basis  of  data.  Having  a  flexible  yet  standardised  set  of 
objectives  and  targets  can  help  to  take  better  account  of  the  diversity  in  the  Roma 
population. A flexible indicator framework would allow NRIS to adapt the shared set of 
targets to national priorities and improve the quality of NRIS in several ways: 

25 The ROMACT programme and FRA’s Local engagement in Roma inclusion (LERI) project are examples 
of  how  participatory  action  methods  can  facilitate  local  partnerships  and  build  cohesion  at  local  level. 
http://coe-romact.org/; 
http://fra.europa.eu/en/project/2015/local-engagement-roma-inclusion-leri-multi-
annual-roma-programme   
26  2004/38/EC 

10 

 
 
                                                           
  The  objectives  of  NRIS  need  to  be  specified  and  disaggregated,  including 

nationally adapted targets for sub-sectors. 

  The objectives should identify specific, quantified and time-bound targets in each 

area detailed according to country situations. 

  Targets  should  be  set  in  absolute  terms  in  order  to  reflect  improvement  in  the 
situation of Roma, as well as relative terms in relation to the general population in 
order to reduce gaps. 

  Gender inequalities and group specific needs should be present in the targets. 
  NRIS  should  define  reporting  on  its  respective  targets  and  pay  attention  to 

interconnections across targets in key areas. 

The  evaluation  shows  that  data  collection,  monitoring  and  reporting  systems  should 
be  strengthened.  Lack  of  reliable  data  disaggregated  by  ethnicity  and  lack  of 
transparency and accountability mechanisms are key challenges that make the process of 
monitoring difficult and unreliable. This makes it difficult to measure progress. The NRIS 
monitoring  and  reporting  system  should  provide  information  at  policy  and  programme 
level,  information  on  explicit  targeted  actions  and  actions  to  promote  more  inclusive 
mainstream  policies.  Data  availability  and  disaggregation  should  be  improved  to  ensure 
appropriate  data  exists  to  monitor  implementation,  achievements  and  efficiency.  Data 
collection,  reporting  and  analysis  to  track  progress  should  be  developed  using  FRA's 
support  and  technical  assistance  to  national  authorities27  in  order  to  facilitate  evidence-
based  policy  learning.  Qualitative  data  should  be  used  to  contextualise  quantitative 
indicators and better understand the specific factors contributing to the success or failure 
of individual interventions. 

Improved  national  reporting  coordinated  by  NRCP  and  involvement  of  civil  society  in 
monitoring can contribute to the systematisation of knowledge and the documentation of 
existing experiences, which can strengthen mutual learning and policy transfer.  

4.6. Increase ambition on progress during the pre-accession phase 

The  evaluation  included  also  pre-accession  countries  and  the  findings  shows  the 
importance  of  ambitious  objectives  on  Roma  integration  in  the  pre-accession  phase. 
Therefore,  promoting  equal  access  to  education,  employment,  health  and  housing  has 
been included  as  part of the enlargement process. All  enlargement  countries adopted an 
NRIS, nominated NRCPs and implemented an annual reporting system similar to Member 
States. Even if progress  has been achieved, much remains to be done on the challenges 
identified above. 

5. Conclusions  

Roma  exclusion  and  discrimination  has  existed  for  centuries.  Overcoming  it  requires 
long-term commitments and a comprehensive approach. Structural changes need time and 
real impact may not be seen for at least a generation.  
The evaluation of the EU framework for national Roma integration strategies up to 2020 
has  shown  that  the  EU  framework  is  the  beginning  of  a  process  that,  despite  many 
limitations and taking into account the massive task involved, has shown positive results 
and  an  initial  change  in  trends.  However  the  evaluation  also  suggests  that  further  steps 
could be taken to come closer to achieve the objectives. . 

27 

applies also to candidates countries with observer status 

11 

 
 
 
                                                           
Annex 1 

Table 1 Summary of assessment by evaluation criterion 

i COM/2017/0458 final, COM/2016/0424 final, COM/2015/0299 final, COM/2014/0209 final, 
COM/2013/0454 final 

12