COMMISSION STAFF WORKING DOCUMENT […] Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex-post evaluation of the 2019 European Capitals of Culture (Plovdid and Matera)
Date: 2021-08-27
Institutions: Directorate-General for Education, Youth, Sport and Culture, European Commission
Subjects: Basilicata, Bulgaria, EU action, European cultural event, European symbol, Italy, Plovdiv region, capital city

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 27.8.2021  SWD(2021) 232 final  COMMISSION STAFF WORKING DOCUMENT […] Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions  Ex-post evaluation of the 2019 European Capitals of Culture (Plovdid and Matera)  {COM(2021) 493 final}  
 
 
INTRODUCTION .......................................................................................................................................... 3 

1. 

2. 

BACKGROUND TO THE INTERVENTION ..................................................................................... 4 

IMPLEMENTATION / STATE OF PLAY .......................................................................................... 7 

3.  METHOD ............................................................................................................................................ 11 

4.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS ........................................... 13 

5. 

CONCLUSIONS ................................................................................................................................. 23 

ANNEX 1: PROCEDURAL INFORMATION ............................................................................................ 25 

ANNEX 2: STAKEHOLDER CONSULTATION ....................................................................................... 27 

ANNEX 3: METHODS AND ANALYTICAL MODELS ........................................................................... 29 

2 

 
 
 
 
 
 
 
INTRODUCTION 

This  document  describes  the  methodology  and  findings  of  the  ex-post  evaluation  of  the 
European Capital of Culture (ECOC) Action for 2019.  
Article  12  of  Decision  No  1622/2006/EC1  (here  after,  the  “Decision”)  requires  that  the 
Commission ensures the external and independent evaluation of the results of the ECOC from 
the  previous  year.  The  aim  of  the  evaluation  is  to  better  understand  how  the  two  European 
Capitals of Culture of the previous year contributed to the objectives of the Action, whether 
they  achieved  their  objectives  and  whether  implementation  proceeded  in  line  with  their 
original  application.  It  is  also  to  contribute  to  reinforcing  the  existing  evidence-base  on  the 
ability of the Action to produce cultural, social and economic impact. Finally, the objective is 
to draw lessons, conclusions and recommendations that may be useful for Union institutions, 
future ECOC or cities wishing to bid for the ECOC title in the future. 

The Commission started these yearly evaluations with the 2007 titles and the evaluation of the 
ECOC 2019 is  the latest  to  date in  a series of 12 similar  yearly exercises.  It  is  also  the last 
yearly evaluation to be done by the Commission, as a new legal basis applies from the ECOC 
2020  titles  onwards2,  making  cities  themselves  (instead  of  the  Commission)  responsible  for 
the evaluation of the results of theirs years as ECOC3.  

This staff-working document is the fifth covering the ECOC Action in as many years, while 
the  Action  in  itself  has  not  changed  in  the  period  considered,  with  only  the  pair  of  cities 
hosting the Action being different each year. 

This  staff-working  document  summarizes  the  findings  of  the  external  evaluation  of  the 
implementation of the ECOC 2019, including the selection and monitoring procedures as well 
as the operational delivery by the two hosting cities, namely Matera in  Italy and Plovdiv in 
Bulgaria4. The evaluation examined how the two cities developed their respective applications 
and cultural programmes, how they delivered the planned initiatives throughout the year and 
any cultural, general and longer-term impact generated by the Action. 

The  evaluation  constitutes  a  valuable  opportunity  to  reconsider  critically  the  past  year  with 
the  intention  of  collecting  further  insight  and  lessons  based  on  the  experiences  of  the  host 
cities.  

1   Decision  No  1622/2006/EC  of  the  European  Parliament  and  of  the  Council  of  24  October  2006  establishing  a 
Community Action for the European Capital of Culture event for the years 2007 to 2019 (OJ L 304, 3.11.2006, p. 1), 
available at:  

- http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:304:0001:0006:EN:PDF. 

2   Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union action 
for the European Capitals of Culture for the years 2020 to 2033 and repealing Decision No 1622/2006/EC (OJ L 132, 
3.5. 2014, p. 1), available at: No 1622/2006/EC, available at: 

- http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2014.132.01.0001.01.ENG 

3   Article 16(1) of Decision No 445/2014/EU. 

4  

document 

Full 
available 
01aa75ed71a1/language-en. 

at: 

https://op.europa.eu/en/publication-detail/-/publication/01ae1db0-3a98-11eb-b27b-

3 

 
 
                                                           
 
It  is  worth  reiterating  here  what  was  underlined  in  last  year’s  staff-working  document5, 
namely  that  the  aim  of  the  yearly  evaluation  is  not  to  lead  to  any  change  in  the  regulation 
governing  the  ECOC  Action.  Indeed,  as  mentioned  above,  those  rules  already  changed  in 
20146, but the new rules only apply for cities designated as ECOC for the years from 2020 to 
20337.  It  would  therefore  be  premature  to  use  the  outcomes  of  the  evaluation  of  the  two 
ECOC 2019, governed by previous Decision No 1622/2006/EC, to draw conclusions related 
to Decision No 445/2014/EU, which introduced changes in particular regarding the selection 
procedure, monitoring arrangements or the payment of the Melina Mercouri Prize8. 

1.  BACKGROUND TO THE INTERVENTION 

1.1.  The European Capital of Culture Action 

The initial scheme of "the European City of Culture" started at an intergovernmental level in 
19859  upon  an  idea  of  the  then  Greek  Minister  of  Culture,  Melina  Mercouri.  The  scheme 
recognized  Europe  as  a  centre  for  artistic  development,  underpinned  by  an  exceptional 
cultural richness and diversity, with cities playing a vital role in the formation and spread of 
cultural expressions. 
In  1999,  Decision  1419/1999/EC  of  the  European  Parliament  and  the  Council10  gave  the 
scheme the status of a Community Action under the name of "European Capital of Culture" 
and set up a more predictable, consistent and transparent system for the designation of hosting 
cities. Its legal foundation was Article 151 of the Treaty (now Article 167), which calls on the 
Union to “contribute to the flowering of the cultures of the Member States, while respecting 
their  national  and  regional  diversity  and  at  the  same  time  bringing  the  common  cultural 
heritage  to  the  fore”.  The  Decision  introduced  new  selection  procedures  and  evaluation 
criteria while Member States were ranked in a chronological order of entitlement to host the 
event each year.  
In  2005,  the  European  Parliament  and  the  Council  adopted  Decision  649/2005/EC11  to 
integrate the ten countries that joined the European Union in 2004. 

5   SWD(2020) 165 final of 18.08.2020. 

6  

See footnote n° 2. 

7   Article 17 of Decision No 445/2014/EU. 

8   The new Decision, which covers the ECOC titles 2020 to 2033, retains the general structure and main elements of the 
previous Decision but introduced improvements to maximise the benefits of holding the title as well as taking part in the 
competitive  process  for  all  bidding  cities  and  their  citizens.  Improvements  include  among  others  the  introduction  of 
more explicit and measureable criteria, the reinforcement of conditionality for the payment of the Melina Mercouri Prize 
and the obligation for the cities – instead of the Commission – to carry out the ex-post evaluation of the ECOC year. 

9   Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the 
annual event 'European City of Culture' (85/C 153/02), on the initiative of the former  Greek Culture Minister, Melina 
Mercouri.  

10  Decision  1419/1999/EC  of  the  European  Parliament  and  of  the  Council  of  25  May  1999  establishing  a  Community 

action for the European Capital of Culture event for the years 2005 to 2019 (OJ L 166, 1.7.1999). 

11   Decision  649/2005/EC  of  the  European  Parliament  and  of  the  Council  of  13  April  2005  amending  Decision 
No 1419/1999/EC establishing a Community action for the European Capital of Culture event for the years 2005 to 2019 
(OJ L 117, 4.5.2005). 

4 

 
 
                                                           
 
In 2006, a new Decision – Decision 1622/2006/EC12 – entered into force. This Decision kept 
the principle of a chronological order of Member States but further refined the objectives of 
the  Action 
arrangements. 
Decision 1622/2006/EC is the legal basis governing the European Capital of Culture event for 
the titles 2007 to 2019. 

and  monitoring 

introduced 

selection 

new 

and 

As  mentioned  under  section  1,  in  April  2014,  the  European  Parliament  and  the  Council 
adopted  a  new  Decision,  but  cities  designated  as  ECOC  for  the  years  up  to  2019  continue 
being regulated by Decision No 1622/2006/EC.  

The  2014  Decision  was,  in  turn,  amended  in  September  2017  to  open  the  ECOC  Action  to 
European Free Trade Association countries that are parties to the Agreement on the European 
Economic Area13. It was again amended, in December 2020, to take account of the impact of 
the  COVID-19  pandemic  on  the  implementation  of  the  two  ECOC  2020  (Rijeka  in  Croatia 
and  Galway  in  Ireland)  and  on  the  preparatory  work  of  the  three  ECOC  2021  (Elefsina  in 
Greece, Timisoara in Romania and Novi Sad in Serbia)14.  

1.2.  Objectives of the ECOC Action 

The ECOC Action aims to highlight the richness and diversity of European cultures and the 
features  they  share,  thereby  promoting  greater  mutual  understanding  among  European 
citizens, as well as to foster the contribution of culture to the long-term development of the 
cities. ECOC shall strive to foster cooperation between cultural operators, artists and cities in 
Europe,  foster  the  participation  in  cultural  activities  of  the  citizens  living  in  the  city  and 
surroundings while raising the interest of citizens from abroad, to be sustainable and to be an 
integral part of the long-term cultural and social development of the city. 

The hierarchy of objectives presented in the table below is based on the objectives as stated in 
Decision  No 1622/2006/EC,  but  it  has  been  updated  to  reflect  the  content  of  the  new  legal 
basis  for  ECOC  post-2019.  The  general  and  strategic  objectives  are  taken  directly  from 
Article 2 of Decision No 445/2014/EU, with the operational objectives flowing logically from 
these.  They  are  also  informed  by  the  selection  criteria  detailed  in  Article  5  of  the  2014 
Decision.

12   Decision 1622/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Community 

action for the European Capital of Culture event for the years 2007 to 2019 (OJ L 304, 3.11.2006). 

13   Decision (EU) 2017/1545 of the European Parliament and of the Council of 13 September 2017 amending Decision No 
445/2014/EU establishing a Union action for the European Capitals of Culture for the years 2020 to 2033 (OJ L 237, 
15.9.2017). Text available at: 
http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32017D1545. 

14    Decision (EU) 2020/2229 of the European Parliament and of the Council of 23 December 2020 amending Decision No 
445/2014/EU establishing a Union action for the European Capitals of Culture for the years 2020 to 2033 (OJ L 437, 
28.12.2020). This Decision provides Rijeka and Galway with the possibility to extend their title-year until end of April 
2021  and  postpones  the  title-year  of  Novi  Sad  from  2021  to 2022  and  the  title-years  of  Elefsina  and  Timisoara  from 
2021 to 2023. Text available at : 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32020D2229 

5 

 
 
                                                           
 
Table on ECOC hierarchy of objectives 

General objective 

Safeguard and promote the diversity of cultures in Europe, highlight the common features they share, and foster the contribution of culture to the long-term development 
of cities 

Specific objectives (SO) 

SO1:  Enhance  the  range,  diversity  and  European 
dimension of the cultural offer in cities, including 
through transnational co-operation 

SO2:  Widen  access  to  and  participation  in 
culture 

SO3:  Strengthen  the  capacity  of  the 
cultural  and  creative  sector  and  its 
links with other sectors 

SO4: Raise the international 
profile  of  cities 
through 
culture 

Operational objectives 

Stimulate  a  diverse  range  of  cultural  activities  of 
high artistic quality  

Create new and sustainable opportunities for 
a  wide  range  of  citizens  to  attend  or 
participate in cultural events 

Implement  cultural  activities  promoting  cultural 
diversity, dialogue and mutual understanding 

Implement cultural activities highlighting (shared) 
European cultures and themes  

Involve  local  citizens,  artists  and  cultural 
organizations 
and 
implementation 

development 

in 

Improve cultural infrastructure 

Develop 
governance of the cultural sector 

skills,  capacity  or 

the 

Stimulate 
operation with other sectors 

partnership 

and 

co-

Attract  the  interest  of  a 
broad 
and 
European 
international public  

Involve  European  artists,  promote  cooperation 
transnational 
with  different 
partnerships  

countries 

and 

Provide  opportunities  for  volunteering  and 
foster links with schools and other education 
providers 

Combine  traditional  art  forms  with 
new types of cultural expression 

6 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2. 

IMPLEMENTATION / STATE OF PLAY 

2.1.  The selection and monitoring of the European Capitals of Culture 2019 

In accordance with Decision No 1622/2006/EC, Italy and Bulgaria were the two Member 
States entitled to host the ECOC in 2019.  

Under  this  Decision,  host  countries  are  responsible  for  the  procedure  leading  to  the 
selection of one of their cities as "European Capital of Culture". This is done through an 
open  competition  within  the  Member  State  concerned.  Against  this  background,  the 
relevant  managing  authorities of  Italy  and Bulgaria, respectively the  Italian Ministry of 
Cultural  Heritage  and  Activities  and  Tourism,  on  the  one  hand,  and  the  Bulgarian 
Ministry of Culture, on the other hand, ran parallel competitions, which both started with 
the publication of a call to trigger applications from interested cities.  

The selection happens in two phases: a pre-selection round (candidate cities are reduced 
to a short-list) followed by a selection round (one city is recommended for the title). A 
panel of thirteen members – six of whom nominated by the Member State concerned and 
the  other  seven  by  European  Union  institutions  and  bodies  (European  Parliament, 
Council,  Commission  and  Committee  of  the  Regions)  –  examines  the  bids  from 
candidate cities based on the objectives and criteria laid down in the Decision. 

In Italy, the Ministry issued its call in November 2012. It  received a record number of 21 
applications by the deadline of 20 September 2013, which illustrates the high popularity 
and strong attractiveness of the ECOC title in the country. At the pre-selection meeting in 
November  2013,  the  panel  recommended  the  ministry  to  invite  six  cities  to  the  final 
stage: Cagliari, Lecce, Matera, Perugia, Ravenna and Siena. Finally, in October 2014, the 
panel recommended that Matera become, in 2019, the fourth Italian city to get the ECOC 
title after Florence in 1986, Bologna in 2000 and Genoa in 200415. 

In  Bulgaria,  the  Ministry  published  its  call  for  applications  in  December  2012.  By  the 
deadline  of  18  October  2013,  eight  cities  had  submitted  a  bid.  At  the  pre-selection 
meeting  in  December  2013,  four  of  them  (Plovdiv,  Sofia,  Varna  and  Veliko  Tarnovo) 
were short-listed. In October 2014, the panel recommended that the ECOC 2019 title in 
Bulgaria, the first ever in the country, be awarded to Plovdiv. 

The Council of the European Union formally designated Matera  and Plovdiv as ECOC 
2019 in a Decision adopted in May 201516. 

In  line  with  Decision  No  1622/2006/EC,  once  designated  as  ECOC  and  until  the  title-
year,  the  two  cities  had  to  adhere  to  a  monitoring  procedure  directly  managed  by  the 
Commission.  

All along this phase, the progress in the cities' preparations was monitored and guided by 
a  panel  composed  of  the  seven  independent  experts  appointed  by  the  European  Union 

15  All  panel’s  pre-selection,  selection  and  monitoring  reports  are  available  at:  https://ec.europa.eu/culture/matera-

and-plovdiv. 

16   Council Decision (EU) 2015/809 of 19 May 2015 designating the European Capitals of Culture for the year 2019 

in Bulgaria and Italy, at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32015D0809 

7 

 
 
 
                                                           
institutions  and  bodies,  which  also  checks  compliance  with  the  programme  and 
commitments  on  the  basis  of  which  the  two  cities  had  been  selected.  As  part  of  this 
process,  Matera  and  Plovdiv  submitted  progress  reports  and  attended  two  formal 
monitoring  meetings  with  the  panel  that  the  Commission  convened  respectively  in 
October 2016 and in April 2018. On top of that, the Commission organized an informal 
post-designation  meeting  (that  does  not  result  in  public  reports,  as  it  does  not  have  a 
formal status) between the panel and both Matera and Plovdiv in September 2015. There 
were  also  in  situ  visits  (whereby  a  delegation  of  the  panel  and  staff  members  of  the 
Commission visit a hosting city to get a better insight of the development of the project) 
in both Matera and Plovdiv during the monitoring phase.  

Over  this  period,  the  panel  made  a  number  of  recommendations  to  the  two  cities,  in 
particular to ensure that they adequately involve all relevant groups of citizens, develop 
an  ambitious  European  dimension,  put  in  place  a  clear  governance  setting,  clarify  and 
accelerate the financial commitments and reflect on a proper legacy of their ECOC-year.  

As  was  the  case  for  all  previous  ECOC,  during  the  development  period,  the  two  cities 
introduced  modifications  into  the  programme  described  in  their  original  applications, 
either  in  response  to  a  changing  environment  or  to  the  recommendations  made  by  the 
panel.  

The monitoring process culminated with the panel making a positive recommendation to 
the  Commission  on  awarding  the  €1.5m  prize  in  honour  of  Melina  Mercouri  to  both 
cities  after  the  last  monitoring  meeting  in  April  2018.  Based  on  this  positive 
recommendation, in the autumn 2018, the Commission awarded and paid the prize to the 
two ECOC 2019 from the budget of the EU Creative Europe programme17. 

The sub-sections below describe the main features of the ECOC programmes of Matera 
and Plovdiv. They are based on the final report of the contract supporting the evaluation. 
This final report contains further information, including on the way the programmes had 
a  European  dimension  and  involved  citizens  (as  these  are  the  main  two  criteria  of  the 
ECOC Action) as well as on the development of the applications and the governance and 
funding structures. 

2.2.  Matera 2019 

Matera is a relatively small city of 60,000 inhabitants located in the Basilicata region of 
Southern Italy. It is the capital of the Province of Matera and is also known as the “Cittá 
Sotteranea”  (the  “Underground  City”).  Today,  Matera  is  known  worldwide  for  its 
historical  centre,  known  as  “I  Sassi”,  with  ancient  cave  dwellings  inhabited  since  the 
Palaeolithic period. In 1993, this part of the city was awarded a World Heritage Site by 
Unesco. 

Whilst  affected  by  high  levels  of  poverty  and  infant  mortality  rates  until  the  1950s, 
Matera  is  also  a  good  example  of  urban  rebirth  and  regeneration,  which  started  in  the 
1960s and have ultimately  turned the city into a  regional capital  for leisure and culture 
and a destination for regional visitors. Matera has a strong rural focus and tradition, and 
agriculture  has  represented  the  main  economic  activity  for  many  centuries.  Today, 
Matera is characterised by a diversified economic sector, largely impacted by the rise of 

17   Regulation (EU) No 1295/2013 of the European Parliament and of the Council of 11 December 2013 establishing 
the  Creative  Europe  Programme  (2014  to  2020)  and  repealing  Decisions  No  1718/2006/EC,  No  1855/2006/EC 
and No 1041/2009/EC (OJ L 347, 20.12.2013, p.221). 

8 

 
                                                           
the  tourism,  handcraft  and  research  sectors,  which  are  replacing  the  local  “furniture 
district” emerged in the last decades of 20th century. 

The decision to prepare an ECOC application was taken by the municipality of Matera in 
2010, as a result of the growing interest towards this initiative within the city and among 
local cultural and  youth  associations. The application highlighted the importance of the 
cultural  renovation  process,  not  only  for  Matera,  but  for  the  entire  South  of  Italy,  in  a 
period characterised by an economic and social decline.  

Under  the  title  “OpenFuture”,  the  cultural  programme  aimed  to  “strengthen  broad 
ranging, open and diversified citizenship; to enhance international relations and above all 
to  transform  Matera  into  the  most  important  platform  of  open  culture  in  Southern 
Europe”. The programme was articulated around five different thematic lines: 

  Ancient Futures: to explore the relationship with nature and landscape; 
  Continuity and disruptions: to implement  a process aimed at understanding how 
to collectively give shape to our cities and find beauty in the spaces inhabited on a 
daily basis; 

  Utopias  and  dystopias:  to  test  new  radical  models  that  challenged  specific 

assumptions with the aim to changing attitudes and mentality; 

  Roots  and  routes:  to  focus  on  the  culture  of  mobility  that  brings  together 

European citizens; and 

  Reflections and connections: to re-examine identity and provide citizenship with 
a new model of ‘everyday life’ based on art, science and the widespread practice 
of cultural citizenship. 

Furthermore,  two  infrastructural  projects  were  at  the  heart  of  the  programme  and 
explored  across  these  five  overarching  themes:  I-DEA  (Institute  of  Demo-Ethno-
Anthropological  Archive)  –  an  archive  meant  to  collect  a  large  number  of  public 
documents  and  material  from  private  collections  and  cultural  associations  –,  and  the 
Open Design School, meant as the first European design school founded on the principles 
of  open  culture,  as  it  brought  together  authors,  bloggers,  designers,  craftsmen,  hackers, 
graduates, students and professionals. 

The  responsibility  for  the  organisation  and  implementation  of  the  ECOC  project  was 
entrusted to a new body, the Matera-Basilicata Foundation 2019, which was created as a 
public  entity  in  2014.  The  overall  ECOC  programme  was  overseen  by  a  Board  of 
Directors and a Board of Trustees, including the dean and associate professors from the 
Basilicata University, the mayor of Matera and representatives from the Matera Province 
and the Basilicata regional authority. In addition, a dedicated and international Scientific 
Board provided relevant expertise and advice to the managing boards. Local authorities 
and officials were also directly involved in the Foundation with managing roles. While a 
General  Manager  was 
the  general  administration,  marketing, 
communication and fundraising, three managers were in charge of the implementation of 
three specific departments: Networking, Administration and Cultural programming. 

responsible 

for 

As far as budget is concerned, the figure indicated in the application amounted to €52m. 
The actual budget for the delivery of the ECOC was very close to the initial planning at 
€54,8m. Approximately €47m was covered by the public purse: €33.2m was provided by 
the  national  Government,  €12.2m  came  from  the  regional  authority  and  the  Matera 
municipality  while  the  rest  of  the  public  contribution  (€1.6m)  was  secured  through  EU 
and  national  funding  applications,  including  the  Melina  Mercouri  prize.  Finally,  some 
limited funding was secured through private partnerships and sponsorships. 

9 

 
2.3. 

Plovdiv 2019 

Plovdiv  is  the  second  largest  city  in  Bulgaria  after  the  capital  city  Sofia,  with  a 
population  of  around  347,000  people.  It  is  Southern  Bulgaria’s  cultural  and  business 
centre.  The  city  is  unique  by  its  8,000-year  long  history  and  is  one  of  the  oldest,  still-
inhabited  cities  in  the  world.  Originally  a  Thracian  settlement,  it  was  subsequently 
inhabited  by  Persians,  Greeks,  Celts,  Romans,  Goths,  Bulgars,  Slavs  and  Turks.  As  a 
result, it boasts a diverse cultural heritage and a multi-ethnic population. Its location on 
the  banks  of  the  Maritsa  River,  the  longest  river  in  the  Balkans,  and  at  the  foot  of  the 
Rhodope Mountains makes it an attractive tourist centre. 

The  economy  of  Plovdiv  has  long  traditions  in  manufacturing,  commerce,  transport, 
communications  and  tourism.  The  city  has  also  a  vibrant  atmosphere  for  young  people 
with  its  six  universities  and  is  known  among  Bulgarians  as  the  city  of  “Ayliak”,  a 
Turkish  word  used  to  express  a  relaxed  attitude  to  life.  Plovdiv  has  a  large  variety  of 
museums,  galleries,  theatres, an opera and the only Centre  for Contemporary Art of its 
kind  in  Bulgaria,  situated  in  the  premises  of  an  ancient  bath  from  the  period  of  the 
Turkish  Empire.  It  also  boasts  numerous  remains  from  antiquity,  including  an  Ancient 
Theatre,  one  of  the  world's  best  preserved  ancient  theatres,  which  is  frequently  used  to 
host cultural events. 

The main motivation for Plovdiv bidding for the ECOC was to use culture as a driver of 
change  for  challenges  related  to  its  multi-ethnic  background,  cultural  heritage  and 
belonging to the wider European context. These typical features and specific challenges  
of  the  city  context  became  fundamental  elements  of  the  ECOC  application  and  overall 
programme  and  is  where  most  themes  and  projects  were  focused.  Under  the  motto, 
“Together”, inspired by the co-existence of different ethnic minorities, social groups and 
religious communities, the cultural programme was structured around the following four 
thematic platforms: 

  Fuse: to explore how best to integrate Plovdiv’s ethnic and minority groups and 
to  bring  together  different  generations  and  social  groups,  in  order  to  overcome 
exclusion in isolated territories; 

  Transform: to focus on the revitalization of forgotten and abandoned urban spaces 

and on changing the perceptions of such spaces among citizens; 

  Revive: to preserve cultural heritage in a contemporary context by improving the 
access to it and stimulating the use of new technology and digital media; and 

  Relax: to promote sustainable living, slow life, slow food and de-growth. 

Furthermore, with regard to content, for the very first time in Bulgaria, there was a clear 
willingness  from  the  ECOC  delivery  team  to  attempt  to  add  a  social  dimension  to  the 
cultural  offer  in  the  city,  addressing  through  the  cultural  programme  issues  such  as 
injustice, inequality and environmental matters but also ‘deeper’ issues in society such as 
the  integration  of  the  Roma  minority  in  the  city’s  Stolipinovo  district  and  raising 
awareness on LGBT issues. 

As with other ECOC, the implementation of the ECOC was entrusted to an ad hoc entity, 
the Plovdiv 2019 Foundation, which was created in September 2011 first to develop the 
application  and  then  to  implement  the  project.  The  Board  of  Directors  was  the  main 
governing body of the Foundation, consisting of members designated by the City Council 
of  Plovdiv.  The  role  of  the  Board  was  to  examine  and  approve  the  main  documents 
related to the Foundation's activities such as the annual programmes, reports, budgets and 
expenditures.  The  day-to-day  implementation  of  the  ECOC  was  led  by  the  Executive 

10 

 
Director and the Artistic Director, who shared the same level of responsibility. They were 
assisted  by  three  Deputy  Directors,  respectively  in  the  areas  of  Marketing,  Advertising 
and Commercial Activity; International Relations; and the Programme itself. 

The  original  total  budget  (covering  both  operation  and  infrastructure)  planned  at 
application stage for the delivery of the ECOC project was estimated at €38.2m, with the 
national Government, the City of Plovdiv and private sponsors contributing respectively 
€10m,  €23.2m,  and  €3.5m.  In  practice,  the  total  budget  amounted  to  €29.4m.  The 
decrease was partly due to the fact that the Foundation failed to reach its ambitious goal 
in  terms  of  private  sector’s  income  (the  actual  figure  was  €0.5m,  far  from  the  original 
target18)  and  the  contribution  from  the  City  budget  was  also  lower  than  expected  at 
€5.7m19. 

3.  METHOD 

In order for results to be comparable with previous evaluations, the methodology for this 
evaluation closely followed the approach adopted in previous assessments of the Action. 
The focus has been on research at city level and, in particular, the gathering of data and 
stakeholders' views from both Matera and Plovdiv. The main evaluation sources can be 
identified as follows: 

-  EU  level  literature20:  this  included  generic  EU  policy  and  legislative  briefings, 
papers,  decisions  and  other  documents  relating  to  ECOC.  This  mainly  focussed 
on  reports  of  the  panels  for  selection  and  monitoring  and  the  original  bidding 
guidance  to  understand  how  the  two  ECOC  established  themselves  in  the  early 
days. Academic research was also consulted regarding the ECOC Action and the 
role of culture in the development of cities; 

-  ECOC  level  literature  from  Matera  and  Plovdiv:  this  included  original  bids  and 
applications,  internal  reports  linked  to  the  application,  monitoring  or  evaluation 
processes,  application  packages  for  calls  for  proposals  and  numerous  pieces  of 
literature collected on the cultural programme itself; 

-  Quantitative data: where available, evidence linked to each ECOC was collected 
in relation to budgets and expenditures, projects' numbers and types, participation 
and audience figures, social media data, statistics from  AirBnB  and Vrbo rental 
properties, number of domestic and international tourists, as well as other pieces 
of quantitative data to show and describe the work and benefits of the ECOC; 
-  A total of 60 interviews were conducted (30 in relation with Matera, 29 in relation 

with Plovdiv and one with an EU expert). These included: 

o  Interviews  with  managing  teams:  those  responsible  for  the  day-to-day 
design and delivery of the ECOC were interviewed in each city remotely 
in the period from April to June 2020. Almost all of key staff within the 
delivery  agencies  were  interviewed,  including  those  responsible  for  the 

18   It  should  be  noted  in  this  respect  that  using  private  sponsors’  funding  for  cultural  content  is  not  a 

widespread practice in Bulgaria, as compared to other EU countries. 

19   This figure however does not include funding from the City of Plovdiv for infrastructure investments 
made from the start of the bidding process until the actual designation in 2015, as well as investments 
made in the title year, which would be finalised in 2020 and 2021. 

20   Full  list  of  EU  level  and  ECOC  level  literature  is  in  Annex  1  of  the  final  report  submitted  by  the 

contractor supporting the evaluation. 

11 

 
                                                           
executive  management,  artistic  direction,  marketing  and  communication, 
international relations, project implementation and financial management; 
o  Interviews  with  other  key  stakeholders:  remote  interviews  were  also 
undertaken  with  stakeholders  both  directly  and  indirectly  involved  in 
either the planning or delivery of the ECOC along with those more widely 
linked  to  the  cultural,  social,  economic  or  political  agenda  of  the  host 
cities. Stakeholders included those working in cultural organisations, local 
authorities,  representatives  of  the  business  sector  and  tourism  agencies, 
media  organisations,  as  well  as  voluntary  and  community  organisations. 
Managers  of  individual  projects  and  activities  supported  through  the 
ECOC  that  made  up  the  cultural  programme  of  each  city  were  also 
interviewed.  In  addition,  some  international  partners  of  the  host  cities 
were interviewed, which provided an external perspective on the ECOC; 
o  Interviews  with  EU-level  stakeholders:  one  member  of  the  selection  and 

monitoring panel gave feedback; 

-  Survey  among  project  leaders  and  cultural  operators  in  Matera:  in  order  to 
corroborate  evaluation  findings,  a  survey  among  project  leaders  and  cultural 
operators  in  Matera  was  conducted,  but  the  response  rate  was  extremely  low  at 
5%, most likely due to survey fatigue as the same stakeholders had been already 
surveyed by the Foundation and other research organisations. The survey aimed 
to collect evidence of impact of the ECOC on the cultural sector and the capacity 
of  cultural  operators.  To  assess  this  aspect  in  absence  of  representative  results 
from the survey, the study team used: 1) data on the capacity building activities 
implemented;  2)  qualitative  evidence  from  interviews  (including  projects);  3) 
statistical data on cultural sector; 
In  Plovdiv,  in  the  period  from  2017  until  2020,  several  surveys  (i.e.  a 
representative  survey  among  the  adult  population  of  Plovdiv,  a  poll  among 
visitors of ECOC events and a survey among Roma households in the city district 
of Stolipinovo) as well as 50 interviews with cultural institutions were conducted 
by  the  private  consultancy  company  commissioned  to  monitor  the  ECOC.  As 
there  was  already  extensive  data  collected  and  in  order  to  avoid  duplication  of 
stakeholder consultation activities, no survey was launched in Plovdiv within the 
ex-post evaluation of ECOC 2019; 

- 

-  The evaluation does not include a wider public consultation. As explained in the 
roadmap21, the Action is considered to be local while international participation is 
scattered within and outside Europe and is difficult to reach. On the other hand, 
the opinions would be based on attendance to specific events and would not give 
useful insights for the evaluation of the ECOC Action as a whole.  

The  final  report  of  the  contract  supporting  the  evaluation  provides  a  detailed 
understanding of the 2019 ECOC Action and within this an assessment of the work and 
progress of Matera and Plovdiv. There are, however, issues to consider when assessing 
the strengths of the evidence base used for this study: 

-  As  already  mentioned  in  previous  evaluations,  an  ideal  way  of  conducting  this 
evaluation would entail a before ('baseline') study and an after-picture ('ex-post') 
study, instead of carrying out the latter alone. The impossibility of comparing the 
two studies  affects  the accuracy of the evaluation. This  means that any  baseline 

21   See https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-3143701_en. 

12 

 
                                                           
information and assessment prior to the ECOC year relies on secondary evidence 
supplied by the ECOC cities; 

-  Moreover,  since  the  evaluation  was  undertaken  from  December  2019  until 
October  2020,  i.e.  at  the  very  end  and  shortly  after  the  end  of  the  ECOC  time 
framework, some of the effects of the programme had not manifested themselves 
entirely.  Many  stakeholders  involved  in  the  evaluation  commented  that  the  real 
impact  of  the  ECOC  on  the  city  and  its  residents  would  take  time  to  filter 
through; 
In this respect, the Commission once again highlights that budget22 and timing23 
only  allow  for  an  ex-post  evaluation  to  take  place  and  therefore  only  an  after 
picture has been studied; 

- 

-  An ulterior consequence of the modest yearly budget allocated to the evaluation is 
the fact that the primary evidence data gathering tends to be more of qualitative 
than  quantitative  nature;  while  qualitative  data  still  holds  a  great  importance  in 
the  evaluation,  the  lack  of  diversity  of  data  sources  translates  into  a  lesser 
dependability, for instance, in the process of proving the objective outcomes and 
impacts of ECOC on widening participation in culture; 

-  Detailed  modelling,  economic  impact  assessments  or  large-scale  surveys  were 
outside the scope of this study due to the budget constraints. This evidence used 
outside of the interviews is dependent on the local evaluation and other research 
commissioned  by  the  ECOC  cities.  If  the  cities  have  commissioned  large  and 
ambitious  evaluations  which  provide  quantitative  data  on  impacts,  economic 
benefits and population surveys before, during and after the ECOC year then the 
European evaluation will greatly benefit from this information. 

4.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

This part of the report presents a diagnosis of the implementation of the ECOC Action in 
2019. It draws together the results relating to the two ECOC 2019 and to a lesser extent 
the findings from previous ECOC evaluations. 

All  figures and evidence appearing in the following sub-sections come directly from the 
report of the external ex-post evaluation of the two ECOC 2019. The reader will find in 
the full text of the document many more examples illustrating the conclusions presented 
below. 

Very often, examples and figures can illustrate different points made under "relevance", 
"efficiency",  "effectiveness",  "sustainability",  "EU  added  value"  and  "coherence". They 
are  therefore  sometimes  repeated  in  various  sub-sections.  However,  for  the  ease  of  the 
reading,  repetitions  have  been  limited  to  the  minimum  and  the  reader  is  invited  to 
correlate some conclusions presented below with examples or figures that may have been 
given in previous paragraphs or may be given in following ones. 

4.1.  Relevance 

Findings from the ECOC 2019 evaluation show that the two title-holders developed and 
implemented cultural programmes that were consistent with the EU Treaty, in particular 

22   The budget allocated to the evaluation work (70 000 €) is proportionate to the low level of EU funding directly 
provided to the ECOC (i.e. an award based on a recommendation of the panel after the final monitoring meeting 
in the form of the €1.5m Melina Mercouri Prize). 

23   Decision No 1622/2006/EC requires that the Commission conduct the evaluation immediately after the title year. 

13 

 
                                                           
Article 16724, as well as with Decision No 1622/2006/EC. Indeed, the ECOC Action was 
a  good  vehicle  for  the  two  cities  hosting  the  title  in  2019  to  promote  cultural  diversity 
and  highlight  commonalities,  contributing  in  this  way  to  the  "flowering"  of  the  local, 
national  and  European  cultural  scene.  As  is  shown  in  following  sub-sections  through 
many  concrete  examples,  the  two  cities  organized  thousands  of  events  on  top  of  their 
usual  cultural  offer.  These  events  covered  all  types  of  cultural  and  artistic  disciplines, 
targeted  many  groups  of  audiences  and  presented  various  aspects  of  the  diversity  and 
common features of cultures in Europe, notably through co-operations with partners from 
other European countries and beyond. 

Matera’s drivers for bidding – i.e. using the ECOC title as an opportunity for a cultural 
renovation, opening up the city to Europe, strengthening a diversified cultural offer and 
widening access to and participation in culture – were quite in line with the objectives of 
the  overall  ECOC  Action,  and  the  initial  intentions  became  a  reality  during  the 
preparation  and  implementation  years  of  the  ECOC  project,  in  particular  thanks  to  the 
ongoing commitment of local organisations and citizens all along the process. The main 
motivations  for  Plovdiv  wanting  to  get  the  ECOC  title,  using  culture  as  a  driver  of 
change  for  challenges  related  to  its  multi-ethnic  background,  cultural  heritage  and 
belonging  to  the  wider  European  context,  also  corresponded  to  the  objectives  of  the 
ECOC  Action,  and  found  a  concrete  translation  in  the  many  projects  included  in  the 
ECOC cultural programming. This is illustrated in the paragraphs below. 

The European dimension in Matera 

The  cultural  activities  that  were  part  of  Matera  2019  ECOC  programme  contributed  to 
promote cultural diversity, dialogue and mutual understanding. 

First, Matera 2019 involved a large number of international artists, project managers and 
partner associations. Out of the total number of artists and the project leaders involved by 
the  Foundation,  45%  were  international.  In  addition,  over  1,000  international  mobility 
programmes  were  realised  with  55  different  countries.  A  total  of  15  debates  involving 
international  artists  took  place  on  themes  such  as  urban  regeneration,  cultural 
engagement and productions. This is all the more remarkable because, prior to its ECOC 
year,  it  was  widely  recognized  that  the  city  and  the  region  suffered  from  a  lack  of 
international  connections  and  partnerships  and  had  a  limited  capacity  to  attract 
international  cultural  operators.  To  address  this  challenge,  the  Foundation  put  in  place 
several activities already during the lead in period to the title-year. A good illustration is 
the  ‘Go&See’  action,  which  allowed  27  local  cultural  operators  to  visit  another  EU 
country to meet artists or other cultural organisations they wanted to collaborate with or 
learn from. 

Another  aspect  that  strongly  contributed  to  the  promotion  of  the  European  dimension 
were the efforts made to host artists from all over the world, in particular from Europe.  
The involvement and participation of these artists in many cultural events and initiatives 
led  to  a  diversification  of  the  local  cultural  offer  and  greatly  contributed  to  promote  a 
European  and  international  dimension  in  the  programme.  As  an  example,  the  initiative 
‘Residenze Matera 2019’ established different artistic residences that promoted European 
cultural  values  and  diversity.  In  this  framework,  a  total  of  41  artists  from  all  the  EU 
countries,  but  also  from  Switzerland,  the  UK  and  Turkey,  were  hosted  in  Matera25.  In 

24   http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A12008E167.  

25   Foundation  Matera  2019.  Final  Monitoring  Report:  A  Matera  si  produce  cultura.  Accessed  from:  

https://www.matera-basilicata2019.it/it/report-2019/a-matera-si-produce-cultura.html  

14 

 
                                                           
addition, 421 international artists were hosted in Matera or in other cities in Basilicata to 
cooperate with the Foundation and project leaders on the design and implementation of 
the original productions of Matera 2019.  

Finally, leveraging on the ECOC brand, the Foundation aimed to reinforce the links with 
former  ECOC  by  organising  capacity-building  activities  with  Aarhus  (sustainability 
policies and volunteers), Leeuwarden (work with communities) and Marseille (safety of 
public  events,  circus  and  legacy).  There  was  also  collaboration  with  Plovdiv  as  further 
illustrated below in the corresponding paragraph dedicated to Plovdiv. 

The “City and Citizens” dimension in Matera 

Matera’s original application stressed that the  cultural  programme  would be centred on 
the  citizens,  who  were  seen  as  the  real  protagonists  of  the  ECOC  2019.  As  already 
mentioned above, the ultimate original goal was to make Matera a European co-creation 
capital through the direct involvement of citizens in the development and implementation 
of cultural projects, activities and laboratories.  

According to the evaluator, this was achieved through the very pro-active willingness of 
the Foundation: i) to include different sub-segments of citizens in the cultural activities 
organised  (such  as,  for  example,  migrants,  refugees  and  people  with  disabilities  in  the 
project  Silent  Academy26),  ii)  to  encourage  local  citizens  to  actively  participate  in  the 
preparation of cultural events through open public calls (they resulted in 80 applications 
and  ultimately  27  projects)  and  iii)  to  provide  them  with  tools  to  facilitate  their  direct 
involvement  (with  a  strong  emphasis  on  co-creation  and  capacity  building  through 
workshops, trainings and brainstorming sessions). 

(and  continues  providing)  a 

A very telling example of such a commitment was (and still is) the Open Design School, 
which  provided 
interdisciplinary 
experimentation  and  innovation.  It  rapidly  became  a  meeting,  sharing,  mutual  learning 
and  practising  place,  where  citizens  had  the  opportunity  to  cooperate  with  over  100 
national  and  international  artists.  With  the  direct  involvement  of  citizens,  the  School 
contributed to the realisation of nine large exhibitions, 11 urban installations, 10 cultural 
events (including the opening ceremony) and to the mapping of 400 venues and locations 
for cultural activities scattered all over Matera.  

laboratory  of 

The European dimension in Plovdiv 

As  was  the  case  in  Matera,  the  ECOC  project  in  Plovdiv  was  a  good  vehicle  to 
implement  cultural  activities  promoting  cultural  diversity,  dialogue  and  mutual 
understanding. 

The  main  challenge  here  was,  as  in  Matera,  the  insufficient  level  of  international 
cooperation among the city’s cultural operators prior to the ECOC year. To remedy this, 
Plovdiv 2019 had a proactive approach. First, it implemented specific capacity building 
activities to encourage and help cultural operators to find European partners. Second, it 
identified  suitable  topics  in  the  cultural  programme  that  would  help  stimulate  the 
European  dimension,  such  as  –  in  particular  –  the  30th  anniversary  of  the  fall  of  the 
Berlin,  which  has,  of  course,  important  connotations  for  all  the  countries  in  Southeast 
Europe as it marked the end of communist regimes in the region, but also a very strong 
Europe resonance across the whole European Union.  Finally, it embedded the European 
dimension in the assessment criteria of the open calls for project proposals: if a project 

26   Il Sicomoro. Official website. Accessed from: http://ilsicomoro.net/progetti/silent-academy/ 

15 

 
                                                           
stimulated  local,  national  or  international  partnerships  with  cultural  operators  and 
organisations in the Balkans and Europe, their application would receive a higher score. 

As  a  result,  and  as  further  developed  in  sub-section  5.3  below,  there  were  four  times 
more  international  cooperation  arrangements  and  co-productions  in  2019  than  in  the 
baseline year of 2017, which the evaluator seems as a very promising achievement.  

Finally, Plovdiv 2019 also worked with Matera 2019, especially on communication (with 
joint presentations and dedicated info points established in the two cities to promote the 
cultural  activities  of  the  other  ECOC),  exchange  of  volunteers  and  six  joint  cultural 
projects  (including  for  the  opening  ceremonies)  as  a  way  to  reinforce  the  links  and 
visibility of the two 2019 ECOC title-holders. 

The “City and Citizens” dimension in Plovdiv 

Stimulating access and participation in cultural events was a strong element of the ECOC 
project in Plovdiv, in line with the “Together” motto of Plovdiv 2019, inspired by the co-
existence of different communities.  

According  to  the  external  evaluation,  a  positive  element  in  this  respect  was  that  the 
original application resulted from consultations with citizens at a rather early stage, with 
the  team  having  numerous  meetings,  discussions  and  focus  groups  with  local 
organisations,  institutions,  cultural  communities  and  individual  citizens  from  2013 
onwards.  

As  a  consequence,  the  cultural  activities  developed  all  along  the  years  were  related  to 
topics that were of interest for the local citizens and communities, such as the heritage of 
ethnic minorities and urban regeneration issues. 

Activities  implemented  all  along  the  year  were  also  targeted  at  different  groups  of 
society, including children and youth (such as the “Listen to Us  – Artistic Intelligence” 
project – which included an educational programme aimed to get the children in schools 
and  kindergartens  acquainted  with  contemporary  art  and  stimulate  their  creative 
thinking),  people  with  disabilities,  elderly  people  and  minority  groups  (such  as  the 
“GetToStolipinovo”  project,  which  presented  this  Roma  populated  neighbourhood  as 
“the place to go” and included in particular workshops and seminars to have discussions 
with the inhabitants on youth field work and ways to solve community problems). In this 
regard,  it  is  worth  mentioning  that  citizen  involvement  was  a  key  aspect  of  the  Fuse 
platform and more specifically of two of its clusters, i.e. “Mahala”, which focused on the 
integration  of  Roma  and  Turkish  communities,  and  “RegionalE”,  targeted  at  people 
living in the small villages and towns of the entire Plovdiv region.  

With  regard  to  financial  accessibility  of  the  programme  for  citizens,  it  is  important  to 
note that 61% of the ECOC events in 2019 (i.e. 313 out of 513 public events in total) had 
free  entrance,  against  only  10%  in  the  baseline  year  2017,  which  is  also  a  positive 
development. 

4.2.  Efficiency 

As was also systematically the case in previous ex post evaluations, it appears from the 
report of the contractor that overall, the ECOC remains an efficient EU Action, providing 
good  levels  of  benefits  for  the  EU  for  relatively  little  EU  investment.  Indeed,  the  only 
direct contribution from the European Union, in the form of the €1,5m Melina Mercouri 
prize awarded to each ECOC, is dwarfed by the total amount of money invested by the 
host  cities  in  designing  and  delivering  the  ECOC  (the  operational  budgets  were 
approximately €54.8m for Matera and €10.0m for Plovdiv). 

16 

 
Without  the  initial  EU  impetus  to  support  the  ECOC,  it  is  unlikely  that  the  host  cities 
would have invested anywhere near the amount of funding they did in  connection with 
the ECOC title. The possibility of securing the title typically stimulates cities  – but also 
their  respective  regional  /  national  public  authorities  and  private  partners  –  to  invest 
much more heavily in culture than they would in the absence of the ECOC. This is true in 
terms  of  both  infrastructure  and  expenditure  in  cultural  events  and  activities.  The  final 
report  of  the  contract  supporting  the  evaluation  concludes  that  this  means  the  Action 
remains highly efficient in terms of returns from the Melina Mercouri prize.  

As  far  as  the  efficiency  of  management  arrangements  is  concerned,  both  ECOC 
encountered some challenges, which they however ultimately overcome to a good extent. 

After it was created in 2014, the Matera 2019 Foundation went through an ‘interrupted 
rhythm’ of work, mainly due to the lack of continuity in local government resulting from 
the  2015  municipal  elections.  This  impasse  was  unblocked  around  the  time  of  the  first 
monitoring  meeting  in  October  2016  in  Brussels,  and  the  process  accelerated  in  the 
following  months,  with  the  recruitment  of  most  vacant  key  team  positions  throughout 
2017.  Despite  the  initial  political  instability  that  characterised  part  of  Matera’s  ECOC 
experience, the Foundation could finally count on support at local, regional and national 
levels. However, according to the evaluator, the delays in finalising the recruitment of the 
Foundation  staff  (completed  only  few  months  before  2019)  hindered  the  timely 
preparation  of  some  activities  and  events.  Moreover,  structuring  the  Foundation  as  a 
public entity increased the administrative and bureaucratic burdens of its activities, while 
according  to  key  stakeholders,  it  is  necessary  to  have  an  agile  structure  able  to  act 
immediately  and  respond  to  changing  circumstances  in  order  to  work  with  local  and 
international cultural operators. 

As  for  the  Plovdiv  2019  Foundation,  there  is  a  consensus  among  the  stakeholders 
interviewed that it was very effective in implementing the cultural programme. However, 
the overall oversight of the Foundation generated a somewhat more mixed set of views. 
In  particular,  the  snap  parliamentary  elections  in  Bulgaria  in  2014  were  followed  by 
several  rounds  of  cabinet  reshuffle,  including  three  changes  of  Ministers  of  Culture.  In 
addition, two rounds of local elections took place in Plovdiv in 2015 and in 2019, which 
led to  changes in  key personnel  in  the Board of  Directors during the preparation phase 
and the title year. Throughout the preparation and implementation phases, the Board of 
Directors insisted on closely monitoring the activities of the Foundation through weekly 
meetings  and  regular  reporting,  which  added  to  the  workload  and  delayed  procedures 
within the executive team. 

The  EU-level  selection  procedure  ensured  the  designation  of  two  ECOC  that  were 
relevant to the objectives at EU level and that had the potential to achieve the intended 
effects.  The  procedure  ensured  healthy  competition  in  both  countries,  with  a  record 
number of 21 bidding cities in Italy and a high number (i.e. eight) of cities competing for 
the first ever ECOC title in Bulgaria.  

Finally,  the  EU-level  monitoring  procedure,  with  formal  and  informal  monitoring 
meetings  as well as in  situ visits  with  the panel,  has proven a very  valuable process  in 
giving  impartial  and  professional  advice  and  support  to  the  two  ECOC  from  highly 
experienced experts, some of whom have implemented previous ECOC and encountered 
difficulties similar to the ones faced by Matera and Plovdiv.  

4.3.  Effectiveness 

17 

 
Ultimately, the effectiveness of the ECOC in achieving the objectives set for them (at EU 
level and local level) can only be determined in the long-run. That being said, the final 
report of the contract supporting the evaluation shows that the two ECOC 2019 presented 
cultural  programmes  that  were  more  extensive,  diverse,  innovative  and  international 
compared to the baseline cultural offering in previous years.  

In total, the ECOC cultural programme in Matera was ambitious in its scale and focus. It 
featured more than 1,300 events of various sizes and formats, representing a whole range 
of  cultural  disciplines,  ranging  from  opera  to  circus  shows,  from  workshops  on  digital 
tools  to  exhibitions  focusing  on  traditions  and  cultural  heritage  elements  of  the  local 
community, with many of them having a strong co-design and co-creation component. It 
is estimated that 80% of these events were original creations. This diversity was stressed 
by the respondents of the Foundation’s survey, where the majority reported satisfaction 
with the content, quality, and originality of the cultural programme (64%) and with the 
diversity of the cultural offer (66%). Moreover, a large majority of respondents believed 
that Matera is now more culturally vibrant (78%), more culturally diversified (65%) and 
more open to cultural differences and diversity more in general (82%).  

In  Plovdiv,  despite  the  decrease  in  the  overall  operational  budget  for  the  ECOC  2019 
project,  a  total  of  513  cultural  events  took  place  during  the  title  year.  Some  of  the 
highlights in the ECOC programme were large-scale, ambitious productions, which were 
praised by audiences and critics, and received several artistic awards. On top of that, 54 
ECOC related events were held in other cities in Bulgaria and abroad, which was – in the 
opinion  of  the  evaluator  –  a  very  interesting  way  to  disseminate  and  increase  the 
visibility of the ECOC 2019 beyond the city and its surrounding region. To achieve this 
in a challenging financial context, the Foundation proceeded to a costs re-evaluation for 
each  project  and  –  for  the  projects  resulting  from  open  calls  –  transferred  the 
responsibility  for  attracting  sponsors  to  applicants  through  the  requirement  for  self-
financing. Compared to  previous years, the content of the cultural programme was also 
very  diverse  and  ranged  from  standard  art  forms  like  gallery  and  museum  exhibitions, 
music festivals and street art to more innovative content such as ecological interventions 
and co-creation initiatives with disadvantaged groups. The general view is that the ECOC 
project mainly benefitted the city’s larger, well-established cultural operators such as the 
State  Opera,  the  Drama  Theatre  and  the  State  Puppet  Theatre,  but  it  was  also 
instrumental in shedding light on a number of emerging young artists and smaller scale 
organisations. 

As  described  below,  both  ECOC  helped  widen  access  to  and  participation  in  culture. 
Both  ECOC  also  helped  strengthen  the  cultural  capacity  of  their  cultural  and  creative 
sectors as  well as their links with  other sectors. Finally, both  ECOC widened access  to 
and participation in culture during 2019.  

Effectiveness in delivering a cultural programme with a European dimension 

The  Matera  cultural  programme  was  of  the  scale  and  quality  proposed  in  the  original 
application  and  had  a  genuine  European  dimension.  According  to  the  evaluation,  the 
ECOC  project  has  been  successful  in  contributing  to  making  the  cultural  offering  of 
Matera  more  European.  Indeed,  as  already  mentioned  above,  the  cultural  programme 
featured  performances  by  a  diversity  of  international  artists  and  exhibitions  of 
international  works,  strengthened  European  networks  and  connections,  European  co-
productions and residencies by European artists.  

The performance of Plovdiv in this respect is also good, as 80 events with a cross-border 
dimension took place in 2019, which is four times higher than the number of such events 

18 

 
during  the  baseline  year  2017.  Over  130  partnerships  with  creative  organisations  and 
artists  from  Europe  were  established  over  the  ECOC  year.  Plovdiv  also  stimulated  the 
European  dimension  through  cross-border  collaborations  on  topics  of  European 
importance and the active involvement of the EUNIC27 members in the country.  

These  endeavours  and  activities  in  both  cities  have  contributed  to  reinforcing  the 
transnational functioning of the local cultural community, with also scope for continuity 
after the title-year.  

Effectiveness  in  reaching and engaging  with local  citizens, in  targeting  specific groups 
and in strengthening the local cultural and creative sectors 

Both ECOC widened access to and participation in culture during 2019.  

In  Matera,  65%  of  the  1,300  events  of  the  programme  were  completely  free  and  the 
remaining (about 450) were accessible with the purchase of the Matera 2019 Passport28. 
Free  events  managed  to  gather  about  350,000  people,  while  events  accessible  with  the 
Passport  were  attended  by  over  140,000  participants.  Flagship  events  included  the 
opening ceremony with  an attendance estimated at 60,000 people or the Ars Excavandi 
exhibition29 with more than 17,500 visitors. According to the evaluator, such figures can 
be seen as a positive outcome, as the citizens of Matera were not used to having such a 
wide  cultural  offer  in  their  city  (which  is  further  reflected  in  the  lack  of  theatres  or 
venues that could host large exhibitions prior to the ECOC year).  

The  involvement  of  young  people  was  also  quite  important,  as  approximately  30,000 
students  were  involved  in  projects  realised  in  collaboration  with  local  and  national 
schools, as a result of an active collaboration between the Matera 2019 Foundation and 
the  Italian  Ministry  for  Education.  Finally,  almost  600  volunteers  (out  of  the  1,500 
registered)  took  an  active  part  in  Matera  2019.  It  is  worth  noticing  that  about  95%  of 
those  volunteers  who  responded  to  a  survey  carried  out  by  the  Foundation30  said  they 
were  satisfied  with  their  participation  in  the  ECOC  project  and  recognised  personal 
growth  in  relation  to  relation  to  the  interaction  with  citizens,  teamwork  or  self-
confidence. 

Perhaps equally impressive is the fact that according to estimations, about 57,000 citizens 
were directly involved in cultural productions. The co-creation process and the capacity 
building programme put in place by the Foundation, bringing together cultural operators, 
citizens and volunteers, gave citizens the opportunity to contribute to  about 70% of the 
cultural programme of Matera 2019. 

As  far  as  Plovdiv  2019  is  concerned,  the  cultural  programme  attracted  high  attendance 
levels,  with  1,528,432  people  attending  ECOC  events  in  Plovdiv  and  the  wider  South 
Central region in 2019. Out this total, 80% were national visitors, 11% were residents of 
Plovdiv and about 9% were international visitors. The share of those who visit relatively 

27   EUNIC  stands  for  “European  Union  National  Institutes  for  Culture”.  It  is  a  European  network  of 

organisations engaging in cultural relations from all EU member states. 

28   The  Passport  was  a  unique  ticket  giving  access  to  the  whole  panorama  of  cultural  activities  taking 

place in the context of Matera 2019. Some 74,000 passports were sold. 

29   Under  the  “Utopias  and  Dystopias”,  it  looked  at  the  history  and  culture  of  subterranean  architecture 

from the Palaeolithic period to the present and towards the future. 

30   A total of 172 answers were collected. 

19 

 
                                                           
often cultural events in  Plovdiv has increased from 27% in 2017 to 44% in 2019 while 
respectively  60%  of  the  city  residents  and  32%  of  the  wider  region’s  residents  have 
attended at least one cultural event in the title-year (against targets of respectively 50% 
and 30%)31. 

The  social  dimension  of  certain  projects  made  culture  accessible  for  social  groups  that 
previously  were  not  active  participants  in  the  cultural  life.  These  groups  include  the 
young people from deprived neighbourhoods and elderly people from smaller towns and 
villages in the wider Plovdiv region, as already illustrated above. There was also a strong 
focus  on  the  Roma  community  (Stolipinovo  being  the  largest  Roma  district  in  the 
Balkans with a population of about 80,000 people). A total of 61 events took place in this 
district, throughout the ECOC year, compared to just 10 events in the baseline year 2017. 
A good example of enhanced participation, widely regarded as one of the most successful 
projects within Plovdiv 2019, was Medea32, a theatre performance in which children from 
the  Roma,  Turkish,  Jewish  and  Armenian  ethnicities  in  Plovdiv  participated  as  actors. 
The  three-year  long  rehearsal  process  included  a  series  of  training  sessions  and  art 
workshops  for  75  children  in  Plovdiv  and  culminated  in  a  large-scale,  open-air 
performance at the Ancient Theatre in June 2019 with an audience of 1,753 people. 

Finally,  Plovdiv  2019  also  counted  on  the  active  involvement  of  volunteers,  with  400 
people  volunteering  over  the  ECOC  year  and  over  2,000  people  being  engaged  in 
volunteering activities over the entire period of the initiative. 

Both ECOC also helped strengthen the cultural capacity of the local cultural and creative 
sectors and their links with other sectors.  

As far as Matera is concerned, one of the challenges identified at the bidding stage was 
the lack of cultural infrastructure. The ECOC project was instrumental in mapping about 
400  venues  and  spaces  that  could  be  used  for  cultural  events  as  well  as  launching  and 
driving the process for the renovation of a historic location abandoned until recently, and 
its  repurposing  into  a  multifunctional  space.  Activities  were  also  implemented  to  help 
local  cultural  operators  developing  new  skills  and  strengthening  their  ability  to  seek 
partnerships and operate internationally33.  

In  Plovdiv,  the  local  cultural  capacity  was  also  reinforced  by  an  improved  cultural 
infrastructure,  most  notably  in  the  Kapana  Creative  District34,  which  was  a  flagship 
project of Plovdiv 2019 and has turned into one of the most recognisable city landmarks, 
attracting citizens and tourists alike. In the period from 2014 to 2019, and as a result of 
annual thematic open calls launched by the Foundation with the view to developing and 
supporting  creative  spaces  in  the  district,  more  than  60  projects  were  implemented  and 
over  300  events  took  place  in  Kapana.  Furthermore,  the  Foundation  implemented 
FORUM  2019,  a  flagship  project  and  an  educational  platform  aimed  at  creating 
partnership networks, capacity building and audience development, in order to ensure the 

31   These are monitoring data collected by the Plovdiv 2019 Foundation. 

32   https://plovdiv2019.eu/en/platform/fuse/121-mahala/482-medea 

33   In  particular,  about  27  project  leaders  and  40  cultural  operators  (defined  as  ‘makers’  and  ‘linkers’) 
participated  in  ‘Build-up’  activities,  among  which  was  a  10-day  workshop  covering  topics  ranging 
from effective collaboration to overcoming State assistance dependency, entrepreneurship and setting 
up international networking. 

34   https://plovdiv2019.eu/en/platform/transforms/130-urban-dreams/418-kapana-creative-district 

20 

 
                                                           
smooth realisation of the ECOC. Throughout the five-year period of implementation, 60 
workshops and training events took place, covering a variety of themes and topics, with a 
total of 4,900 participants. 

Effectiveness in reaching out to European and international audiences  

According to the evaluation, the ECOC raised the international profile of both 2019 title-
holding cities. 

As  far  as  Matera  is  concerned,  it  appears  that  the  Foundation  implemented  a  solid 
communication and marketing strategy with emphasis on digital communication, and that 
the  approach  developed  to  reach  local,  national  and  international  audiences  positively 
contributed  to  raising  awareness  and  informing  all  relevant  audiences  of  the  ECOC 
programme and its different initiatives. Key stakeholders interviewed also underlined the 
development of a digital team at an early stage (i.e. in 2013, before the city even got the 
title) and the role of the digital  volunteers (over 100 in  2019) as key  success  factors in 
this respect. On a more traditional note, a specific partnership with Euronews35, resulting 
in a series of 10 episodes on Matera 2019, also contributed to increasing the awareness of 
the  ECOC  year.  All  in  all,  Matera  2019  and  its  cultural  programme  was  discussed  in 
almost  58,000  articles  in  national  and  international  newspapers  and  journals  and  in 
approximately 1,300 radio and TV reports. 

Unfortunately, no data on the overall attendance  of foreign people in  the ECOC events 
have  been  collected.  However,  in  the  period  between  2012  and  2019,  the  Basilicata 
region experienced a higher increase in the number of tourists than the rest of Italy, from 
about  520,000  tourists  in  2012  to  almost  950,000  in  2019.  A  survey  conducted  among 
some  1,743  tourists  in  2019  shows  that  for  almost  70%  of  the  respondents,  Matera 
hosting the ECOC impacted on their decision to visit the city.  

The  evidence  suggests  that  Plovdiv  2019  achieved  significant  results  related  to  raising 
the  international  profile  of  the  city  in  two  main  directions,  namely  stimulating  the 
tourism  sector  and  attracting  unprecedented  international  media  attention.  While  the 
share of Bulgarians visiting Plovdiv per year increased from 28% in 2015 to over 39% in 
the  title  year,  the  number  of  international  tourists  rose  by  27%  from  2015  to  2019  to 
reach  a  total  of  121,478  people36.  However,  as  the  National  Statistical  Institute  of 
Bulgaria tracks only tourist accommodation establishments with 10 and more bed places, 
this figure is only a rough estimation.  

The ECOC also succeeded in creating an unprecedented international media coverage. In 
the  period  from  2017  to  2019,  over  800  publications  appeared  in  international  media, 
among  which  CNN  Travel,  Lonely  Planet,  The  New  York  Times,  Euronews  (with 
promotional  videos  in  English,  French,  German,  Italian,  Spanish,  Greek,  Portuguese, 
Hungarian  and  Russian  languages),  La  Repubblica,  Daily  Telegraph,  Deutsche  Welle, 
RAI  TV,  France-Presse,  Reuters,  TASS, the Chinese agency  Xinhua, Associated Press, 
and many more. The large number of travel related articles helped raise the visibility and 
attractiveness of the city for international tourists. For example, Plovdiv was named one 

35   Complemented  with  important  partnerships  with  the  national  radiobroadcaster  RAI  Radio  3  and  a 

regional TV channel. 

36   Monitoring data from the Foundation. 

21 

 
                                                           
of CNN Travel’s best places to visit in 201937 whereas, in celebration of the ECOC 2019, 
National Geographic featured a six-series documentary film dedicated to Plovdiv38. 

Effectiveness in ensuring sustainability.  

Positive impacts are also expected with regard to the legacy of the ECOC in both cities, 
as significant efforts were to keep the momentum going after 2019.  

From the very  start,  Matera 2019 intended to start a process  of renovation and cultural 
change  of  the  city  and  the  local  community  that  would  last  well  beyond  2019.  It  was 
planned that the Foundation  would carry on its activities for two  years after the ECOC 
year,  becoming  a  co-creation  platform,  and  a  budget  has  been  allocated  to  this,  though 
the legacy plans were greatly impacted by the COVID-19 outbreak. 

As  far  as  Plovdiv  is  concerned,  it  is  the  view  of  the  evaluator  that  a  positive  aspect 
related to ensuring the legacy of the initiative, which sets Plovdiv apart from most other 
ECOC  cities,  was  the  decision  of  the  Foundation  to  reserve  funding  from  the  Melina 
Mercouri Prize for projects in the years following the title. At the end of 2019 and, again, 
at  the  end  of  2020,  two  “legacy”  open  calls  were  published,  which  were  aimed  at 
sustainable  development  of  cultural  initiatives  and  events  in  the  city  and  intended  to 
provide  a  continuation  of  the  positive  changes  brought  by  the  ECOC.  However,  as  in 
Matera, there is a considerable degree of uncertainty surrounding the implementation of 
approved projects due to the COVID-19 pandemic.  

4.4.  Coherence  

The ECOC Action is coherent and complementary to the Creative Europe Programme in 
that it promotes the objectives of Creative Europe and is distinct from the other activities 
supported  by  the  programme.  Plovdiv  was  involved  in  the  Creative  Europe-funded 
project,  European  Academy  of  Outdoor  Arts  School  of  Spectacle,  with  leading 
organization Walk the Plank (UK), Pafos ECOC 2017 (Cyprus), a partner from Kaunas 
(Lithuania)  and  two  from  Limerick  (Ireland)  aiming  to  give  creative  practitioners  from 
Bulgaria  and  abroad  a  better  understanding  of  the  creative  possibilities  of  outdoor  art 
celebrations,  gaining  skills  in  production,  sound,  performance,  3D  mapping  and  site 
decoration. 

The  ECOC  Action  is  also  complementary  to  other  EU  programmes.  As  an  example, 
Matera  2019  is  the  coordinator  of  an  Erasmus+  project  named  DeuS  (European  Open 
Design  School  for  Sustainable  Regional  Development).  In  collaboration  with  other 
partners  from  eight  other  countries,  the  project  aims  to  co-create  a  European-wide 
learning  and  training  approach  in  design,  critical  thinking  and  entrepreneurship  to  find 
participatory,  creative and cost-effective solutions  to  local  challenges, by unlocking the 
potential of the cultural and creative sectors39. 

The ECOC Action is  also  coherent  with  and  complementary to  the European Cohesion 
Policy Funds, depending on the context of each city holding the title. As an illustration, 
in  cooperation  with  Leeuwarden  (ECOC  2018),  Aarhus  (ECOC  2017)  and  other 
European  partners,  Matera  implemented  the  ‘Night  Light’  Interreg  project,  aiming  at 

37   https://edition.cnn.com/travel/article/places-to-visit-2019/index.html 

38   https://www.youtube.com/watch?v=d4Caj7Mvrno 

39   DeuS. Official website. Accessed from: https://www.deuscci.eu/  

22 

 
                                                           
bringing  together  key  lessons  and  practices  experienced  in  different  European  contexts 
related  to  reducing  light  pollution.  This  Interreg  project  was  connected  to  two  other 
events  of the Matera 2019 cultural  programme,  namely Social  Lights  and Onda,  which 
were built around the preservation of the night sky and astronomy. 

4.5.  EU added value 

As  illustrated  above,  the  ECOC  Action  achieved  an  impact  that  would  not  have  arisen 
through the actions of Member States alone. 

The designation of Matera and Plovdiv as ECOC attracted benefits that would have been 
unlikely  to  arise  to  the  same  extent  in  the  absence  of  the  Action.  Indeed,  the  evidence 
presented  in  the  chapters  above  suggest  that  the  ECOC,  as  an  EU  initiative  and  an  EU 
brand, provides the stimulus for stakeholders to commit resources and effort to a shared 
vision  and  collaborative  programme  at  a  much  greater  scale  than  would  otherwise 
happen. The EU value added consists largely in the force of the 'brand' itself to act as a 
significant  generator  of  interest  from  stakeholders  not  only  from  the  city  and  but  also 
from far beyond. 

This is evidenced, for example, by the creation of dedicated organisations to implement 
the ECOC and the allocation of resources that are additional to the “mainstream” funding 
for cultural activities in the cities and regions concerned. 

More  particularly,  the  internationalisation  of  the  activities  and  the  wide  horizon  of  the 
programmes in both cities would have been of much lower scale if it had been a national 
action. Also, the European nature of the action represented the stimulus for the title cities 
to  look  for  partnerships  abroad  and,  most  importantly,  it  acted  as  a  link  with  other  EU 
initiatives and cities. 

Regarding the visibility of the EU, both title-holders gave full prominence to the title of 
“European  Capital  of  Culture”  in  their  communication  and  promotional  materials  and 
gave prominence to the fact that the ECOC is an EU Action.  

The  results  consistently  achieved  by  the  Action  through  the  years  make  sure  that  its 
models  and  mechanisms  –  for  example  in  terms  of  building  a  solid  governance  for  the 
delivery of the ECOC, increasing the capacity of local cultural organizations or attracting 
projects'  ideas  from  local  residents  –  can  be  applied  not  only  within  one  individual 
Member  State  that  has  its  administrative  and  cultural  specificities  but  in  any  Member 
States, finally confirming its EU added value.  

5.  CONCLUSIONS 

The Commission concludes that the ex post evaluation of the two ECOC 2019 confirms 
what  already  emerged  from  all  previous  twelve  yearly  ECOC  evaluation  exercises,  i.e. 
that the ECOC Action is highly valued by the hosting cities, which can obtain positive 
impacts during the development and implementation of their title years. The action also 
remains relevant at EU level. 

The Commission also concludes that the programmes implemented by the two 2019 title-
holders were consistent with the objectives of the ECOC Action: 

-  They reflected its European dimension: as an example, in Matera 45% out of the 
total  number  of  artists  and  project  leaders  involved  by  the  Matera  2019 
Foundation, were international and over 1,000 international mobility programmes 
were realised with 55 different countries. In Plovdiv, 80 events had a cross-border 

23 

 
dimension  in  2019,  which  is  four  times  higher  than  the  number  of  such  events 
during the baseline year 2017; 

-  They involved local residents and stakeholders: in Matera, about 57,000 citizens 
were  directly  involved  in  cultural  productions  through  an  unprecedented  co-
creation  process  whilst  in  Plovdiv,  60%  of  the  city’s  residents  and  32%  of  the 
wider region’s residents have attended at least one cultural event in the title-year; 
-  They  widened  access  to  and  participation  in  culture  during  2019:  in  Plovdiv, 
stimulating access and participation in cultural events was a strong element of the 
ECOC  project  as  reflected  in  high  attendance  levels  (with  a  total  of  1,528,432 
people);  in  Matera,  audiences  reached  500,000  people  (in  line  with  the 
expectations)  and  the  ECOC  was  successful  in  providing  new  opportunities  for 
citizens to be involved as creators, performers and audiences; 

-  They also helped strengthen the cultural capacity of the local community and/or 
cultural and creative sectors and their links with other sectors: for example, 4,900 
participants  took  place  in  the  educational  platform  run  by  the  Plovdiv  2019 
Foundation during its  five  years of implementation  while in  Matera, suffering  a 
lack  of  cultural  infrastructures,  the  ECOC  project  was  instrumental  in  mapping 
about 400 venues and spaces that could be used for cultural events; 

-  The ECOC raised the international profile of both cities: In Plovdiv, about 9% of 
people  attending  ECOC  related  events  were  international  visitors  whilst  Matera 
2019  and  its  cultural  programme  was  discussed  in  almost  58,000  articles  in 
national  and  international  newspapers  and  journals  and  in  approximately  1,300 
radio and TV reports; 

-  Both  ECOC-years  may  lead  to  legacies  both  physical  (in  the  form  of  new  or 
refurbished  cultural  and  logistic  infrastructures)  and  intangible  (for  example 
through the new skills developed by the high number of local people who actively 
developed  cultural  projects  in  Matera  or  participated  in  capacity-building 
activities in Plovdiv). 

However, as was the case with all previous ex post evaluations of the ECOC Action, the 
current  evaluation  –  because  it  comes  too  early  after  the  implementation  of  the  ECOC 
year  in  accordance  with  Article  12  of  Decision  No  1622/2006/EC  –  cannot  assess  the 
long-term  impacts  of  the  two  ECOC  2018.  As  already  mentioned  in  the  staff-working 
document accompanying its report on the ECOC 2018, the intention of the Commission 
is  to  address  this  shortcoming  to  analyse  such  impacts  in  the  frame  of  the  evaluation 
exercise foreseen in Article 16 of Decision No 2014/445/EU. 

24 

 
ANNEX 1: PROCEDURAL INFORMATION 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

The  evaluation  was  led  by  the  European  Commission’s  Directorate  General  for 
Education, Youth, Sport and Culture (DG EAC). It is included in the Work Programme 
of  Creative  Europe  for  2019  and  in  the  Agenda  Planning  with  the  reference 
PLAN/2019/5483. 

2.  ORGANISATION AND TIMING 

The evaluation was supported by an external and independent evaluator, under a service 
contract.  The  service  contract  was  implemented  via  a  Framework  Contract  with 
reopening  of  competition  and  in  accordance  to  the  Financial  Rules  Applicable  to  the 
General Budget of the Union40 and its Rules of Application41. 
The evaluation roadmap was adopted on 27 May 201942. 

According to the roadmap, a Steering Committee including staff representatives from the 
European  Commission’s  Directorate  General  for  Education,  Youth,  Sport  and  Culture, 
Directorate General for Communication and Secretariat General was established in May 
2019. The Steering Committee met in four occasions: to prepare the Terms of Reference, 
to kick off the evaluation, to approve the Inception Report and to discuss the draft final 
report. Extensive correspondence between the Steering Committee members was held in 
between the meetings to follow-up on the evaluation. 

3.  EXCEPTION TO THE BETTER REGULATION GUIDELINES 

The only exception is that the evaluation does not include a wider public consultation. As 
explained  in  the  Roadmap,  the  ECOC  Action  is  considered  to  be  local.  International 
participation is scattered within and outside Europe and is difficult to reach. On the other 
hand the opinions would be based on attendance to  specific  events  and  would  not  give 
useful insights for the evaluation of the ECOC Action as a whole. 

4.  EVIDENCE, SOURCES AND QUALITY 

The 2019 evaluation of the ECOC used a series of data sets to inform its findings. The 
main ones being: 

- 

Interviews  with  over  60  stakeholders  directly  and  indirectly  involved  in  the 
planning, operation and delivery of the two ECOC programmes; 

-  A literature review of ECOC and European level information on the two ECOC 
including  application/bid  information,  reports,  cultural  programme  brochures, 
web sites and news articles. The ECOC's own external evaluations have also been 

40   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R0966-20160101&from=EN 

41   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R1268-20160101&from=EN 

42   https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-3143701_en 

25 

 
 
                                                           
 
used to inform the European evaluation process providing access, among others, 
to the results of wide consultation activities; 

-  Together,  the  above  evidence  base  provides  the  evaluation  with  a  valid  and 
rounded  set  of  data  to  inform  the  views  on  the  main  aspects  of  the  ECOC 
evaluation  including  efficiency,  effectiveness,  sustainability  and  relevance.  This 
view has been informed by: 
  The scale of the  consultation  exercise. Around 60  individuals have  fed their 
views and opinions into the evaluation process through telephone interviews/ 
video conference; 

  The nature of the consultation exercise. The evaluators were keen to consult 
with  those  who  had  a  more  indirect  and  external  view  of  the  two  ECOC. 
These  stakeholders  included  journalists,  local  authorities,  representatives  of 
the  business  sector,  those  not  directly  benefitting  from  the  ECOC  (e.g. 
rejected projects) as well as those working in the wider cultural policy agenda 
at regional and city level. Furthermore, the evaluators consulted organisations 
representing  local  communities  and  minorities  (i.e.  Roma,  LGBT  NGOs  , 
etc.). This ensures the evaluation is not simply based on those who benefitted 
the most from the ECOC; 

  Triangulation  of  quantitative  and  qualitative  data  collected  through  primary 

and secondary research. 

26 

 
ANNEX 2: STAKEHOLDER CONSULTATION 

The stakeholders were consulted via targeted consultations (phone interviews).  

The consultations included the teams responsible for the implementation of the ECOC in 
both cities, the political stakeholders involved in the project, the projects participating in 
the programme or having submitted proposals to participate that were rejected, as well as 
personalities  attached  to  the  cultural  tissue  in  both  cities  and  stakeholders  at  EU  level. 
The  objective  of  the  consultations  was  to  have  evidence  supporting  the  findings  and 
conclusions of the evaluation.  

Due  to  the  COVID-19  outbreak  in  late  February-early  March,  the  methodology  was 
revised  and  initially  planned  visits  to  both  cities  did  not  take  place  and  face-to-face 
interviews  were  replaced  by  remote  ones  and  online  meetings  with  stakeholders.  As 
agreed with the Steering Committee, for safety reasons interviews were conducted over 
the telephone, Skype or Microsoft Teams. 

The  interview  lists  in  Matera  and  Plovdiv,  as  well  as  the  topic  guides  for  the  various 
types of interviews can be found in the Annexes 1 and 2 of the report produced by the 
experts  assisting  the  Commission43.  The  list  of  stakeholders  consulted  was  partly 
developed by the ECOC delivery teams but also through an internet search for relevant 
stakeholders.  

For  the  reasons  already  mentioned  in  Chapter  4  of  the  staff-working  document  and  in 
Annex 1, no open public consultation was conducted in the framework of the evaluation 
exercise. The consultation was restricted to relevant stakeholders in the two cities hosting 
the title. The characteristics of the Action and the scope of the evaluation do not make it 
necessary  to  extend  the  consultation  to  a  wider  public  (as  indicated  in  the  roadmap 
published). 

Annex  3  details  the  evaluation  questions  for  which  the  stakeholders'  consultation  was 
used.  These  regard  mainly  the  questions  which  answers  are  not  based  on  factual  data. 
The  stakeholders'  consultation  was  particularly  useful  to  find  information  about  the 
impact of the ECOC in the cultural offer of the city, the participation of citizens and local 
cultural  operators,  the  building  of  capacity  for  local  cultural  operators  and  legacy 
prospects.  

43   https://op.europa.eu/en/publication-detail/-/publication/6312a17a-1b6a-11ea-8c1f-

01aa75ed71a1/language-en 

27 

 
 
 
                                                           
 
ANNEX 3: METHODS AND ANALYTICAL MODELS 

The figure below presents the overview of the methodology. A more detailed overview 
of the methodology and sources used for each of the evaluation questions is presented in 
the subsequent tables and text. 

1. 

 Overview of methodology and tasks of the evaluation: 

Inception phase 

Task 1: Kick-off meeting  

Task 2: Initial consultations 

Task 3: Desk research  

Task 4: Inception Report  

Main Research phase 

Task 5: Visits to the ECOC cities 

Task 6: On-line survey of project 

Analysis and reporting phase 

Task 7: Interim Report 

Task 8: Factual check of the city reports 

Task 9: Final report 

Table 1.1  Evaluation questions: Relevance, EU added value and coherence 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ1: To what extent are the objectives of the 
ECOC Action? 

What  was  the  main  motivation  behind  the  city 
bidding  to  become  a  European  Capital  of 
Culture? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

28 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

the  process  of  determining 
What  was 
objectives? 
there  a  process  of 
  Was 
consultation  in  each  city  to  define  aims  and 
objectives? 

What  were  the  objectives  of  the  city  in  being 
ECOC?  What  was  the  relative  importance  of 
each objective? 

To  what  extent  were  the  objectives  consistent 
with  the  Decision  and  with  the  ECOC's  own 
application?  (special  focus  on  the  European 
dimension)  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Have any specific objectives of the ECOC event 
been related to social impacts? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

In  this  connection,  did  the  objectives  of  the 
ECOC  event  include  reaching  out  to  all  groups 
of 
excluded, 
disadvantaged, disabled people and minorities? 

including 

society, 

the 

EQ2:  To  what  extent  were  the  ECOC's 
associated 
cultural 
activities relevant to their own objectives? 

programmes 

and 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

To  what  extent  were  the  activities  consistent 
with the ECOC's own objectives? (special focus 
on the European dimension) 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

extent 

To  what 
specific 
themes/orientations  of  the  cultural  programme 
proved to be relevant to the objectives defined? 

have 

the 

How  was  the  European  dimension  reflected  by 
the themes put forward by the ECOC event and 
in terms of cooperation at European level? How 
did  the  Capitals  of  Culture  seek  to  make  the 
European dimension visible? To what extent did 
the two ECOC cooperate? 

29 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 1.2 Evaluation questions: Effectiveness 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ3:  To  what  extent  were 
objectives achieved? 

the  EU-level 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Provide  typology  of  outputs,  results  and  possible 
impacts of the Action at different levels (European, 
national, regional etc.) 

To  what  extent  has 
the  ECOC  event  been 
successful in attaining the objectives of the Action 
(refer to list in the intervention logic)? 

Was the cultural programme perceived as being of 
high artistic quality? To what extent did the ECOC 
prove  successful  in  bringing  their  chosen  artistic 
themes/orientations to the fore? 

To what extent did the ECOC title contribute to an 
enhanced  cultural  offer  in  the  cities  holding  the 
title  (e.g.  in  terms  of  scope  and  scale)  with 
stronger European dimension? 

To  what  extent  did  the  ECOC  implementation 
widen access to and participation in culture in the 
two cities? What actions were taking to include the 
elderly,  young  people,  people  with  special  needs 
in the cultural activities? How accessible were the 
activities carried out? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

did 

programmes 

the  ECOC 

help 
How 
strengthening  the  capacity  of  the  cultural  and 
creative  sectors  and  its  links  with  other  sectors? 
Which  help  was  available  to  cultural  operators  to 
extend  their  networks  and  work  transnationally 
and internationally? 

Y 

Y 

Y 

Y 

Y 

Y 

To  what  extent  did  the  Action  in  the  two  cities 
raise their international profile through culture? 

EQ4:  To  what  extent  were  the  cities'  own 
objectives achieved? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

30 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

What  quantitative  indicators  (number  of  visitors, 
overnight  stays,  cultural  participation  of  people, 
etc.)  of  the  social,  tourist  and  broader  economic 
impacts  of  the  event  have  been  gathered  by  the 
ECOC?  

To what extent did the ECOC achieve the outputs 
hoped  for  by  the  city  and  as  set  out  in  the 
application?  

To what extent have specific objectives related to 
social impacts been met? 

To  what  extent  were  the  objectives  related  to 
reaching out to all groups of society, including the 
excluded, disadvantaged, disabled and minorities, 
met? 

EQ5: To what extent has the Action resulted in 
unintended effects? 

Are  there  any  instances  where  the  ECOC  event 
has  exceeded  initial  expectations?  What  positive 
effects has this had?  

Where  expectations  have  not  been  met,  what 
factors  have  hindered  the  development  of  the 
Action? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Have  any  other  unintended  effects  been 
identified? 

Y 

Y 

Y 

Y 

Y 

EQ6: To what extent can the positive effects of 
the  ECOC  Action  be  considered 
to  be 
sustainable? 

Which  of  the  activities  or  elements  of  the  ECOC 
event are likely to continue and in which form once 
the ECOC-year is over? 

Has  any  provision  been  made  to  continue  and 
follow  up  the  cultural  programme  of  the  ECOC 
event after the closure?  

How will the city continue to manage its long-term 
cultural development following the ECOC event? 

What  will  be  the  role  of  the  operational  structure 
after the end of the ECOC event and how will the 
organizational structure change? 

31 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

What  has  been  the  contribution  of  the  ECOC 
event 
improved  management  of  cultural 
development in the city? (in the medium-term) 

to 

What are the impacts of the ECOC event likely to 
be  on  the  long  term  cultural  development  of  the 
city?  
What are the impacts of the ECOC event likely to 
be  on  the  long  term  social  development  of  the 
city? 
What are the impacts of the ECOC event likely to 
be on  the  long term urban and broader economic 
development of the city? 

Table 1.3 Evaluation questions: Efficiency 

EQ7:  How  did  the  management  arrangements 
of each ECOC contribute to the achievement of 
outputs, results and impacts? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

s
w
e
i
v
e
r

e
r
u
t
a
r
e
t
i
L

a
t
a
D

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a

h
t
i

w

s
w
e
i
v
r
e
t
n

I

Y 

s
m
a
e
t
y
r
e
v
i
l
e
d

s
n
o
i
t
a
t
l
u
s
n
o
c

l

r
e
d
o
h
e
k
a
t
S
Y 

s
t
c
e
j
o
r
p
f
o
y
e
v
r
u
S

s
w
e
i
v
r
e
t
n

i

t
c
e
j
o
r
P

s
i
s
y
l
a
n
A

Y 

How  have  the  organizational  models  of  the 
formal  governing  Board  and  operational 
structures played a role in the European Capital 
of  Culture?  What  role  have  the  Board  and 
operational  structures  played  in  the  ECOC 
event's  implementation?  At  what  stage  were 
these  structures  established?  How  did 
it 
improve  management  of  culture  in  the  city 
during the event? 

Who  chaired  the  Board  and  what  was  his/her 
experience?  What  were  the  key  success  and 
failure elements related to the work of the Board 
and  operational  structure  used  and  personnel 
involved?  

Has  an  artistic  director  been  included  into  the 
operational  structure  and  how  was  he/she 
appointed?  What  were  the  key  success  and 
failure  elements  related  to  the  work  of  the 
artistic director and personnel involved? 

Y 

Y 

Y 

Y 

Y 

Y 

What  was 
programme? 

the  process  of  designing 

the 

Y 

Y 

Y 

Y 

Y 

32 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
How were activities selected and implemented? 

How  did  the  delivery  mechanism  contribute  to 
the achievement of outputs? 

to 

been 

strategy 

To  what  extent  has  the  communication  and 
successful 
promotion 
in/contributed 
the  promotion  of  city 
image/profile,  promotion  of  the  ECOC  event, 
awareness-raising  of  the  European  dimension, 
promotion  of  all  events  and  attractions  in  the 
city? 

To  what  extent  has  the  communication  and 
promotion  strategy  including  the  use  of  social 
the 
media 
communication's 
local, 
regional,  national,  European  and  international 
levels? 

target  groups  at 

successfully 

reached 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ8:  To  what  extent  were  the  selection, 
co-financing 
monitoring 
procedures, 
by  Decision 
2006/1622/2006/EC efficient? 

EU 
and 
introduced 

the  Commission 

To  what  extent  have  the  mechanisms  applied 
by 
line  with  Decision 
in 
2006/1622/EC for the selection of the European 
Capitals  of  Culture  and 
the  subsequent 
implementation  and  monitoring  mechanisms 
influenced the results of the ECOC event?    

To  what  extent  has  the  informal  meeting 
following  the  designation  as  well  as  other 
the 
advice  offered  by 
the 
Commission 
ECOC event?  

the  panel  and  by 
the  results  of 

influenced 

L

Y 

Y 

Y 

Y 

Y 

How was the Melina Mercouri Prize used? 

EQ9:  To what  extent  did the  ECOC  manage 
to raise the necessary resources?  

Y 

Y 

Y 

Y 

Y 

Y 

33 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

What was the process of securing the financial 
inputs? 

Y 

Y 

Y 

Y 

What  was  the  total  amount  of  resources  used 
for  each  ECOC  event?  What  was  the  final 
financial outturn of the year?  

Y 

Y 

Y 

What  were  the  sources  of  financing  and  the 
respective  importance  of  their  contribution  to 
the  total?  How  much  came  from  the  European 
Union Structural Funds (e.g. ERDF - European 
Regional Development Fund,  ESF  –  European 
Social Fund) or other sources of EU funding? 

To  what  extent  did  the  ECOC  title  trigger 
complementary sponsorship? 

What  was  the  total  expenditure  strictly  for  the 
implementation  of  the  cultural  programme  of 
the  year  (operational  expenditure)?  What  was 
the proportion of the operational expenditure in 
the total expenditure for the ECOC event?  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  proportion  of  expenditure  was  used  for 
infrastructure 
tourism 
infrastructure, including renovation)? 

(cultural 

and 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

EQ10: To what extent were the financial and 
human  resources  secured  by  each  ECOC 
appropriate and proportionate? 

Was  the  total  size  of  the  budget  sufficient  for 
reaching  a  critical  mass  in  terms  of  impacts? 
Could  the  same  results  have  been  achieved 
with less funding? Could the same results have 
been achieved if the structure of resources and 
their respective importance was different?  

To  what  extent  have  the  human  resources 
deployed for preparation and implementation of 
the  ECOC  event  been  commensurate  with  its 
intended outputs and outcomes?  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

34 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

be 

As a result, could the total budget for the ECOC 
event 
and 
proportional to what the each ECOC set out to 
achieve?  

appropriate 

considered 

Y 

Y 

Table 1.4 Evaluation Questions: Coherence 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ11:  To  what  extent  were 
complementary to other EU initiatives? 

the  ECOC 

Y 

Y 

Y 

Y 

As far as the conclusions made for the two cities 
allows it, to what extent has the Action proved to 
be  complementary  to  other  EU  initiatives  in  the 
field of culture? 

To what extent has each  ECOC been reinforced 
by and added  impetus to investments by the EU 
Structural Funds? 

To  what  extent  have 
complemented 
European Year of Cultural Heritage? 

other  EU 

the 

two  ECOC 
e.g. 

initiatives, 

EQ12:  What  is  the  EU  added  value  and  the 
visibility of the ECOC Action? 

As far as the conclusions made for the two cities 
allow,  what  is  the  added  value  of  the  European 
Capital  of  Culture  being  an  EU 
initiative, 
compared to what could be achieved if the Action 
was a purely national or local action? 

Was the fact that this is a European Union action 
sufficiently communicated by the cities?  

To  what  extent  were  the  general  public  and  the 
cultural operators aware of this fact? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

2.  Details of the relevant evaluation tasks and methodologies:  

35 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Inception phase 

The  aim  of  the  inception  phase  was  to  finalise  the  evaluation  framework  and  research 
tools, address the project steering group's comments regarding the proposed method and 
collect background information on the two 2019 host cities. 

Initial consultations   

The  task  consisted  in  talking  to  each  of  the  two  ECOC,  getting  evidence  about  the 
efficiency  and  effectiveness  of  processes  at  EU  level,  obtaining  qualitative  information 
on key issues for each ECOC to inform later research, identifying additional sources of 
data  and  additional  interviewees  at  EU,  local  and  national  levels,  as  well  as  informing 
proposals on monitoring and evaluation arrangements. Discussion at the kick off meeting 
identified  a  need  for  strong  triangulation  and  to  gain  opinions  from  a  broader  set  of 
stakeholders. 

Desk research 

This involved collection and review of literature, primary data from the delivery agency 
and  available  secondary  data,  as  well  as  web  and  social  media  statistics  in  order  to 
develop a descriptive picture of each ECOC and serve as a source of evidence for later 
analysis.  literature  collected  at  the  European  level  was  reviewed  and  assessed  to  either 
understand the policy drivers at EU level which support the two ECOC 2019 or provide 
background  on  the  bidding  and  application  stages  for  both  2019  cities  (i.e.  little  of  it 
provides information useful for the content, delivery and impact of the two programmes). 
More academic literature at the EU level linked to the ECOC was also searched for. To 
complement  the  above  desk  research  task  the  contractor  assessed  the  extent  to  which 
each  ECOC  had  used  big  data  as  well  as  analysed  web  and  social  media  to  increase 
visibility  and  interest  in  the  ECOC  among  country  residents  and  internationally.  There 
was however no evidence of big data having been used by the Foundation or any of the 
other stakeholders in Matera and Plovdiv. 

Visits to the cities  

In agreement with the Commission, due to the COVID-19 outbreak the visits to Matera 
and Plovdiv did not take place. Instead, there were various rounds of online meetings and 
interviews  with  representatives  of  the  Matera  2019  and  Plovdiv  2019  Foundations,  as 
well  as  with  cultural  operators  and  local  stakeholders.  The  feedback  provided  by  the 
interviewees  also  contributed  to  guide  and  tailor  the  desk  research,  by  identifying 
relevant and additional sources of evidence on the main subjects of the evaluation. These 
interviews also sought to gather an in-depth understanding of the effect that ECOC had 
for the individuals involved, their organizations and the local culture sector as a whole. It 
finally allowed to ensure that findings were based on consensus across a range of target 
groups as well as to identify key differences in the experience of different stakeholders. 
In  the  case  of  projects,  the  criteria  for  inclusion  focused  on  whether  it  represents  good 
practice in relation to at least one of the key dimensions of the evaluation, including (but 
not limited to) increased European cooperation, the effective targeting of key groups of 
citizens or neighbourhoods, audience development strategies, or legacy effects. 

Further interviews were conducted: 

36 

 
  with national and local  media representatives in order to  assess media strategies 

 

and impact on communication efforts; 
representatives  of  local  commerce,  including  businesses  in  the  tourism, 
hospitality and transport sectors as well as key commercial sponsors of ECOC. 

The interviews with the delivery teams served four purposes: i) gathering further factual 
data and information about the cultural programme and its achievements; ii) identifying 
the  "story"  of  the  ECOC  throughout  its  lifecycle,  i.e.  conception,  application, 
development, delivery, legacy; iii) gaining a critical (albeit "insider") perspective on the 
relevance,  efficiency,  effectiveness  and  sustainability  of  the  ECOC;  iv)  triangulating 
emerging  findings,  for  example,  those  emerging  from  the  desk  research  and  project 
survey. 

Interviews  with  other  key  stakeholders  in  the  city  (e.g.  municipality,  chamber  of 
commerce, commercial sponsors, media representatives and cultural operators) were also 
essential to identifying the ‘external’ view of the ECOC by those who were involved in 
the urban or cultural  agenda but  were not  necessarily involved in  the direct  delivery of 
the ECOC. As with the previous evaluations, such individuals offered an alternative and 
broader  perspective  on  the  ECOC.  Indeed,  the  interviews  allowed  the  evaluator  to 
explore particular issues in more depth, for example, relating to the effectiveness of the 
governance structure, or the strength of artistic direction. 

Analysis and reporting phase 

Once  all  the  information  gathered,  the  experts  fulfilled  an  analysis  of  the  available 
information,  drawing  conclusions  and  triangulating  data  to  ensure  consistency  and 
accuracy. The limitations of the data gathered have been explained in  the report and in 
the staff-working document. After the first drafting exercise, the results were shared with 
the  two  ECOC  for  a  factual  check.  Both  the  Matera  and  Plovdiv  ECOC  were  asked  to 
undertake  a  check  on  the  completeness  of  data  being  used  to  address  each  evaluation 
topic. Their comments, where relevant, were included in the final report submitted by the 
contractor supporting the evaluation. 

37