COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex Post evaluation of the 2015 European Capitals of Culture (Mons and Pilsen)
Date: 2017-04-27
Institutions: Directorate-General for Education, Youth, Sport and Culture, European Commission
Subjects: Belgium, Czechia, EU action, European cultural event, European symbol, capital city, project evaluation

EN 

   EN 

   EUROPEAN COMMISSION  Brussels, 27.4.2017  SWD(2017) 135 final   COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex Post evaluation of the 2015 European Capitals of Culture (Mons and Pilsen) {COM(2017) 193 final} {SWD(2017) 137 final}  
 
 
1. 

INTRODUCTION 

This  document  describes  the  methodology  and  findings  of  the  ex-post  evaluation  of  the 
European Capital of Culture event (ECoC) for the year 2015. Decision No 1622/2006/EC of 
the  European  Parliament  and  of  the  Council  of  24  October  2006  establishing  a  Community 
action for the European Capital of Culture event for the years 2007 to 20191 requires, in its 
Article 12, that the Commission ensures the external and independent evaluation of the results 
of  the  European  Capitals  of  Culture  from  the  previous  year.  The  purpose  of  the  yearly 
evaluation  exercise  is  to  establish  a  comprehensive  understanding  of  the  performance  and 
achievements  of  the  action  in  the  relevant  year,  as  well  as  to  draw  lessons  and 
recommendations for the future based on the experiences of the two host cities for that year. 

The scope of this evaluation study  covers the implementation  of the ECoC action for 2015, 
including the selection and monitoring procedures and the implementation of the action by the 
two  2015  ECoC  hosts:  Mons  in  Belgium  and  Pilsen  in  the  Czech  Republic.  The  evaluation 
investigated  how  these  two  cities  developed  their  respective  applications  and  cultural 
programmes, how they delivered their year, the benefits they achieved and any legacy issues 
they experienced. 

The  findings  of  the  evaluation  lead  to  lessons  for  the  future  implementation  of  the  ECoC 
action. 

2. 

BACKGROUND 

2.1.  The European Capital of Culture action 

The initial scheme of "the European City of Culture" was launched at intergovernmental level 
in  19852.  On  the  basis  of  this  experience,  Decision  No 1419/1999/EC3  established  a 
Community  action  for  the  ECoC  event  for  the  years  2005  to  2019.  Member  States  were 
ranked  in  a  chronological  order  of  entitlement  to  host  the  event  each  year.  Decision 
No 1419/1999/EC was replaced by Decision No 1622/2006/EC which kept the principle of a 
chronological  order  of  Member  States  but  further  refined  the  objectives  of  the  action  and 
introduced new selection and monitoring arrangements. A new Decision was adopted by the 
European Parliament and the Council in April 20144.  

2.2.  Objectives of the ECoC action 

1 OJ L 304, 3.11.2006, p. 1. 
2 Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the 
annual event 'European City of Culture' (85/C 153/02); 
 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:1985:153:0002:0003:EN:PDF. 
3 Decision 1419/1999/EC of the European Parliament and of the Council of 25 May 1999 establishing a Community action 
for  the  European  Capital  of  Culture  event  for  the  years  2005  to  2019;  OJ  L  166,  1.7.1999,  p.  1.  Decision  as  amended  by 
Decision No 649/2005/EC (OJ L 117, 4.5.2005, p. 20). 
4 Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union action for 
the European Capitals of Culture for the years 2020 to 2033 and repealing Decision No 1622/2006/EC (OJ L 132, 3.5. 2014, 
p.  1).  This  Decision,  which  covers  the  ECoC  titles  2020  to  2033,  retains  the  general  structure  and  main  elements  of  the 
previous Decision while introducing improvements to maximise the benefits of holding the title as well as taking part in the 
competitive process for all bidding cities and their citizens. Improvements include among others the introduction of more 
explicit and measureable criteria, the reinforcement of conditionality for the payment of the Melina Mercouri Prize and the 
obligation for the cities – instead of the Commission – to carry out the ex post evaluation of the ECoC year 

2 

 
 
  
 
 
                                                            
The ECoC overall aims are to highlight the richness and diversity of European cultures and 
the  features  they  share  thereby  promoting  greater  mutual  understanding  among  European 
citizens, and to foster the contribution of culture to the  long-term development of the cities. 
ECoC  shall  strive  to  foster  cooperation  between  cultural  operators,  artists  and  cities  in 
Europe, foster the participation of the citizens living in the city and surroundings while raising 
the interest of citizens from abroad, and be sustainable and an integral part of the long-term 
cultural and social development of the city. 

The hierarchy of objectives presented in the Table below is based on the objectives as stated 
in  Decision  No 1622/2006/EC,  but  has  been  updated  to  reflect  the  content  of  the  new  legal 
basis  for  ECoC  post  2019.  The  general  and  strategic  objectives  are  taken  directly  from 
Article 2 of Decision No 445/2014/EU, with the operational objectives flowing logically from 
these. They are also informed by the selection criteria detailed in Article 5 of the new 2014 
Decision.

3 

 
 
Table on ECoC hierarchy of objectives 

General objective 

Safeguard and promote the diversity of cultures in Europe, highlight the common features they share, and foster the contribution of culture to the long-term 
development of cities 

Specific objectives (SO) 

SO1: Enhance the range, diversity and 
European dimension of the cultural offer in 
cities, including through transnational co-
operation 

Operational objectives 

Stimulate a diverse range of cultural activities 
of high artistic quality  

Implement cultural activities promoting 
cultural diversity, dialogue and mutual 
understanding 

Implement cultural activities highlighting 
(shared) European cultures and themes  

Involve European artists, promote cooperation 
with different countries and transnational 
partnerships  

SO2: Widen access to and participation 
in culture 

SO3: Strengthen the capacity of the 
cultural and creative sector and its 
links with other sectors 

SO4: Raise the 
international profile of 
cities through culture 

Create new and sustainable opportunities 
for a wide range of citizens to attend or 
participate in cultural events 

Improve cultural infrastructure 

Develop the skills, capacity or 
governance of the cultural sector 

Attract the interest of a 
broad European and 
international public  

Involve local citizens, artists and cultural 
organisations in development and 
implementation 

Stimulate partnership and co-
operation with other sectors 

Provide opportunities for volunteering 
and foster links with schools and other 
education providers 

Combine traditional art forms with 
new types of cultural expression 

4 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3. 

EVALUATION QUESTIONS 

The  evaluation  assessed  the  relevance,  efficiency,  effectiveness  and  sustainability  of  the 
European  Capitals  of  Culture  2015.  It  also  examined  the  EU  added  value  as  well  as  the 
coherence and complementarity of the action to other EU initiatives. 

To measure the relevance, questions were asked about the extent to which the objectives of 
each  ECoC  action  as  well  as  the  extent  to  which  the  ECoC's  cultural  programmes  and 
associated activities were relevant to EU objectives. 

The efficiency  section included questions about how the  management  arrangements of each 
ECoC contributed to the achievement of outputs, results and impacts; the extent to which the 
selection,  monitoring  and  EU  co-financing  procedures, 
introduced  by  Decision 
2006/1622/2006/EC  were  efficient,  the  extent  to  which  the  ECoC  managed  to  raise  the 
necessary resources, the extent to which the financial and human resources secured by each 
ECoC were appropriate and proportionate. 

Effectiveness had questions on the extent to which the EU-level objectives were achieved and 
the extent to which the ECoC's own objectives were achieved. 

Sustainability  was  about  the  extent  to  which  the  positive  effects  of  the ECoC  action  can  be 
considered to be sustainable, the EU added value of the ECoC action and the extent to which 
the ECoC action is complementary to other EU initiatives. 

4. 

METHOD 

The  basis  for  the  evaluation  was  set  up  in  the  evaluation  roadmap5.  The  evaluation  was 
overseen by a Steering Group and supported by an external study contract. 

The methodology for the evaluation of the 2015 ECoC largely followed the approach adopted 
in  previous  studies  of  the  action6.  The  focus  of  the  evaluation  methodology  has  been  on 
research at the city level and in particular the gathering of data and stakeholders' views from 
Mons and Pilsen. Key evaluation sources were as follows: 

-  EU  level  literature:  this  included  higher  level  EU  policy  and  legislative  briefings, 
papers,  decisions  and  other  documents  relating  to  ECoC.  This  mainly  focussed  on 
reports of the selection panels and the original bidding guidance to understand how the 
two ECoC established themselves in the early days; 

-  ECoC  level  literature  from  Mons  and  Pilsen:  this  included  the  original  bids  and 
applications, internal reports linked to the application processes and numerous pieces 
of  literature  collected  on  the  cultural  programme  itself.  Key  monitoring  and  in 
particular evaluation reports were also collected and analysed; 

-  Quantitative  data:  where  available,  evidence  linked  to  each  ECoC  was  collected  in 
relation to budgets and spend details, project numbers and types, participation levels 
and audience figures as well as other pieces of quantitative data to show and describe 
the work and benefits of the ECoC in each city; 

5 http://ec.europa.eu/smart-regulation/roadmaps/index_en.htm. 
6 See evaluation reports at:  
http://ec.europa.eu/dgs/education_culture/more_info/evaluations/index_en.htm?page=1&mxi=12. 

5 

 
 
 
 
                                                            
- 

- 

those 

linked 

including 

Interviews  with  managing  teams7:  those  responsible  for  the  day-to-day  design  and 
delivery  of  the  ECoC  were  interviewed  in  each  city  both  during  2015  and  again  in 
2016.  Almost  all  of  the  key  individuals  linked  to  the  delivery  agencies  were 
interviewed 
to  strategic  development,  marketing  and 
communication, project implementation and financial management; 
Interviews  with  key  stakeholders8:  mainly  face  to  face  interviews  were  undertaken 
with  stakeholders  both  directly  and  indirectly  involved  in  either  the  planning  or 
delivery  of  the  ECoC  along  with  those  more  widely  linked  to  the  cultural,  social, 
economic or political agenda of the host cities. Stakeholders included those working in 
cultural  organisations,  city/  regional/  national  level  administrations,  tourism  and 
visitor  agencies,  media  organisations  as  well  as  voluntary  and  community 
organisations.  Managers  of  individual  projects  and  activities  supported  through  the 
ECoC action that made up the cultural programme of each city were also interviewed; 
-  Survey  of  ECoC  projects:  a  specific  survey  of  projects  was  undertaken  by  the 
evaluators in Mons,  whilst in  Pilsen a  survey  already  undertaken  by the Pilsen  2015 
Foundation  (the  body  responsible  for  the  ECoC)  was  used  to  gain  further  insight  of 
project managers' views on a variety of different issues linked to the design, delivery, 
benefits and legacy of the ECoC9; 

-  The  evaluation  does  not  include  a  wider  public  consultation.  As  explained  in  the 
roadmap  which  was  published10,  the  action  is  considered  as  a  local  action. 
International participation is scattered within and outside Europe, and it is difficult to 
reach. On the other hand the opinions would be based on attendance to specific events 
and would not give useful insights for the evaluation of the ECoC action as a whole. 

The final report of the contract supporting the evaluation provides a detailed understanding of 
the 2015 ECoC action and within this an assessment of the work and progress of Mons and 
Pilsen. There are however a number of issues to consider when assessing the strengths of the 
evidence base used for this study: 

-  The  time  and  budget  available  to  undertake  the  evaluation  were  limited.  A  study 
which provides a before ("baseline") and an after picture would be ideal for assessing 
the  full  benefits  and  impact  of  the  ECoC  action.  However,  budget11  and  timing12 
constraints only allow an ex-post evaluation to take place and therefore only an after 
picture has been studied; 

-  Although both cities have undertaken some form of evaluation work themselves, some 
of the results of these studies were still in draft form at the time of the evaluation. The 
European  evaluation  of  the  ECoC  action  has  used  as  much  of  this  secondary 
information as possible, but could not benefit from its final results; 

-  The  impact  of  an  action  such  as  ECoC  will  often  only  manifest  itself  fully  after  the 
ECoC  year  itself.  Therefore  an  evaluation  undertaken  close  to  the  end  of  the  ECoC 

7 See list of interviewees in Annex 1 of the Final Report of the contract supporting the evaluation. A total of 17 
people in the Mons 2015 Foundation and 7 people in the Pilsen 2015 Foundation were interviewed. The Final 
Report  is  available  at:  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-
2015-evaluation_en.pdf. 
8 A total of 14 other stakeholders in Mons and 20 other stakeholders in Pilsen were interviewed. 
9 43 projects responded to the survey in Mons while 120 projects responded to the project survey carried out 
by the Pilsen 2015 Foundation. 
10 http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_012_capitals_of_culture_en.pdf 
11 The budget allocated to the evaluation work is proportionate to the level of EU funding directly provided to 
the ECoC (€1.5m Melina Mercouri Prize). 
12 Decision No 1622/2006/EC requires that the Commission conducts the evaluation immediately after the title 
year. 

6 

 
 
                                                            
year,  as  required  by  the  legal  base,  is  not  likely  to  identify  long-term  benefits  and 
impacts.  Therefore  the  evaluation  methodology  must  rely  more  on  the  views  and 
opinions of stakeholders rather than empirical evidence of impact. 

5. 

IMPLEMENTATION STATE OF PLAY  

5.1.  The selection and monitoring of the 2015 European Capitals of Culture 

In  accordance  with  Decision  No  1622/2006/EC,  Belgium  and  the  Czech  Republic  were 
entitled to host the ECoC in 2015. The selection is conducted in two phases: a pre-selection 
phase (candidate cities are reduced to a short-list) followed by a selection phase (the short-list 
is reduced to one single candidate). A panel of thirteen members – six of whom nominated by 
the Member State concerned and the other seven by European Union institutions  – examine 
the  bids  from  candidate  cities  on  the  basis  of  the  objectives  and  criteria  laid  down  in  the 
Decision. 

The  two  Member  States  launched  their  competitions  in  parallel.  In  Belgium,  only  Mons 
applied for the title, being pre-selected in June 2009 and then recommended in February 2010 
to host the title in 2015. In the Czech Republic, there were three candidates, Hradec Králové, 
Ostrava  and  Pilsen.  In  December  2009,  the  latter  two  were  shortlisted  at  the  pre-selection 
meeting. In September 2010, Pilsen was recommended to host the title in 2015. The two cities 
were  nominated  by  the  Council  of  the  European  Union,  upon  a  recommendation  from  the 
Commission, in November 2010 and May 2011 respectively. 

After  their  nomination,  Mons  and  Pilsen  were  subjected  to  monitoring  arrangements:  the 
progress  in  the  cities'  preparations  was  monitored  and  guided  by  a  panel  composed  of  the 
seven independent experts appointed by the European Union institutions, which also checked 
compliance with the programme and commitments on the basis of which the cities had been 
selected.  Mons  and  Pilsen  attended  two  formal  monitoring  meetings  convened  by  the 
Commission, in November 2012 and April 2014. 

During  the  monitoring  phase,  the  two  cities  introduced  modifications  into  the  programme 
described  in  their  original  applications,  in  response  to  a  changing  environment  and  to  the 
recommendations  of  the  monitoring  panel.  The  panel  also  visited  the  cities  where  it  found 
appropriate.  The  monitoring  process  ended  up  with  the  panel  making  a  positive 
recommendation to the Commission on awarding a €1.5m prize in honour of Melina Mercouri 
to both Mons and Pilsen. 

The section below describes the main features of the ECoC projects for Mons and Pilsen, in 
particular the cultural programme and the implementation of the two criteria in the Decision: 
"European  Dimension"  and  "City  and  Citizens".  The  report  from  the  supporting  study 
contains  further  information  including  the  development  of  the  applications  and  the 
governance and funding structures. 

5.2.  Mons 2015 

Mons is a city of about 93,000 people and the capital of Hainaut province in Belgium, close to 
the  French  border.  The  city  is  in  the  eastern  end  of  an  area  known  as  the  Borinage,  which 
comprises  around  thirty  municipalities.  Mons  and  the  Borinage  suffered  industrial  decline 
during the post-war period, but in recent years, Mons has benefitted from the arrival of hi-tech 
companies and as a centre for higher education. 

7 

 
 
 
 
 
The idea for Mons to host the ECoC emerged from the wider strategy of the municipality to 
regenerate the city based on culture, tourism and new technologies. Mons's application stated 
its overall aim as putting itself on the European map as a symbol of economic restructuring 
based on culture and of successful alliance between the economic and cultural spheres. In line 
with that aim, the overall theme was "where technology meets culture". 

Its cultural programme was divided into four seasons: "Dazzle" (to bring light and warmth to 
the winter months); "Metamorphosis" (to emphasize the arrival of spring and changes taking 
place  in  Mons  with  new  infrastructure  developments  and  possibilities  offered  by  new 
technology);  "Escale"  (to  encourage  visiting  or  staying  in  Mons  during  the  holiday  period); 
and "Renaissance" (to emphasize the rebirth of Mons after the decline of key industries with a 
focus both on the historical characters of the city's "golden age" and on future developments). 

Mons  2015  was  implemented  by  a  dedicated  public  utility  foundation  ("Fondation  Mons 
2015"), that was founded in March 2006. The Fondation Mons 2015 was entrusted with the 
task  of  developing  and 
the  associated 
communications  activities.  The  four  main  public  authorities  that  oversaw  the  Foundation 
(Fédération Wallonie Bruxelles, Région wallone, Province de Hainaut, Ville de Mons) offered 
strong  political  support  and  guaranteed  the  Foundation's  artistic  independence.  The  team 
largely  remained  intact  throughout  the  application,  development  and  implementation  phases 
of the ECoC. 

the  cultural  programme  and 

implementing 

Mons 2015 was one of the better-funded ECoC to date with a total budget of €72.8m (which 
includes  in-kind  support  from  corporate  sponsors  valued  at  more  than  €2m).  The  four  main 
authorities committed 68 % of the proposed budget at an early stage, in line with their initial 
commitment. EU funding in the form of the Melina Mercouri Prize was used to increase the 
overall budget. 

One  of  the  main  objectives  of  Mons's  application  was  to  "involve  citizens  in  a  process  of 
cultural  democracy".  To  this  end,  local  citizens  were  involved  as  creators,  performers  and 
audiences. Furthermore, specific events were held in the towns and communes neighbouring 
Mons and in the rest of the Borinage. 

The  ECoC  has  gone  hand-in-hand  with  a  very  substantial  development  of  the  cultural 
infrastructure of the city, in  terms of new venues and increased  exhibition and performance 
space; public and private investments of more than €143m were made during the development 
phase.  

Finally,  Mons  has  put  in  place  very  concrete  plans  for  continuation  activity  including  a 
biennial,  the  first  edition  of  which  will  be  "Mons  2018".  This  is  intended  to  be  "a  major 
cultural date on an international scale that will revive the spirit of the festive European Capital 
of Culture" and be "based on the values that brought the success of Mons 2015". The biennial 
will be preceded by events in 2016 and 2017. A new body, the "Fondation Mons 2025", will 
retain some of the staff of the "Fondation Mons 2015" and operate from the same premises 
and under the same governance structure. It will continue to operate in partnership with local 
businesses in the context of "Mons 2025 Business Club", which will serve as a successor to 
the Club 2015, a not-for-profit association set up in the context of the ECoC project. 

5.3.  Pilsen 2015 

Pilsen  is  the  fourth  largest  city  in  the  Czech  Republic  and  is  situated  approximately  90  km 
southwest of Prague. With a population of 165,000 people, Pilsen is the largest city and the 

8 

 
 
 
administrative  and  industrial  hub  of  the  Pilsen  region,  which,  with  550,000  inhabitants, 
accounts for about 5 % of the total Czech Republic's population. 

Pilsen  is  a  relatively  prosperous  city  within  the  Czech  Republic  with  a  generally  thriving 
industrial hub and comparatively high levels of employment. Although some social issues do 
exist  within  the  city,  it  does  not  perceive  itself  nor  do  statistics  suggest  that  it  is  a  city 
suffering  common  urban  problems  linked  to  issues  such  as  unemployment,  social  unrest, 
crime or pollution.  

The motivations to apply for ECoC status were therefore generally not focused on  "tackling 
urban  problems  such  as  unemployment  or  industrial  decline"  like  many  ECoC  cities  have 
stated  in  their  bids  in  the  past.  Instead,  the  main  driver  for  ECoC  was  around  more  simple 
messages  linked  to  strengthening  and  diversifying  the  cultural  offer,  making  the  city  more 
outward looking and fundamentally using ECoC as a "vehicle for positive change" throughout 
the city. 

Against this background, the general aim of the Pilsen ECoC application was to explain how 
the European Capital of Culture would contribute to the opening up of Pilsen towards Europe 
and other external influences. Hence, the development of the slogan "Pilsen, Open Up!". The 
cultural  events  were  delivered  across  four  main  streams:  "Arts  and  Technologies"  (to 
celebrate  and  strengthen  the  link  between  Pilsen's  industrial  background,  crafts,  skills  and 
business); "Relationships and Emotions" (to open up the public space of the city and engage 
the public in a discussion about their personal and national identity); "Transit and Minorities" 
(to  highlight  the  diversity  of  the  city  and  its  population);  and  "Stories  and  Sources"  (to 
promote  tourism  based  on  some  of  Pilsen's  personalities  and  to  reminisce  about  past  events 
and experiences). 

At  the  heart  of  the  governance  arrangements  for  the  ECoC  was  the  Pilsen2015  Foundation. 
The Foundation was a non-profit organisation established by the city authority in 2010 with 
the  mandate  to  prepare  and  implement  the  ECoC  programme  overall.  The  Pilsen2015 
Foundation provided strategic direction for the entire ECoC programme, managed a number 
of  key  projects  and  supported  the  design  and  delivery  of  ECoC  activities  run  by  other 
organisations within the city and beyond. 

The  total  budget  for  Pilsen2015  was  €18.2m,  which  was  largely  in  line  with  what  was 
projected at the bid stage. The majority of the funds (86 %) were from the public sector (City: 
45 %,  Region:  10 %,  State:  21 %  and  EU:  10 %),  whilst  the  remainder  came  from 
sponsorship, ticketing or merchandising. 

There  were  a  large  number  of  individual  and  practical  activities  put  in  place  to  ensure  the 
involvement  and  empowerment  of  residents.  In  particular,  a  volunteering  programme  also 
engaged active volunteering from Pilsen residents across a range of ECoC projects. A total of 
515  volunteers  were  identified  and  trained  through  the  ECoC  to  help  undertake  a  range  of 
roles including crowd control and signposting at larger events, undertaking local marketing as 
well as helping to set up various events and activities. 
Pilsen  2015  included  no  large  (and  expensive)  capital  projects,  outside  of  the  Depot201513 
and the New Theatre. 

13 DEPO2015 was originally a derelict bus depot located about half a mile from the centre of Pilsen. As part of 
the ECoC project, the bus depot was converted into a living space where businesses and culture are combined 
in innovative solutions. It hosted many activities of Pilsen 2015 and is now hosting co-working offices as well as 
a workshop for creative industries to make and build various products and art installations. 

9 

 
 
 
                                                            
6. 

ANSWERS TO EVALUATION QUESTIONS 

The  evaluation  confirms  that  many  of  the  findings  from  previous  reports14,  especially  those 
pertaining to the overall relevance, efficiency and effectiveness of the ECoC action, are still 
valid. These findings have been partially updated and refreshed with the information gathered 
during  the  2015  evaluation  wherever  possible.  This  report  concentrates  on  these  new 
elements. 

6.1.  Relevance 

The experience of 2015 reinforces the finding from previous evaluations that ECoC remains 
highly  relevant  to  the  EU  Treaty,  particularly  Article  167,  through  contributing  to  the 
flowering  of  Member  States'  cultures,  highlighting  common  cultural  heritage  as  well  as 
cultural  diversity  and 
increasing  cultural  co-operation  within  Member  States  and 
internationally. 

The cultural programmes in Mons and Pilsen demonstrated a high degree of relevance to the 
policy  objectives  set  at  EU  level  for  the  ECoC  action.  Indeed,  the  cultural  programme 
reflected the cities' own culture and history and expressed the diversity of European cultures 
in  different  and  innovative  ways.  In  particular,  it  featured  themes  and  personalities  with  a 
connection  to  Mons  (such  as  Van  Gogh  and  Verlaine)  or  Pilsen  (such  as  Trnka)  but  with  a 
European resonance. The experience of both cities shows that the ECoC is relevant to a range 
of  different  European  policy  areas  going  beyond  culture,  such  as  urban  and  regional 
development, employment, tourism as well as social and territorial cohesion etc., as illustrated 
by the many examples given in this section of the document.  

Both  Mons  and  Pilsen  have  seen  their  cultural  offer  greatly  strengthened  because  of  ECoC 
which has not only helped them to diversify their cultural scene but also diversify the types of 
audiences enjoying culture.  

For  example,  77 %  of  respondents  in  Mons  share  the  opinion  that  Mons  2015  has  attracted 
audiences  that  did  not  usually  visit  cultural  venues  or  events15.  In  both  cities,  the  cultural 
programme during the ECoC year was more extensive, more innovative and more European 
in nature compared to the cities' cultural offering in previous years. In Mons, it included 219 
projects featuring 2,390 events of different sizes, cultural  disciplines and art  forms,  most  of 
which were new for 2015. Mons 2015 found new and creative ways to use public spaces for 
artistic purposes. It also featured a significant number of new works that were performed or 
exhibited for the first time in 2015.  

In Pilsen, 600 cultural events and experiences were delivered throughout 2015. Local people 
themselves  were  often  the  subject  of  various  ECoC  exhibitions,  shows  and  performances, 
meaning that local citizens became a key aspect of the overall ECoC programme, which was a 
relatively  new  experience  in  the  city  and  a  way  to  reach  out  to  new  types  of  audiences. 
Examples  include  the  organisation  of  "neighbourhood  walks"  run  by  local  people  who 
delivered "professional" guided walks or the Family Photo Album consisting of around 200 
photographs taken in Pilsen and borrowed from local residents and visited by 5,150 people. 

14 See footnote 6. 
15  Evaluation  conducted  by  KEA  European  Affairs 
http://www.keanet.eu/wp-content/uploads/Mons-2015-ECoC-demystifying-the-risk-of-cultural-investment-
ENG.pdf 

the  Mons  2015  Foundation 

for 

(p.  15); 

10 

 
 
 
                                                            
Both cities have used ECoC to help them internationalise their cultural offer, thus making it 
relevant to the European dimension. This is particularly true in Pilsen which used ECoC as a 
vehicle to help open up the city to Europe: 53 % of cultural players taking part in the ECoC 
project survey16 now say that they had good international links with partners because of their 
participation in the ECoC while this participation was low or non-existent prior to 2015. The 
ECoC has also strengthened the international dimension of cultural activity within Mons and 
the Borinage. International collaborations have increased the number of connections with new 
partners  performing  in  other  countries.  In  Mons,  approximately  40 %  of  the  respondents 
report having collaborated with some of their partners in other countries for the first time. 

The ECoC label brings a significant amount of profile to the host city: this is particularly true 
for smaller host cities like Pilsen or Mons who could not hope to generate the amount of press 
coverage, visitor numbers and overall interest in its cultural offer without it hosting an ECoC. 
In Mons, data confirms that there were visits from 450 accredited international journalists and 
3,717 articles in the international press or items on international radio and television. The city 
tourist office experienced a five-fold increase in tourist visits during 2015, reaching a total of 
250,000. In Pilsen, the 1.4 million visitors who attended ECoC projects in 2015 represent a 
28 % increase from the two years prior to 2015. 

The ECoC "label" also acts as a significant generator of interest from stakeholders in the city 
around culture itself, with both Mons and Pilsen stating that ECoC helped raise the profile of 
culture  among  a  wide  range  of  policy  makers  in  the  city.  As  an  example,  Mons  2015  has 
helped cement the link between culture and tourism in the city and beyond. A key factor here 
has  been  the  close  co-operation  between  the  Ville  de  Mons,  the  Foundation  and  Wallonie 
Bruxelles  Tourisme  (including  through  the  local  tourist  office  "Visit  Mons").It  has  also 
created  new  capacity  for  corporate  sponsorship  of  culture.  Whilst  corporate  sponsorship  of 
culture was not absent before 2015, Mons lacked a comprehensive and co-ordinated approach 
to the attraction of corporate sponsors. A key achievement of the ECoC has thus been the new 
partnership(s)  created  with  the  local  corporate  sector,  encompassing  large  companies  and 
multinationals, as well as local SMEs. The Club Mons 2015 gathered 841 members and raised 
a total contribution of 841.000 €.  

6.2.  Efficiency 

Overall,  the  ECoC  action  seems  to  have  been  implemented  with  a  relatively  high  level  of 
efficiency  at  EU  level.  Indeed,  the  selection  process  enabled  the  selection  of  cities  with  the 
capacity, resources and vision to implement ECoC responding to the objectives of the action. 
Both cities have also benefited from the monitoring at EU level and from the informal support 
given  by  the  monitoring  panel  and  the  European  Commission.  In  particular,  the  concerns 
expressed  by  the  monitoring  panel regarding  the  worryingly  slow  and  stuttering  start  of  the 
development of Pilsen 2015 prompted the city to react and put forward a clear action plan to 
help negate the main barriers and problems identified.  

At  the  same  time,  the  modest  funding  provided  by  the  EU  (EUR1.5m  in  the  form  of  the 
Melina Mercouri Prize for each ECoC) can be said to have had a considerable leverage effect 
as  getting  the  ECoC  title  and  Melina  Mercouri  Prize  prompted  the  two  cities  to  invest 
considerable sums in their ECoC programmes (approximately EUR72,8m in the case of Mons 
and  EUR18,2m  in  the  case  of  Pilsen)  and  in  associated  infrastructure  developments 
(EUR143,5m  for  Mons  and  EUR48,6m  for  Pilsen).  Both  cities  also  report  that  the  Melina 

16 Pilsen2015 project survey carried out by the West Bohemian University. 

11 

 
 
 
                                                            
Mercouri Prize offers important symbolic value, as it represents an endorsement by the EU of 
their  activities  and  offers  opportunities  to  positive  publicity.  However,  the  impact  of  the 
Melina Mercouri Prize could be enhanced by greater publicity at EU level. 

At the city level, both Mons and Pilsen have delivered their ECoC in an efficient way. Mons 
had  a  budget  that  was  around  four  times  larger  than  Pilsen  but  both  used  national  and  EU 
funds to  implement cultural  programmes  of high artistic quality and of considerably  greater 
size than the cities' "usual" cultural offering. Both ECoC cities designed and delivered a large 
amount  of  cultural  activity  with  many  hundreds  of  performances,  exhibitions  and  other 
activity  taking  place  as  a  direct  consequence  of  ECoC.  Both  cities  also  continued  ECoC 
traditions  by  implementing  a  programme  that  was  wide,  varied  and  innovative.  The 
programme  used a range of cultural genres to entertain its audiences ranging from street art 
through to world class dance. 

To  some  extent,  the  2015  evaluation  shows  that  "money  matters"  when  it  comes  to  ECoC, 
with the larger Mons budget helping the city achieve more in terms of reach, the content of its 
cultural programme and the amount of legacy and sustainability being achieved. However, the 
2015 evaluation also shows that a small ECoC budget in a comparatively small city can still 
produce  a  very  strong  outcome  across  a  range  of  different  issues.  Pilsen  should  be 
congratulated in putting on an ECoC which made a large difference across the city and which 
used its smaller resources in a very efficient way. Although the cultural programme perhaps 
had  fewer  benefits  in  helping  showcase  Pilsen  to  Europe,  it  had  a  great  benefit  in  helping 
promote  Europe  to  Pilsen.  Indeed,  two  thirds  of  respondents  to  the  survey  carried  out  by 
Pilsen  2015  said  that  their  work  and  activities  were  much  more  diverse  internationally  as  a 
consequence of taking part in the ECoC year and that this diversity would continue beyond 
the year itself. 

The  delivery  mechanisms  established  in  both  cities  were  strong  and  there  were  very  few 
negative views placed on this aspect coming from the interviews and the surveys carried out. 
Although both Mons and Pilsen had a different scale of cultural programme and activity, both 
had similar delivery mechanisms, similar partnership arrangements and similar development 
processes.  Compared  to  other  ECoC,  the  operation  of  the  governance  and  management 
arrangements of Mons 2015 have been relatively smooth and stable, despite the complexity of 
the  Belgian  governance  context.  Mons  2015  benefited  from  the  strong,  high-level  political 
support  offered  by  its  mayor  as  well  as  from  the  continuity  within  the  key  members  of  the 
operational team throughout  the process. Pilsen had a  very difficult start  to  its development 
process with it lacking progress in terms of its cultural programme, key ECoC projects and its 
funding.  However,  Pilsen  also  shows  that  ECoC  that  have  difficult  starts  can  also  change 
direction  and  make  the  year  positive  with  the  delivery  of  thousands  of  events  attracting  1.4 
million  visitors  through 2015  and  within  the  limited  budget  allocated17,  as  long  as  the  right 
people are in charge who have the power to influence a turn-around in proceedings. 

Mons had much stronger involvement of the relevant regional Ministries and related agencies 
compared to Pilsen. Pilsen was relatively unusual to other ECoC in its relative isolation from 
national  Government  and  national  support.  As  a  consequence,  Pilsen  struggled  to  secure 
budgets and reach international visitors. 

17 65 % of projects taking part in the Pilsen2015 project survey carried out by the West Bohemian University 
reported that they thought the overall programme was well run, which is relatively high considering the large 
difficulties and negative press which it encountered in its early days. The survey carried out by Ondrej Jirkovsky 
(as yet unpublished) on the perceptions of local Pilsen residents on the quality of culture in Pilsen showed that 
76-80 % of residents felt that the city now has a much higher quality offer for them to enjoy. 

12 

 
 
 
                                                            
6.3.  Effectiveness 

The ECoC action in 2015 has proved effective against the objectives set for it at EU level, as 
well as the objectives set by the cities holding the title. The action has achieved an impact that 
would not have arisen through the actions of Member States alone. In the absence of ECoC, 
both  the  2015  title-holders  would  have  been  free  to  invest  their  own  resources  in 
implementing  cultural  programmes  and  developing  their  cultural  infrastructure.  However, 
their  designation  as  ECoC  has  attracted  additional  resources,  including  from  private 
sponsors18,  as  well  as  greater  media  coverage19,  an  increase  in  international  tourist  visits20  
and enhanced local pride in the city. Regarding the latter, research21 shows for example that 
86 %  of  the  Mons's  residents  felt  that  the  ECoC  had  been  a  positive  thing  and  the  project 
survey suggests that more than half of respondents feel that the ECoC has greatly improved 
the image of Mons with its own residents. At the same time, the precise magnitude of impact 
is hard to determine, given the limited baseline data submitted with the ECoC applications.  

The  effectiveness  of  the  ECoC  in  2015  is  particularly  strong  in  terms  of  the  two  cities 
maximising the opportunity to strengthen the cultural organisations in Mons and Pilsen. Both 
cities  recognised  that  ECoC  is  a  powerful  tool  in  helping  capacity  build  and  develop  local 
organisations,  whether  in  terms  of  them  developing  stronger  business  plans,  helping  with 
marketing or helping their staff put on bigger and better productions. ECoC has also helped 
these  local  cultural  organisations  employ  more  staff  and  also  buy  new  equipment  that  will 
again make their cultural offer much better quality. The cultural sector in the two cities has 
acquired skills which it will benefit from in its future activities beyond 2015, according to the 
answers to the surveys and interviews carried out for the two cities22. 

The  2015  ECoC  evaluation  has  also  found  that  the  two  cities  put  on  "new"  and  "better" 
cultural content than was previously the case, such as new uses of public spaces with open air 
events,  festivals  and  urban  art  installations.  In  Pilsen,  an  independent  survey  (as  yet 
unpublished)  on  the  perceptions  of  local  Pilsen  residents  showed  that  76-80 %  of  them  felt 
that the city now has a much higher quality offer for them to enjoy. ECoC did not replace or 
substitute existing cultural content that would have happened in the absence of the year and 
the review of the cultural programmes shows that Mons and Pilsen made the most of the title-
year  and  were  ambitious  and  innovative.  None  of  the  stakeholders  taking  part  in  the 
evaluation  felt  that  2015  was  a  lost  opportunity  and  that  more  could  have  been  done  to 
maximise its content and benefit23. 

The ECoC in both cities also used the year to encourage cultural organisations in the city to 
work  with  one  another  more  than  they  did  previously.  Both  Mons  and  Pilsen  made  it  a 
condition of grant (or involvement in the ECoC programme) to work in partnership with other 
local cultural players. Joint ticketing, the sharing of equipment and joint marketing were just 
some  of  the  examples  of  where  the  ECoC  has  helped  stimulate  better  partnership  working. 

18 See example of Mons given under point 6.1 "Relevance". 
19 See data given under point 6.1 "Relevance".  
20 See data already provided under point 6.1 "Relevance". 
21  Evaluation  conducted  by  KEA  European  Affairs 
http://www.keanet.eu/wp-content/uploads/Mons-2015-ECoC-demystifying-the-risk-of-cultural-investment-
ENG.pdf. 
22 
http://www.keanet.eu/wp-content/uploads/Mons2015-Rapport-IV-final-
19072016.pdf?4f4eb7,  p.  47.  For  Pilsen,  42 %  of  ECoC  projects  reported  that  the  technical  skills  linked  to 
performance production, marketing, ticketing, lighting and social media were all being used beyond the year 
itself to provide stronger, bigger and higher quality activities post-2015; 
23 https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation_en.pdf 

the  Mons  2015  Foundation 

(p.  16); 

Mons, 

see 

For 

for 

13 

 
 
                                                            
Even in cities the size of Mons and Pilsen, the cultural sector can often work in isolation and 
ECoC is a good vehicle to strengthen this aspect of the cultural infrastructure of future ECoC 
cities. 

The  two  2015  cities  have  been  effective  in  showcasing  local  culture  to  the  large  number  of 
audiences attending ECoC events. In Mons, total audiences were nearly 2.2m people (a record 
number of 180,000 visitors or the Van Gogh exhibition), most of which must be considered as 
additional to the audiences of previous years, as most events were new in 2015 and there is no 
evidence  that  events  and  venues  outside  the  Mons  2015  cultural  programme  suffered  any 
significant loss of audiences. The ECoC also attracted audiences from further afield in 2015 
than  in  previous  years.  For  example,  data  from  the  tourist  office  provides  evidence  of  a 
marked  increase  in  tourist  visits  to Mons  during  2015,  of  which  many  were  specifically  for 
cultural reasons. 

In  Pilsen,  with  over  600  cultural  events  delivered  throughout  2015,  1.4  million  visitors 
attended  activities  organised  through  ECoC  projects  throughout  the  year,  who  spent  around 
€20m directly with ECoC projects. This number of visitors was a 28 % increase from the two 
years  prior to  2015.  Main  events included  the  Liberation  Festival  with  219,000  visitors,  the 
Giant puppets in Pilsen (Skupa's Pilsen festival) with 73,000 visitors or the exhibitions "Jiri 
Trnka Studio" and "Trnka's Garden" with 44,000 visitors. Both Mons and Pilsen ensured that 
local  cultural  talent  enjoyed  as  much  attention  as  possible  with  many  local  cultural  venues 
enjoying audience numbers much higher than they had seen before. This has given them vital 
experience and confidence to use in the future and overall will help the cultural operators raise 
their profile beyond their normal reach. There was some tension in Pilsen around the need to 
involve  the  European  dimension  (with  European  cultural  organisations)  at  the  same  time  as 
involving  local  organisations.  Pilsen  dropped  some  local  cultural  projects  in  favour  of 
European  ones  in  order  to  reduce  the  programme  which  caused  a  certain  amount  of  "anti-
ECoC"  activity.  Linked  to  this,  although  both  ECoC  had  some  star  performers,  neither  of 
them  included  high  profile  cultural  celebrities  in  their  programme  and  instead  focussed  on 
nurturing local talent. 

In terms of being effective around targeting specific groups in the city, then Mons has been 
more  successful  than  Pilsen  in  this  respect.  Mons  had  a  number  of  projects  specifically 
targeted at different groups – whether that is the young, old or disadvantaged. Pilsen had less 
opportunity to be effective in this area – again mainly due to its budget which only allowed it 
to focus on the wider audience. This is not to say that more "minority groups" did not benefit 
but rather that they were not a focus of specific cultural activity. 

Although  the  evidence  of  the  number  of  foreign  visitors  was  less  available  for  Mons,  the 
qualitative evidence from both cities backed up by data from Pilsen (where 5 % of the project 
audiences  were  from  outside  of  the  country)  shows  that  ECoC  have  been  less  effective  in 
attracting international visitors. Although evidence from previous ECoC evaluations is mixed, 
the more recent evaluations (2014/2015) show that ECoC programmes need to be relatively 
big  budget  in  order  to  lay  on  and  market  a  cultural  programme  that  is  big  enough  to 
specifically  attract  foreign  visitors  to  make  a  special  journey  to  the  city.  Although  foreign 
visitors do attend and enjoy ECoC projects, it may be that they are already in the city because 
of its wider attractions rather than specifically there just because of ECoC. 

Pilsen  did  not  have  any  key  physical  developments  as  part  of  its  ECoC  year  outside  of  the 
Depot2015.  Interestingly,  instead  of  local  stakeholders  complaining  that  this  lessened  the 
impact and effectiveness of the overall ECoC programme they were adamant that this did not 
reduce any of the benefits. Although it reduced a more obvious legacy for the programme (see 
sustainability)  it  did  not  seem  to  dampen  the  enthusiasm  or  passion  that  local  cultural 

14 

 
 
operators  had  of  the  ECoC  year.  Most  were  quick  to  state  that  new  buildings  do  not 
necessarily  lead  to  successful  ECoC  and  all  were  keen  to  put  on  a  rich  and  varied  cultural 
programme rather than spend their limited budget on new capital projects/ buildings. 

6.4. 

Sustainability 

The timing of this evaluation makes it difficult to draw conclusions about sustainability. The 
research has identified some potential for sustainability of activities and impetus, particularly 
in Mons where there are concrete plans for a legacy event. However, further research would 
be needed to identify the extent of sustainability in practice. 

Both the 2015 ECoC genuinely thought and planned for sustainability and legacy. They were 
both keen to ensure that ECoC lasted more than one year in terms of its benefits and impact. 
Because  of  its  bigger  budget,  Mons  has  more  obvious  sustainability  and  legacy  than  Pilsen 
and has more physical infrastructure in place in terms of new cultural buildings and facilities. 
Mons also had more concrete legacy plans in place including the Mons 2018 which involves a 
high profile festival in line with Mons 2015. 

Pilsen  did  not  have  a  clear  legacy  strategy  in  place.  Although  they  were  aware  of  the 
importance of prolonging the benefits of ECoC there was less in the way of a specific plan to 
help this become a reality. Having said this, the core team from the Pilsen 2015 Foundation is 
still in place and they personally drive forward many of the legacies as well as the learning 
from the year. As an example, the Director of Pilsen 2015, is currently leading the DEPO2015 
cultural venue, one of the most important legacies of the year.  There was a certain amount of 
fragility  around  this  though  and  a  danger  this  could  be  lost  if  key  staff  move  on.  Properly 
planning  for  sustainability  for  the  ECoC  rather  than  "hoping"  that  a  longer  term  legacy 
appears is a key learning point here. 

Both Mons and Pilsen still have their Foundations in place post 2015. This is now becoming 
more  common  practice  among  ECoC  who  are  recognising  that  having  an  independent  body 
driving forward policy and practice within the cultural agenda of the city is highly beneficial. 
The  Foundations  set  up  by  ECoC  are  therefore  increasingly  doing  more  than  simply 
overseeing  the  ECoC  year  and  are  becoming  a  much  more  established  part  of  the  cultural 
infrastructure of the host cities. The Foundation staff in both cities is still very much involved 
in the development and delivery of culture and are still using their vast amount of knowledge, 
skills and experience to drive forward positive change across the city's cultural offer. Again, 
based on the last few ECoC evaluations, ECoC staff seems to be remaining in post after the 
year  rather  than  moving  to  another  city  which  has  benefits  all  round  for  the  sustainability 
theme. 

The main legacies of the ECoC are often less tangible to see but the evaluations carried out 
over the years suggest that there are nevertheless very important longer term impacts of the 
action.  As  with  other  recent  ECoC  evaluations,  stakeholders  in  Mons  and  Pilsen  articulated 
the legacies of the year in terms of stronger skills, stronger relationships and a higher profile 
for  culture  in  the  city  more  widely.  These  less  tangible  legacies  will  equip  the  cultural 
operators  to  deliver  better  quality  cultural  offers  and  will  strengthen  the  organisations 
delivering  cultural  projects  well  beyond  the  year  itself.  For  example    the  main  "softer" 
legacies identified by stakeholders in Pilsen were threefold24: 

-  A stronger international dimension to the cultural offer, with links to mainly European 
partners  formed  in  2015  generally  staying  in  place  beyond  the  ECoC  year.  12 %  of 

24 Pilsen2015 project survey carried out by the West Bohemian University. 

15 

 
 
 
 
                                                            
ECoC  projects  were  continuing  to  strengthen  their  links  with  international  partners 
beyond 2015; 

-  A  stronger  set  of  skills  and  capacity  held  by  cultural  operators  developed  as  a 
consequence  of  designing  and  delivering  ECoC  projects  throughout  2015.  42 %  of 
ECoC  projects  reported  that  the  technical  skills  linked  to  performance  production, 
marketing,  ticketing,  lighting  and  social  media  were  all  being  used  beyond  the  year 
itself; and 

-  A stronger set of links and networks within the city among cultural operators reported 
during the year itself was again continuing beyond 2015. Again, these joint activities 
were still occurring due to the legacy of strong relationships built up over the ECoC 
year. 

Another  longer  term  legacy  of  the  ECoC  in  Mons  and  Pilsen  has  been  around  how  the 
programmes have attracted a new type of audience to experience and enjoy culture. As with 
other recent ECoC, widening participation away from the "converted" and laying on cultural 
projects that appeal to those who "usually watch TV" will have an important legacy for both 
cities.  

6.5.  Coherence  

The  ECoC  Action  has  also  proved  to  be  complementary  to  other  EU  policies  and 
programmes. In particular, it has been reinforced by and added value to investments made the 
ERDF; the ECoC has stimulated cities to use ERDF for investments in cultural infrastructure 
and has given greater impetus to the completion of those investments in time for the title-year. 
In Mons, there was a wider strategy for the development of culture and tourism in Mons from 
the early 2000s, which foresaw investments in physical infrastructure and facilities. However, 
a potential bid was discussed as early as 2002 and the decision to bid was made in 2004 in the 
knowledge that Mons would have a good chance of winning. In that context, it is clear that the 
many investments that were initiated from 2007 onwards were clearly intended to support the 
ECoC  application  and,  in  the  event  of  a  successful  application,  the  title-year25.  In  Pilsen,  a 
total of €48.5 million was invested in infrastructure relating to Pilsen 2015, including two new 
venues, the New Theatre and DEPO2015. The Cultural Factory, that received the approval for 
financing from the ERDF, had to be abandoned after asbestos was found in the roof, with a 
considerable  increase  in  the  related  costs  that  resulted  in  the  project  to  be  abandoned  and 
replaced by DEPO2015. 

ECoC often helps galvanise a city and its stakeholders to get behind culture in a way that was 
never  possible  before,  not  just  in  terms  of  cultural  stakeholders  but  also  those  related  to 
employment, enterprise, tourism and city investment. This means that the overall relevance of 
the ECoC action to a variety of European policy areas is assured. 

6.6.  EU added value 

As  already  mentioned  above,  the  ECoC  action  has  achieved  an  impact  that  would  not  have 
arisen  through  the  actions  of  Member  States  alone.  In  the  absence  of  ECoC,  both  the  2015 
title-holders  would  have  been  free  to  invest  their  own  resources  in  implementing  cultural 
programmes and developing their cultural infrastructure. However, their designation as ECoC 
has attracted additional resources, including  from private sponsors, as well as greater media 

25 See in particular KEA evaluation, op. cit., p. 11. 

16 

 
 
 
 
                                                            
coverage,  increased  international  tourist  visits  and  enhanced  local  pride  in  the  city.  These 
benefits  would  have  been  unlikely  to  arise  to  the  same  extent  in  the  absence  of  ECoC 
designation; in that sense, the ECoC action has generated clear "European added value". 

The "label" which ECoC gives to host cities is one of the key aspects of the European added 
value of the Action. This label brings a significant amount of profile to the host city at a level 
that  would  simply  not  be  possible  without  ECoC  status.  The  ECoC  label  also  acts  as  a 
significant generator of interest from stakeholders in the city around culture itself; both Mons 
and Pilsen stated that ECoC helped raise the profile of culture among a wide range of policy 
makers  in  the  city.  Interestingly,  in  Pilsen,    stakeholders  felt  that  the  added  value  of  ECoC 
around the European dimension was particularly high because many of the cultural operators 
in the city did not previously have links with European organisations.  

7. 

CONCLUSIONS 

The ECoC action is highly valued by the hosting cities that can obtain positive impacts during 
the year, as well as during the preparation phase. The action also remains relevant at EU level. 
However, its long-term impacts cannot be assessed at this stage. 

The lack of baseline data for comparison of the situation in the city before winning the title, 
after  nomination  and  after  the  ECoC  year  makes  the  evaluation  of  impacts  difficult.  In  this 
sense, the provisions of Decision 445/2014/EU, which will cover the ECoC titles from 2020 
to  2033,  foresee  that  the  cities  shall  be  responsible  for  the  evaluation  of  the  results.  This 
requirement  is  part  of  the  criteria  used  to  assess  the  applications  and  should  allow  for  the 
collection of data at early stages to establish a baseline. 

The programmes  implemented by the two 2015 title-holders  were  innovative and consistent 
with the objectives of the ECoC action; they reflected its European dimension, involved many 
local residents and stakeholders, brought culture to new audiences through specific strategies 
(in particular in Mons, to a lesser extent in Pilsen) and have a planned legacy both physical 
(new  cultural  venues)  and  intangible  (in  the  form  of  a  biennale  in  Mons,  in  Pilsen  with  the 
continuation of the activities in Depo2015 and in the form of increased capacity and increased 
cultural offer in the city). However, deeper assessments of the extent of the benefits produced 
against the costs incurred will be useful to confirm the impact of the programme. This is now 
the responsibility of both cities to invest in research so as to better understand how they have 
optimised cultural, social and economic benefits and be able to demonstrate the impact of the 
title-year for the development of the city, thereby justifying the value of public spending. 

17 

 
 
 
 
 
ANNEX 1 – PROCEDURAL INFORMATION 

The evaluation was led by Directorate General for Education, Culture, Youth and Sports (DG 
EAC). It is included in the Work Programme of Creative Europe for 201526 and in the Agenda 
Planning with the reference 2015/EAC/012. 

The  evaluation  was  supported  by  an  external  and  independent  evaluator,  under  a  service 
contract.  The  service  contract  has  been  implemented  via  a  Framework  Contract  with 
reopening of competition and in accordance to the Financial Rules Applicable to the General 
Budget of the Union27 and its Rules of Application28. 

The evaluation Roadmap was adopted in December 201529. 

According to the Roadmap, a Steering Committee including staff from DG EAC and from the 
Directorate  General  from  Regional  Policy  was  established  in  May  2015.  The  Steering 
Committee met in  four  occasions: to prepare the Terms of  Reference (approved on 23 June 
2015), to approve the Inception Report in November 2015, to discuss the draft final report in 
October 2016 and to  approve the final  report in November  2016. Extensive correspondence 
between the Steering Committee members was held in between the meetings to follow-up on 
the evaluation. 

The evaluation initial schedule foresaw a final report in the third quarter of 2016. Due to the 
late availability of data used for the evaluation (and gathered by the cities) it was agreed to 
delay the submission of the final report to the fourth quarter of 2016. 

The  evaluation  did  not  need  to  be  submitted  to  the  Regulatory  Scrutiny  Board,  as  it  is  not 
considered a major evaluation, does not contain an impact assessment and does not constitute 
a fitness check for legislation. 

The 2015 evaluation of the ECoC used a series of data sets to inform its findings. The main 
ones being: 

- 

Interviews with over 50 stakeholders directly and indirectly involved in the planning, 
operation and delivery of the two ECoC programmes; 

-  A survey of 163 projects that were part of the two ECoC programmes who delivered 

the cultural programme attached to the intervention; 

-  A  literature  review  of  ECoC  and  European  level  information  on  the  two  ECoC 
including  application/  bid  information,  EC  Committee  reports,  cultural  programme 
brochures,  web  sites  and  news  articles.  The  ECoC's  own  external  evaluations  have 
also been used to inform the European evaluation process. 

Together,  the  above  evidence  base  provides  the  evaluation  with  a  valid  and  rounded  set  of 
data  to  inform  the  views  on  the  main  aspects  of  the  ECoC  evaluation  including  efficiency, 
effectiveness, sustainability and relevance. This view has been informed by: 

-  The scale of the consultation exercise. Over 200 individuals have fed their views and 
opinions into the evaluation process either through face to face interviews, telephone 
interviews or through the project level survey. The majority was done through face to 
face interviews over a series of visits to each city by the evaluation team; 

26 http://ec.europa.eu/dgs/education_culture/more_info/awp/docs/c-2014-5313_en.pdf 
27 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R0966-20160101&from=EN 
28 http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R1268-20160101&from=EN 
29 http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_012_capitals_of_culture_en.pdf 

18 

 
 
                                                            
-  The  nature  of  the  consultation  exercise.  The  evaluators  were  keen  to  consult  with 
those who had a more indirect and external view of the two ECoC. These stakeholders 
including  journalists,  those  not  directly  benefitting  from  the  ECoC  (e.g.  rejected 
projects) as well as those working in the wider cultural policy agenda at regional and 
city level. This ensures the evaluation is not simply based on those who benefitted the 
most from the ECoC. 

19 

 
 
 
 
ANNEX 2 – STAKEHOLDERS' CONSULTATION  

The stakeholders were consulted via targeted consultations (interviews or  phone interviews) 
and via an on-line survey, mainly on projects that had participated in the ECoC year or had 
submitted  a  proposal  that  was  rejected.  For  Mons,  the  on-line  survey  of  projects  (survey 
questions  below)  was  implemented  by  the  contractor  supporting  the  evaluation,  while  for 
Pilsen, the contractor relied on the survey of projects that was implemented by the Foundation 
itself and which questions aligned with the information sought.  

The  consultations  included  the  team  responsible  for  the  implementation  of  the  ECoC,  the 
political  statements  involved  in  the  project,  the  projects  participating  in  the  programme  or 
having submitted proposals to participate that were rejected, and personalities attached to the 
cultural  tissue  in  both  cities.  The  objective  of  the  consultations  was  to  have  evidence 
supporting the findings and conclusions of the evaluation. The questions used for the survey 
of  projects  in  Mons  can  be  found  in  the  report  produced  by  the  experts  assisting  the 
Commission: 

https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-
evaluation_en.pdf. 

Similar questions were included, where necessary, in the survey of projects carried out by the 
Pilsen 2015 Foundation.  

The Commission's minimum standards have not been met for this stakeholders' consultation. 
The consultation was restricted to relevant stakeholders in the two cities hosting the title. The 
characteristics of the action and the scope of the evaluation do not make it necessary to extend 
the consultation to a wider public, as indicated in the roadmap published.30 

Annex  3  details  the  evaluation  questions  for  which  the  stakeholders'  consultation  was  used. 
These  regard  mainly  the  questions  which  answers  are  not  based  on  factual  data.  The 
stakeholders' consultation was particularly useful to find information about the impact of the 
ECoC in the cultural offer of the city, the participation of citizens and local cultural operators, 
the building of capacity for local cultural operators and legacy prospects.  

The  questionnaire  used  for  the  on-line  survey  of  projects  can  be  found  in  the  Report 
supporting the evaluation31. 

30 http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_012_capitals_of_culture_en.pdf 
31 https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation_en.pdf  

20 

 
 
 
 
                                                            
ANNEX 3 – METHODOLOGY APPLIED 

The figure below presents the overview of the methodology. A more detailed overview of the 
methodology  and  sources  used  for  each  of  the  evaluation  questions  is  presented  in  the 
subsequent tables and text. 

1. 

 Overview of methodology and tasks of the evaluation: 

21 

Inception PhaseAnalysis and Reporting PhaseTask One: Kick Off MeetingTask Two: Initial ConsultationsTask Three: First Visits to CitiesTask Eleven: Interim Report and MeetingTask Twelve: Factual Check of the City ReportsTask Thirteen: Final Report and MeetingTask Five: Inception Report and MeetingMain Research PhaseTask Six: Initial Bulletin of ResultsTask Eight: Second Visits to CitiesTask Nine: Additional InterviewsTask Ten: Information sharing and analysisTask Four: Desk ResearchTask Seven: Online Survey of Projects 
 
 
Table 1.1  Evaluation questions: Relevance, EU added value and coherence 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

a
t
a
D
e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a

h
t
i

w
s
w
e
i
v
r
e
t
n

I

EQ1:  To  what  extent  are  the  objectives  of  the 
ECoC  Action, 
in  Decisions 
as  defined 
1419/EC/1999  and  1622/EC/2006,  consistent  with 
and relevant  to the  objectives of Article 167  (ex-
Article 151) of the EC Treaty? 

Y 

Y 

As  far  as  the  conclusions  made  for  the  two  cities 
allow it, to what extent have the general, specific and 
operational objectives of the Action  proved relevant 
to Article 167 (ex-Article 151) of the EC Treaty? 

Y 

Y 

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Y 

Y 

EQ2:  What  is  the  EU  added  value  of  the  ECoC 
Action? 

Y 

Y 

Y 

Y 

As far as the conclusions made for the 2 cities allow, 
what  is  the  added  value  of  the  European  Capital  of 
Culture being an EU initiative? 

EQ3:  To  what  extent  were 
complementary to other EU initiatives? 

the  ECoCs 

Y 

Y 

Y 

As far as the conclusions made for the 2 cities allows 
it,  to  what  extent  has  the  Action  proved  to  be 
complementary  to  other  EU  initiatives  in  the  field  of 
culture? 

To  what  extent  has  each  ECoC  been  reinforced  by 
and  added  impetus  to  investments  by  the  EU 
Structural Funds? 

To  what  extent  have  ECoCs  complemented  other 
EU 
initiatives,  e.g.  European  Youth  Capital, 
European Green Capital? 

EQ4: To what extent were the objectives of each 
ECoC  relevant  to  the  objectives  set  at  the  EU 
level  and,  by  extension,  to  the  objectives  of 
broader EU policy? 

What  was  the  main  motivation  behind  the  city 
bidding to become a European Capital of Culture? 

How  and  when  was  the  decision  made  to  bid  for 
ECoC?  How was the bid prepared and by whom? 

What  was  the  process  of  determining  objectives?  
Was  there  a  process  of  consultation  in  each  city  to 
define aims and objectives? 

Y 

Y 

Y 

Y 

Y 

22 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
   
 
 
 
 
 
 
 
 
 
 
 
 
e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

a
t
a
D
e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Evaluation Question 

What were the objectives of the city in being ECoC? 
What was the relative importance of each objective? 

 Have  any  specific  objectives  of  the  ECoC  event 
been related to social impacts? 

In  this  connection,  did  the  objectives  of  the  ECoC 
event  include  reaching  out  to  all  sectors  of  society, 
including 
the  excluded,  disadvantaged,  disabled 
people and minorities? 

How did the programme seek to broaden access and 
participation,  were  specific 
themes  or  content 
selected to do this? 

To what extent have the specific themes/orientations 
of  the  cultural  programme  proved  to  be  relevant  to 
the objectives defined? 

EQ5:  To  what  extent  were  the  ECoC's  cultural 
programmes and associated activities relevant to 
their own objectives and to the objectives set at 
the European level? 

To  what  extent  were  the  objectives  consistent  with 
the  Decision  and  with  the  ECoC's  own  application? 
(special focus on the European dimension) 

To what extent were the activities consistent with the 
ECoC's own objectives, with the ECoC's  application 
the 
and  with 
European dimension) 

the  Decision?  (special 

focus  on 

How  was  the  European  dimension  reflected  by  the 
themes put forward by the ECoC event and in terms 
of  cooperation  at  European  level?  How  did  the 
Capitals  of  Culture  seek  to  make  the  European 
dimension  visible?  To  what  extent  did  the  2  cities 
cooperate? 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

To  what  extent  were  international  cooperation  and 
co-productions  reflected  in  the  programme,  how  did 
the  programme  seek 
international 
audiences? 

target 

to 

Y 

Y 

Y 

Y 

Y 

23 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 1.2  Evaluation questions: Efficiency 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ6:  How  did  the  management  arrangements  of 
each  ECoC  contribute  to  the  achievement  of 
outputs, results and impacts? 

How  have  the  organisational  models  of  the  formal 
governing  Board  and  operational  structures  played a 
role  in  the  European  Capital  of  Culture?  What  role 
have  the  Board  and  operational  structures  played  in 
the  ECoC  event's  implementation?  At  what  stage 
were these structures established? 

Who were the key external partners and stakeholders 
in  the  project,  and  how  were  they  involved  in 
governance structures? 

Who  chaired 
the  Board  and  what  was  his/her 
experience?  What  were  the  key  success  and  failure 
elements  related  to  the  work  of  the  Board  and 
operational structure used and personnel involved? 

Has  an  artistic  director  been  included  into  the 
operational structure and how was he/she appointed? 
What  were  the  key  success  and  failure  elements 
related  to  the  work  of  the  artistic  director  and 
personnel involved? 

What was the process of designing the programme? 

How were activities selected and implemented? 

What cultural activities took place in the development 
phase / before the title year? 

How  did  the  delivery  mechanism  contribute  to  the 
achievement of outputs? 

to 

in/contributed 

To what extent has the communication and promotion 
strategy  been  successful 
the 
promotion  of  city  image/profile,  promotion  of  the 
ECoC  event,  awareness-raising  of  the  European 
dimension,  promotion  of  all  events  and  attractions  in 
the city? 
To what extent has the communication and promotion 
strategy 
the  use  of  social  media 
target 
successfully  reached 
groups  at  local,  regional,  national,  European  and 
international levels? 

the  communication's 

including 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

EQ7:  To  what  extent  did  the  ECoC  manage  to 
raise  the  necessary  resources?  How  efficiently 
and cost-effectively were such resources used? 

Y 

Y 

Y 

Y 

Y 

24 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

What  was  the  process  of  securing  the  financial 
inputs? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

 What  was  the  total  amount  of  resources  used  for 
each  ECoC  event?  What  was  the  final  financial 
outturn of the year?  

What  resources  were  allocated  to  the  development 
phase? 

the  sources  of 

the 
What  were 
respective  importance  of  their  contribution  to  the 
total?  
To  what  extent  did 
complementary sponsorship? 

financing  and 

the  ECoC 

trigger 

title 

To  what  extent  were  the  inputs  consistent  with  the 
Action and with the application? (special focus on the 
European dimension)  

the 

total  expenditure  strictly 

What  was 
the 
implementation of the cultural programme of the year 
(operational  expenditure)?  What  was  the  proportion 
of the operational expenditure in the total expenditure 
for the ECoC event?  

for 

How 
is  expenditure  broken  down 
programming, administration, marketing etc.)? 

(for  cultural 

What  proportion  of  expenditure  was  used 
infrastructure  (cultural  and 
including renovation)  

for 
infrastructure, 

tourism 

What  were  the  sources  of  funding  for  the  ECoC 
event?  How  much  came 
the  European 
Commission  structural  funds  (e.g.  ERDF  -  European 
Regional Development Fund, ESF – European Social 
Fund)? 

from 

the  selection, 
EQ8:  To  what  extent  were 
monitoring  and  EU  co-financing  procedures, 
introduced  by  Decision 
2006/1622/2006/EC 
efficient? 

To what extent have the mechanisms  applied by the 
Commission  for  selecting  the  European  Capital  of 
Culture  and  the  subsequent  implementation  and 
monitoring  mechanisms  influenced  the  results  of  the 
ECoC event?  

25 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Y 

To what extent has the informal meeting following the 
designation  as  well  as  other  advice  offered  by  the 
panel  and  by  the  Commission  influenced  the  results 
of the ECoC event?  
Is  there  a  need  or  demand  for  the  Commission  to 
provide  cities  with  additional  support  (i.e.  with 
communication, advice or evidence) 
How was the Melina Mercouri Prize used? 

To what extent did the award of the Melina Mercouri 
Prize  create  symbolic  value  for  the  cities  holding  the 
ECoC title?  
EQ9:  To  what  extent  could  alternative  policy 
instruments  or  mechanisms  be  applied?  To what 
the  Action 
total  budget 
extent 
appropriate and proportional? 

the 

for 

is 

Was the total size of the budget sufficient for reaching 
a  critical  mass  in  terms  of  impacts?  Could  the  same 
results have been achieved with less funding? Could 
the same results have been achieved if the structure 
of  resources  and  their  respective  importance  was 
different?  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

To  what  extent  have  the  human  resources  deployed 
for  preparation  and  implementation  of  the  ECoC 
event  been  commensurate  with  its  intended  outputs 
and outcomes?  

Y 

Y 

the  use  of  other  policy 

instruments  or 
Could 
mechanisms 
cost-
greater 
effectiveness?  As  a  result,  could  the  total  budget  for 
the  ECoC  event  be  considered  appropriate  and 
proportional to what the action set out to achieve?  

provided 

have 

Y 

Y 

Table 1.3  Evaluation Questions: Effectiveness 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

EQ10:  To  what  extent  were 
objectives achieved? 

the  EU-level 

Y 

26 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

Y 

Y 

s
i
s
y
l
a
n
A

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Provide  typology  of  outputs,  results  and  possible 
impacts  of  the  action  at  different  levels  (European, 
national, regional etc.) 

To what extent has the ECoC event been successful 
in  attaining  the  objectives  set  (general,  specific  and 
operational) and in achieving the intended results as 
set out in the application or others (refer to list in the 
intervention logic)? 

Was  the  cultural  programme  perceived  as  being  of 
high  artistic  quality?  To  what  extent  did  the  ECoC 
prove  successful  in  bringing  their  chosen  artistic 
themes/orientations to the fore?  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  were  the  highlights  or  innovative  elements  of 
the cultural programme? 

Y 

To  what  extent  did  the  ECoC  title  contribute  to  an 
increased  cultural  offer  in  the  cities  holding  the  title 
(e.g. in terms of scope and scale)? 

What 
impact  has  ECoC  had  on 
composition of cultural audiences in the city? 

the  size  or 

EQ11:  To  what  extent  were  the  ECoCs'  own 
objectives achieved? 

Y 

Y 

Y 

the  delivery  mechanism 

How  did 
improve 
management  of  culture  in  the  city  during  the  ECoC 
event?  (Explore  role  of  Board,  Chair,  Artistic 
Director, decision-making, political challenges, etc.)  

What  was  the  local  approach  to  evaluation  and 
monitoring  (who  carried  it  out,  what  were  the  key 
results or when will information be available)? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  quantitative  indicators  (number  of  visitors, 
overnight stays, cultural participation of people, etc.) 
of  the  social,  tourist  and  broader  economic  impacts 
of the event have been gathered by the ECoC? 

Y 

Y 

Y 

To  what  extent  did  the  ECoC  achieve  the  outputs 
hoped for by the city and as set out in the application 
(refer to list in the intervention logic)? 

Y 

Y 

Y 

Y 

EQ12:  What  impact  has  the  action  had  on  the 
cities? 

Y 

Y 

Y 

Y 

Y 

27 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

To  what  extent  have  the  ECoC  been  successful  in 
achieving  the  intended  impacts  as  set  out  in  the 
application  or  others  (refer  to  list  in  the  intervention 
logic)? 

Y 

Y 

Y 

Y 

To  what  extent  have  specific  objectives  related  to 
social impacts been met? 

Y 

What  improvements  to  the  cultural  and  tourist 
infrastructure  were  planned  and  what  has  been 
realised?    What  were  the  key  milestones  in  this 
process? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

there  evidence  of  other  social 
participation, 

dialogue, 

impacts 

i.e. 
training, 

Is 
improved 
volunteering? 

To  what  extent  were  the  objectives  related  to 
reaching  out  to  all  sectors  of  society,  including  the 
excluded,  disadvantaged,  disabled  and  minorities, 
met? 

What were the most significant economic  outcomes 
of the Capital of Culture experience?  

What have been the impacts of the ECoC event on 
regional development? 

Can impacts on tourism be identified? What was the 
total  number  of  visitors  (from  abroad  and  from  the 
country)  to  the  ECoC  event:  before  the  title  year, 
during the title year, after the title year?  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

To what extent has the implementation of the action 
contributed  to  the  achievement  of  the  objectives  of 
Article 151 of the EC Treaty? 

Y 

Y 

Y 

Y 

Y 

EQ13:  To what  extent  has  the  action  resulted  in 
unintended effects? 

Y 

Y 

Y 

Y 

Are there any instances where the ECoC event has 
exceeded  initial  expectations? What  positive  effects 
has this had?  

Y 

Y 

Y 

Y 

Where expectations have not been met, what factors 
have hindered the development of the action? 

Y 

Y 

Y 

Y 

28 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Table 1.4  Evaluation Questions: Sustainability 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

EQ14: To what extent can the positive effects of 
the  ECoC  Action  be  considered 
to  be 
sustainable? 

Which  of  the  current  activities  or  elements  of  the 
ECoC event are likely to continue and in which form 
after the EU support is withdrawn? 

Y 

Has any provision been made to continue and follow 
up  the  cultural  programme  of  the  ECoC  event  after 
the closure?  

To  what  extent  has 
infrastructure been improved by ECoC? 

the  cultural  offer  and 

Y 

How  will  the  city  continue  to  manage  its  long-term 
cultural development following the ECoC event? 

What  will  be  the  role  of  the  operational  structure 
after  the  end  of  the  ECoC  event  and  how  will  the 
organisational structure change? 

Will  there  be  a  legacy  body  in  place,  how  will  it  be 
funded? 

What  has  been  the  contribution  of  the  ECoC  event 
to improved management of cultural development in 
the city? (in the long-term) 

How  will  partners  and  stakeholders  be  involved  in 
cultural governance post ECoC? 

What  are  the  likely  impacts  of  the  ECoC  event  on 
the long term cultural development of the city?  

What  are  the  likely  impacts  of  the  ECoC  event  on 
the long term social development of the city? 

What  are  the  likely  impacts  of  the  ECoC  event  on 
term  urban  and  broader  economic 
the 
development of the city? 

long 

29 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2.  Details of the relevant evaluation tasks and methodologies:  

Inception phase 

The aim of the inception phase was to finalise the evaluation  framework and research tools, 
address  the  project  steering  group's  comments  regarding  the  proposed  method  and  collect 
background information on the Mons and Pilsen. 

Initial consultations   

The  task  consisted  in  identifying  stakeholders  to  consult  with,  targeting  people  that  are 
knowledgeable about the two cities, their cultural programmes and the achievements, lessons 
learnt  and  impact  of  the  ECoC,  including  national,  regional  and  local  stakeholders.  Careful 
consideration  was  given  to  identify  stakeholders  to  talk  to  who  were  likely  to  have  a  more 
impartial or ‘external’ perspectives of the ECoC rather than simply  speaking to  people who 
are directly associated with the ECoC cultural programmes in Mons and Pilsen.  Discussion at 
the  kick  off  meeting  identified  a  need  for  strong  triangulation  and  to  gain  opinions  from  a 
broader set of stakeholders. 

During  the  initial  first  visit  to  the  cities  described  below,  the  contractors  supporting  the 
evaluation  specifically  sourced  stakeholders  for  consultation  who  did  not  directly  benefit  or 
take  part  in  the  ECoC  programme  and  who  they  felt  would  have  an  honest,  impartial  and 
unbiased view of how the programme was developed or implemented. While the aim was not  
to look for purely negative views, the contractors  wanted to speak with stakeholder who, for 
example: 

  did not deliver an ECoC project or take part in the cultural programme attached to the 

ECoC; 

  were  a  member  of  the  local  and  national  press  who  had  written  either  an  honest  or 
negative piece on the ECoC (as opposed to simply writing a press release which, for 
example, communicated the content and dates of the cultural programme); 

  had applied for ECoC funding (to be part of the ECoC cultural programme) and were 

rejected; 

  were involved in ECoC activity but were often vocal about a particular aspect (e.g. the 
quality  of  culture  on  offer,  the  lack  of  audiences,  the  lack  of  diversity  of  cultural 
content);  

  were not directly involved in the cultural policy agenda in the cities but instead had an 

economic, social or environmental view on the ECoC. 

The list of stakeholders consulted is provided in the annex to the Report32. This list was partly 
developed  by  the  ECoC  delivery  teams  but  also  through  an  internet  search  for  stakeholders 
who  work  in  the  above  fields.  The  list  was  further  expanded  during  the  second  visits 
themselves (through a snowballing technique - i.e. asking stakeholders for other stakeholders 
who could be useful "external" individuals to consult). With this in mind, in Pilsen, the list of 
stakeholders consulted includes: 

32 https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2015-evaluation-
report_en.pdf 

30 

 
 
 
                                                            
 
  a former Programme  Director who left after a difficult early development period for 

the ECoC; 

 

  a Director of a local theatre in the city (who is a key part of Pilsen's cultural offer) who 
did  not  receive  any  ECoC  funding  but  ran  a  range  of  cultural  activities  during  the 
ECoC year; 
two local journalists who were generally critical of the ECoC during its development 
stage  and  who  produced  press  content  that  pointed  to  weaknesses  to  the  proposed 
cultural programme; 
three  Board  members  of  the  Pilsen  ECoC  who  were  generally  vocal  about  the 
shortcoming of the programme during its design.   

 

Within Mons, the list of confirmed stakeholders interviewed includes: 

  a  journalist  from  a  national  newspaper,  which  provided  coverage  of  positive  and 
negative stories relating to the ECoC during the preparation phase and the title year; 
  an  academic  researcher  from  the  Université  catholique  de  Louvain,  who  has 

undertaken independent research into the ECoC and its effects; 

  a  representative  of  the  City  of  Bruges,  who  had  been  involved  in  Bruges2002  and 
managed the Bruges's participation as a "partner city", although Bruges did not receive 
funding from Mons2015; 

  a current member of the regional parliament of Wallonia, who was previously deputy 

Mayor of Mons between 2012-14 but not during the title year. 

The  following  organisations  were  also  invited  to  participate  in  an  interview  with  respect  to 
Mons: 

  ERDF Managing Authority for the Wallonia region; 
  Regional  government  of  Wallonia  (Direction  générale  opérationnelle  de 

l'Économie, de l'Emploi et de la Recherche); 

  Cultural operators that responded to the call for projects and whose projects were 

not selected for co-financing; and 

  Large  foreign  companies  that  have  located  in  Mons  in  recent  years,  e.g.  Google, 

Microsoft. 

First visits to the cities  

Initial  face  to  face  visits  were  made,  to  Mons  on  10  November  and  Pilsen  at  the  end  of 
November 2015. Those consulted with during the initial visit to Pilsen were: 

 
 

the overall Director for the ECoC responsible for its design and implementation; 
the  Programme  Director  for  the  ECoC  responsible  for  the  content  of  the  cultural 
programme and the delivery of some of its larger projects; 

  a  Programme  Manager  for  the  ECoC  programme  responsible  for  partner  liaison  and 

 

the delivery of ECoC projects run centrally by the delivery team; 
the  Manager  of  the  DEPO  2015  project,  often  seen  as  the  most  significant  and 
important project of the programme (see previous chapter for a description). 

More  ad-hoc  and  shorter  discussions  were  also  undertaken  with  five  staff  working  in  local 
cultural facilities which were toured during the first visit. 

31 

 
 
 
 
 
 
 
Those consulted during the initial visit to Mons were: 

  Yves Vasseur (Commissaire général Mons2015); 
  Marie Noble (Commissaire adjoint artistique Mons 2015); 
  Pascal Keiser (Chef de Projets Technologies Mons2015); and 
  Yoann Waroquier (Welcome Team Mons2015). 

As  in  Pilsen,  short  ad-hoc  discussions  were  also  undertaken  with  staff  working  in  local 
cultural facilities which were toured during the first visit. 

These  visits  have  also  allowed  us  to  establish  contacts  with  the  managing  teams,  introduce 
ourselves  and  outline  the  evaluation  approach,  discussing  our  information  and  data 
requirements.  We have also been able to agree the approach to future research tasks including 
the stakeholder interviews and initial information bulletins, as well as witness some cultural 
events and visit key venues, institutions and projects.   

Desk research including big data feasibility 

This involved collection and review of literature, primary data from the delivery agency and 
available secondary data, as well as web and social media statistics. The list of the literature 
that  has  been  collected  at  the  European  level  which  has  been  reviewed  and  assessed  for 
usefulness  for  the  evaluation  can  be  found  in  the  Report33.  Much  of  the  European  level 
literature is helpful to either understand the policy drivers at EU level which support the two 
2015 ECoC or provide background on the bidding and application stages for both 2015 cities 
(i.e.  little  of  it  provides  information  useful  for  the  content,  delivery  and  impact  of  the  two 
programmes).  More  academic  literature  at  the  EU  level  linked  to  the  ECoC  has  also  been 
searched  for.  Instead  of  looking  at  academic  literature  generally  on  ‘European  culture’  and 
also literature before 2015, we specifically looked at up to date (i.e. produced in 2015) pieces 
which linked to the ECoC Action. This search did not prove fruitful, although some academic 
literature at the city level was found. 

To complement the above desk research task we contractor supporting the evaluation assessed 
the extent to which each ECoC has used big data as well as analyse web and social media to 
increase visibility and interest in the ECoC among country residents and internationally. 

Online survey of projects 

The survey of projects has  proved to be an important  source of complementary quantitative 
data  for  previous  evaluations,  and  was  done  for  2015,  supported  by  additional  qualitative 
interviews of projects .The survey was performed by the contractor supporting the evaluation 
for  Mons,  while  for  Pilsen  they  relied  on  the  survey  performed  by  the  implementing  body 
itself. 

Second visits to the cities 

With a clear view emerging from desk research and project survey as to the overall relevance, 
efficiency and effectiveness of each ECoC, the contractor undertook a three day visit to each 
city. The visit focused on conducting in-depth interviews with the managing teams, their key 
stakeholders and cultural operators identified during the initial visits- please see the annex for 
more information.  

33 See previous footnote.  

32 

 
 
 
                                                            
Building on the interviews undertaken in the first visits, the interviews with the delivery teams 
served  four  purposes:  i)  gathering  further  factual  data  and  information  about  the  cultural 
programme  and  its  achievements;  ii)  identifying  the  "story"  of  the  ECoC  throughout  its 
lifecycle,  i.e.  conception,  application,  development,  delivery,  legacy;  iii)  gaining  a  critical 
(albeit "insider") perspective on the relevance, efficiency, effectiveness and sustainability of 
the  ECoC;  iv)  triangulating  emerging  findings,  for  example,  those  emerging  from  the  desk 
research and project survey. 

Interviews with other key stakeholders in the city (e.g. municipality, chamber of commerce, 
commercial  sponsors,  media  representatives,  and  cultural  operators)  were  also    essential  to 
identifying  the  overall  "story"  of  the  ECoC  from  conception,  through  application,  to 
development  and  delivery.  These  individuals  offered  an  external,  alternative  and  broader 
perspective on the ECoC. 

Additional project and stakeholder interviews 

These interviews seek to gather an in-depth understanding of the effect that ECoC had for the 
individuals  involved,  their  organisations  and  the  local  culture  sector  as  a  whole.  It  also 
allowed to ensure that findings are based on consensus across a range of target groups as well 
as  to  identify  key  differences  in  the  experience  of  different  stakeholders.  In  the  case  of 
projects, the criteria for inclusion focused on whether it represents good practice in relation to 
at least one of the key dimensions of the evaluation, including (but not limited to) increased 
European  cooperation,  the  effective  targeting  of  key  groups  of  citizens  or  neighbourhoods, 
audience development strategies, or legacy effects. 

Analysis and reporting phase 

Once  all  the  information  gathered,  the  experts  fulfilled  an  analysis  of  the  available 
information, drawing conclusions and triangulating data to ensure consistency and accuracy. 
The  limitations  of  the  data  gathered  have  been  explained  in  the  report  and  in  the  Staff 
Working  Document.  After  the  first  drafting  exercise,  the  results  have  been  shared  with  the 
ECoC for a factual check. Both the Mons and Pilsen ECoC were asked to undertake a check 
on  the  completeness  of  data  being  used  to  address  each  evaluation  topic.  Their  comments, 
where relevant, were included in the Final Report submitted by the contractor supporting the 
evaluation. 

33