COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex Post evaluation of the 2017 European Capitals of Culture (Pafos and Aarhus)
Date: 2019-06-11
Institutions: Directorate-General for Education, Youth, Sport and Culture, European Commission
Subjects: Cyprus, Denmark, EU action, European cultural event, European symbol, capital city

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 11.6.2019  SWD(2019) 203 final  COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex Post evaluation of the 2017 European Capitals of Culture (Pafos and Aarhus)  {COM(2019) 266 final}  
 
 
Table of contents 

1. 

2. 

3. 

INTRODUCTION ................................................................................................................................ 2 

BACKGROUND TO THE INTERVENTION ..................................................................................... 3 

IMPLEMENTATION / STATE OF PLAY .......................................................................................... 6 

4.  METHOD.............................................................................................................................................. 9 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS .......................................... 11 

6. 

CONCLUSIONS ................................................................................................................................. 21 

ANNEX 1: PROCEDURAL INFORMATION ............................................................................................ 23 

ANNEX 2: STAKEHOLDER CONSULTATION ....................................................................................... 25 

ANNEX 3: METHODS AND ANALYTICAL MODELS .......................................................................... 26 

1 

 
 
 
 
 
 
 
 
 
1. 

INTRODUCTION 

This  document  describes  the  methodology  and  findings  of  the  ex-post  evaluation  of  the 
European Capital of Culture (ECOC) Action for 2017. 

Article 12 of Decision No 1622/2006/EC1 requires that the Commission ensures the external 
and independent evaluation of the results of the ECOC from the previous year. The purpose of 
the yearly evaluation, which the European Commission carries out since 2008, is to assess the 
achievements of the Action in the relevant year, to establish a comprehensive understanding 
of the overall performance and to put forward conclusions, recommendations and lessons for 
future ECOC title-holders, applicants and EU institutions. 

This  working  document  summarizes  the  findings  of  the  external  evaluation  of  the 
implementation of the ECOC 2017, including the selection and monitoring procedures and the 
operational  delivery  by  the  two  hosting  cities,  namely  Pafos  in  Cyprus  and  Aarhus  in 
Denmark2.  The  external  evaluation  examined  how  the  two  cities  developed  their  respective 
applications and cultural programmes, how they delivered the planned initiatives throughout 
the year, and any cultural, general and long-term impact generated by the Action. 

The  evaluation  constitutes  a  valuable  opportunity  to  critically  reconsider  the  past  year  with 
the intention of reshaping insights and broadening wisdom in the light of the experiences of 
the host cities. 

The aim of the evaluation is not to lead to any change in the regulation governing the ECOC 
Action. Indeed those rules were changed in 20143 on the basis of an ad hoc interim evaluation 
of selection and monitoring procedures of European Capitals  of Culture4, but  the new rules 
only  apply  for  cities  designated  as  ECOC  for  the  years  from  2020  to  20335.  It  is  therefore 
impossible to use the outcomes of the evaluation of the two ECOC 2017 to draw conclusions 
on the new legal basis as this new Decision doesn't apply to these two ECOC title-holders. 

1   Decision  No  1622/2006/EC  of  the  European  Parliament  and  of  the  Council  of  24  October  2006  establishing  a 
Community Action for the European Capital of Culture event for the years 2007 to 2019 (OJ L 304, 3.11.2006, p. 1), 
available at:  

- http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:304:0001:0006:EN:PDF. 

2   Full  document  available  at:  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2017-

evaluation-en.pdf 

3   Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union action 
for the European Capitals of Culture for the years 2020 to 2033 and repealing Decision No 1622/2006/EC (OJ L 132, 
3.5. 2014, p. 1), available at: 

- http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2014.132.01.0001.01.ENG 

This  Decision,  which  covers  the  ECOC  titles  2020  to  2033,  retains  the  general  structure  and  main  elements  of  the 
previous Decision while introducing improvements to maximise the benefits of holding the title as well as taking part in 
the competitive process for all bidding cities and their citizens. Improvements include among others the introduction of 
more explicit and measureable criteria, the reinforcement of conditionality for the payment of the Melina Mercouri Prize 
and the obligation for the cities – instead of the Commission – to carry out the ex-post evaluation of the ECOC year. 

4   Available  at:  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/library/2011-capitals-culture-

assignment-report_en.pdf 

5   Article 17 of Decision No 445/2014/EU. 

2 

 
 
                                                           
 
2.  BACKGROUND TO THE INTERVENTION 

2.1.  The European Capital of Culture Action 

The initial scheme of "the European City of Culture" was launched at intergovernmental level 
in  19856.  On  the  basis  of  this  experience,  Decision  No 1419/1999/EC7  established  a 
Community  Action  for  the  ECOC  event  for  the  years  2005  to  2019.  Member  States  were 
ranked  in  a  chronological  order  of  entitlement  to  host  the  event  each  year.  Decision 
No 1419/1999/EC was replaced by Decision No 1622/2006/EC, which kept the principle of a 
chronological  order  of  Member  States  but  further  refined  the  objectives  of  the  Action  and 
introduced  new  selection  and  monitoring  arrangements.  As  mentioned  in  the  previous 
paragraph, a new Decision was adopted by the European Parliament and the Council in April 
2014,  but  cities  which  were  designated  as  European  Capitals  of  Culture  for  the  years  up  to 
2019  continue  to  be  regulated  by  Decision  No  1622/2006/EC.  The  2014  Decision  was 
amended in September 2017 to open the ECOC Action to European  Free Trade Association 
countries which are parties to the Agreement on the European Economic Area8. 

2.2.  Objectives of the ECOC Action 

The ECOC Action aims to highlight the richness and diversity of European cultures and the 
features  they  share,  thereby  promoting  greater  mutual  understanding  among  European 
citizens, as well as to foster the contribution of culture to the long-term development of the 
cities. ECOC shall strive to foster cooperation between cultural operators, artists and cities in 
Europe,  foster  the  participation  in  cultural  activities  of  the  citizens  living  in  the  city  and 
surroundings while raising the interest of citizens from abroad, to be sustainable and to be an 
integral part of the long-term cultural and social development of the city. 

The hierarchy of objectives presented in the table below is based on the objectives as stated in 
Decision  No 1622/2006/EC,  but  it  has  been  updated  to  reflect  the  content  of  the  new  legal 
basis  for  ECOC  post-2019.  The  general  and  strategic  objectives  are  taken  directly  from 
Article 2 of Decision No 445/2014/EU, with the operational objectives flowing logically from 
these.  They  are  also  informed  by  the  selection  criteria  detailed  in  Article  5  of  the  2014 
Decision.

6   Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the 

annual event 'European City of Culture' (85/C 153/02), available at 

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:1985:153:0002:0003:EN:PDF. 

7   Decision  1419/1999/EC  of  the  European  Parliament  and  of  the  Council  of  25  May  1999  establishing  a  Community 
action for the European Capital of Culture event for the years 2005 to 2019 (OJ L 166, 1.7.1999, p. 1). The Decision was 
amended by Decision No 649/2005/EC to integrate the Member States that joined the EU in 2004 (OJ L 117, 4.5.2005, 
p. 20). Both Decisions are available at: 
- http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A31999D1419 
- http://eur-lex.europa.eu/eli/dec/2005/649(1)/oj 

8   Decision (EU) 2017/1545 of the European Parliament and of the Council of 13 September 2017 amending Decision No 
445/2014/EU establishing a Union action for the European Capitals of Culture for the years 2020 to 2033 (OJ L 237, 
15.9.2017, p. 1). Text available at: 
http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32017D1545  

3 

 
 
                                                           
 
Table on ECOC hierarchy of objectives 

General objective 

Safeguard and promote the diversity of cultures in Europe, highlight the common features they share, and foster the contribution of culture to the long-term development 
of cities 

Specific objectives (SO) 

SO1:  Enhance  the  range,  diversity  and  European 
dimension of the cultural offer in cities, including 
through transnational co-operation 

SO2:  Widen  access  to  and  participation  in 
culture 

SO3:  Strengthen  the  capacity  of  the 
cultural  and  creative  sector  and  its 
links with other sectors 

SO4: Raise the international 
profile  of  cities 
through 
culture 

Operational objectives 

Stimulate  a  diverse  range  of  cultural  activities  of 
high artistic quality  

Create new and sustainable opportunities for 
a  wide  range  of  citizens  to  attend  or 
participate in cultural events 

Implement  cultural  activities  promoting  cultural 
diversity, dialogue and mutual understanding 

Implement cultural activities highlighting (shared) 
European cultures and themes  

Involve  local  citizens,  artists  and  cultural 
organizations 
and 
implementation 

development 

in 

Improve cultural infrastructure 

Develop 
governance of the cultural sector 

skills,  capacity  or 

the 

Stimulate 
operation with other sectors 

partnership 

and 

co-

Attract  the  interest  of  a 
broad 
and 
European 
international public  

Involve  European  artists,  promote  cooperation 
transnational 
with  different 
partnerships  

countries 

and 

Provide  opportunities  for  volunteering  and 
foster links with schools and other education 
providers 

Combine  traditional  art  forms  with 
new types of cultural expression 

4 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3. 

IMPLEMENTATION / STATE OF PLAY 

3.1.  The selection and monitoring of the European Capitals of Culture 2017 

In  accordance  with  Decision  No  1622/2006/EC,  Cyprus  and  Denmark  were  entitled  to 
host the ECOC in 2017. The relevant managing authorities of the two countries (i.e. the 
Culture  Ministry  of  Cyprus  and  the  Culture  Ministry  of  Denmark)  run  parallel 
competitions,  which  both  started  in  2011  with  the  publication  of  a  call  to  trigger 
applications from interested cities. 

The selection is then in two phases: a pre-selection phase (candidate cities are reduced to 
a  short-list)  followed  by  a  selection  phase  (one  city  is  recommended  for  the  title).  A 
panel of thirteen members – six of whom nominated by the Member State concerned and 
the  other  seven  by  European  Union  institutions  and  bodies  (European  Parliament, 
Council,  Commission  and  Committee  of  the  Regions)  –  examined  the  bids  from 
candidate cities on the basis of the objectives and criteria laid down in the Decision. 

In  Cyprus,  three  cities  (Limassol,  Nicosia  and  Pafos)  responded  to  the  call  for 
applications by its  deadline of 31 October 2011. Nicosia and Pafos were preselected in 
December  2011.  In  September  2012,  Pafos  was  recommended  at  the  final  selection 
meeting.  In  May  2013,  the  Council  officially  designated  this  city  as  ECOC  2017  in 
Cyprus on the basis of the panel's recommendation. 

In Denmark, Aarhus and Søndeborg entered into the competition by the call’s deadline of 
30 September 2011 and both were pre-selected in December 2011, with Aarhus winning 
over  at  the  selection  meeting  held  in  Copenhagen  in  September  2012.  Finally,  in  May 
2013, the city was designated by the Council as ECOC 2017 in Denmark. 

From  their  nomination  onwards,  Pafos  and  Aarhus  were  subjected  to  monitoring 
arrangements:  the  progress  in  the  cities'  preparations  was  monitored  and  guided  by  a 
panel  composed  of  the  seven  independent  experts  appointed  by  the  European  Union 
institutions  and  bodies,  which  also  checked  compliance  with  the  programme  and 
commitments  on  the  basis  of  which  the  cities  had  been  selected.  Pafos  and  Aarhus 
attended two formal monitoring meetings  with the panel convened by the Commission, 
in November 2014 and April 2016. 

During this monitoring phase, the panel recommended both cities to adequately involve 
all  various  groups  of  citizens  and  to  ensure  a  proper  legacy  of  the  year.  The  main 
difficulty  encountered  by  Aarhus  was  related  to  the  requirement  of  having  an  EU 
dimension  in  their  activities.  For  Pafos  the  securing  of  funds  –  ultimately  leading  to  a 
budget reduction – and the capacity to implement the programme were key issues. 

During the monitoring phase, the two cities introduced modifications into the programme 
described  in  their  original  applications,  in  response  to  a  changing  environment  (in 
particular the budget drop in the case of Pafos) and to the recommendations made by the 
panel. The panel also visited Pafos where it found it appropriate, but decided not to pay 
any visit to Aarhus as preparatory work there progressed smoothly. 

The monitoring process culminated with the panel making a positive recommendation to 
the Commission on awarding a €1.5m prize in honour of Melina Mercouri to both cities 
after the last monitoring meeting. The Melina Mercouri prize was therefore awarded and 

5 

 
 
 
paid by the Commission (under the EU Creative Europe programme9) to the two ECOC 
in the autumn 2016. 

The  sub-sections  below  describe  the  main  features  of  the  ECOC  programmes  of  Pafos 
and  Aarhus.  The  final  report  of  the  contract  supporting  the  evaluation  contains  further 
information, including the way the programmes had a European dimension and involved 
citizens  (as  these  are  the  main  two  criteria  of  the  ECOC  Action)  as  well  as  on  the 
development of the applications and the governance and funding structures. 

3.2. 

Pafos 2017 

Pafos in Western Cyprus dates back to the Neolithic Period. The city and the country as a 
whole  are  at  the  crossroads  of  the  Eastern  Mediterranean  making  it  a  multicultural 
melting pot. Its geographical proximity to Lebanon, Turkey, Syria, Egypt, Palestine and 
Israel,  as  well  as  continental  Europe  characterises  its  culture.  As  a  consequence,  Pafos 
aspired to become the first European Capital of Culture that linked East and West. The 
mythological  past  of  the  city  that  gave  birth  to  Aphrodite,  the  Goddess  of  love  and 
beauty, makes Pafos a mystical destination with many related relics and monuments. As 
this chapter will show, its historical past was a key re-emerging topic of the ECOC. 

With a population of only 35,000, Pafos is one of the smallest ECOC hosts with its size 
being a significant factor affecting how the ECOC was funded, delivered and benefitted 
the  city.  Another  defining  feature  of  Pafos  is  the  number  of  tourists  –  with  3.9  million 
tourists visiting in and around the city in 2017. Half of those tourists were from the UK 
and  a  third  from  Russia10.  Also,  of  significance  is  the  number  of  ex-patriate  residents 
from  the  UK  living  in  the  city  (estimates  range  from  10-25%  of  the  total  population). 
Linked to  the city’s size, the cultural  sector in  Pafos is  relatively small, less  developed 
than in most ECOC host cities and often relates to the city’s past. The city’s architecture 
is often seen as its main cultural ‘offer’ including the Tomb of the Kings, Mosaics, Castle 
and numerous Churches. 

The central idea of the Pafos2017 Programme was based on the ancient tradition, when 
culture developed in open spaces. The ‘Open Culture Factory’ (a term featuring heavily 
in the bid) that formed an important part of the cultural programme promised to travel to 
all areas and communities in Pafos, to display the activities of the Pafos2017 programme 
and  to  create  a  common  space  of  communication  and  cooperation  for  everyone. This 
central  idea  was  not  only  about  open  spaces  but  also  about  openness  in  terms  of 
tolerance,  acceptance,  encouragement  and  integration  of  different  cultures,  ideas  and 
beliefs. 

The  aim  of  the  ECOC  in  Pafos  was  expressed  in  its  motto  “Linking  Continents  – 
Bridging Cultures”. The motto  expressed the need for interconnection  and bridging the 
separated inhabitants of Pafos including permanent residents, visitors and immigrants. It 
also highlighted how important it was to interconnect the scattered areas of the city and 
to turn the entire province of Pafos into a common space shared by all its citizens, both 
literally  and  metaphorically.  Finally,  it  stressed  the  need  to  bridge  the  differences 
between the Greek Cypriot and the Turkish Cypriot community through various projects. 

9   Regulation (EU) No 1295/2013 of the European Parliament and of the Council of 11 December 2013 establishing 
the  Creative  Europe  Programme  (2014  to  2020)  and  repealing  Decisions  No  1718/2006/EC,  No  1855/2006/EC 
and No 1041/2009/EC (OJ L 347, 20.12.2013, p.221). 

10   http://cyprus-mail.com/2017/12/10/paphos-tourism-figures-last-year/ 

6 

 
                                                           
Responsibility  for  the  organization  of  the  ECOC  was  entrusted  to  the  Pafos2017,  a 
Foundation set up by the Municipality of Pafos soon after the city was awarded ECOC 
status.  Pafos2017  was  a  private  non-profit  organisation  and  was  responsible  for  the 
overall control of the ECOC, its budget and the delivery of some of the key projects. Its 
Board  of  Directors  took  charge  of  key  decision  around  resourcing,  priorities  and  key 
projects  and  none  of  these  directors  had  a  direct  political  role.  The  Board  also  had  the 
overall responsibility to ‘make sure the company is working towards its strategic goals, 
in an effective way, both within the allocated budget and transparency’. 

While  being  as  high  as  €23  million  at  the  bid  stage,  the  Pafos2017  budget  was  small, 
amounting  a  total  of  €8.5  million.  Despite  their  lack  of  involvement  in  terms  of 
governance and practical implementation of the cultural programme, the Cypriot national 
government  was  the  biggest  financial  contributor  with  around  €4.5  million  from  the 
Ministry of Education and Culture, making up almost 60% of the total budget. With the 
exception of a few shifts  in  the timeline, the planned and  actual income  from  the local 
and regional government were fully aligned, and amounted to roughly €1 million or 10% 
of the total budget. 

3.3.  Aarhus 2017 

Aarhus  was  founded  in  the  8th  century  as  a  fortified  Viking  settlement  in  a  natural 
harbour  at  the  mouth  of  a  fjord.  For  centuries,  the  primary  driver  for  growth  was 
seaborne trade in agricultural products and by the 1200s, it was significant enough for a 
large cathedral to be built. 

Today Aarhus is Denmark’s second-largest city with a population of 335,684 inhabitants 
and  is  the  cultural  and  economic  centre  of  the  Central  Denmark  Region  (Region 
Midtjylland). In the past ten years, Aarhus has added more than 15,000 new residents to 
its  population  and  created  20,000  new  jobs,  mostly  within  the  knowledge,  service  and 
innovation industries with many important companies having their headquarters there. It 
is  also  an  important  centre  of  research  and  home  to  Scandinavia’s  largest  university, 
Aarhus University Hospital and the INCUBA Science Park. 

Aarhus is notable for its post-war musical history of jazz and rock music and is home to 
several  annual  music  festivals.  The  city  hosts  the  ARoS  Aarhus  Museum  of  Art,  the 
Moesgaard Museum dedicated to archaeology and ethnography, the Aarhus Theatre and 
the Concert Hall amongst other venues. During one week in summer, the city centre and 
parts  of  the  harbour  are  turned  into  one  large  festival  (Aarhus  Festuge)  that  attracts 
visitors from all over the country. 

Aarhus’s ECOC application emerged as part of a bigger plan for the development of the 
city,  which  focussed  on  construction  and  infrastructure  developments  around  the 
seafront,  including  “Dokk1”:  a  new  public  library  and  culture  centre  featuring  artistic 
installations.  The  application  also  aimed  to  promote  more  cohesive  governance  within 
the  Central  Denmark  Region  (CDR),  one  of  five  regions  created  in  2017.  This  new 
region encompasses 19 municipalities with Aarhus as the largest city whilst the regional 
capital is Viborg. 

Aarhus 2017 presented itself under the narrative “Let’s Rethink”. The overall aim was to 
change  mind-sets  through  cultural  experiences  and  to  highlight  three  core  values  of 
sustainability,  diversity  and  democracy.  The  cultural  programme  was  structured  around 
four  seasons,  each  of  which  opened  with  an  outdoor,  large-scale  MEGA  event.  Each 
season also featured three Full Moon events, on a smaller scale than the MEGA events. 

7 

 
The  implementation  and  management  of  the  event  was  carried  out  by  an  autonomous 
foundation:  Fonden  Aarhus  2017  (“Aarhus  2017  Foundation”),  which  became 
operational  as  soon  as  the  city  was  nominated.  The  Foundation  was  composed  of  a 
Secretariat  that  dealt  with  planning  and  coordination  of  the  programme,  budget, 
communication  and  partnerships,  and  a  Board  that  gave  the  overall  direction  to  the 
project. The Board consisted of 13 members of which seven were political appointments 
and  six  represented  civil  society  in  the  form  of  experts  with  strong  links  to  cultural 
institutions and major businesses in the area. 

To ensure that the ECOC maintained its regional dimension, a Regional Steering Group 
was appointed with representation from all the participating municipalities. Furthermore, 
as  Patron  for  Aarhus  2017,  the  Danish  Queen  was  directly  involved,  participating  in  a 
preparatory  video  and  book  preface,  speaking  at  the  Opening  Ceremony  and  attending 
several events.  Likewise, the Prime Minister and the Minister for Culture were actively 
involved. 

The eventual budget of Aarhus 2017 was €61.9m, which was 93% of the total proposed 
in the application. The shortfall arose due to national government funding being €9m less 
than  hoped  at  the  application  stage,  although  some  of  the  shortfall  was  made  good  by 
increased funding from Aarhus Municipality, the CDR and foundations and sponsors. By 
the end of 2016, the group of sponsors consisted of 28 foundations and 116 companies. 
EU funding was received in the form of the Melina Mercouri Prize, as well as funding for 
specific projects from the Culture and MEDIA sub-programmes within Creative Europe. 

4.  METHOD 

In order for results to be comparable with previous evaluations, the methodology for this 
evaluation  closely  followed  the  approach  adopted  in  previous  assessments  of  the 
Action11,  with  focus  being  placed  on  research  at  city  level  and,  in  particular,  on  the 
gathering  of  data  and  stakeholders'  views  from  both  Pafos  and  Aarhus.  The  main 
evaluation sources can be identified as follows: 

-  EU  level  literature:  higher  level  EU  policy  and  legislative  briefings,  papers, 
decisions and other documents relating to ECOC – such as reports of the selection 
and  monitoring  panels  –  as  well  as  academic  research  concerning  the  ECOC 
Action and the role of culture in the development of cities; 

-  ECOC  level  literature  from  Pafos  and  Aarhus:  original  bids  and  applications, 
internal  reports linked to the application, monitoring or evaluation  processes, as 
well as other pieces of literature analysing the cultural programme itself; 

-  Quantitative data: where available, evidence linked to each ECOC was collected 
in relation to budgets and expenditures, projects' numbers and types, participation 
and  audience  figures  as  well  as  other  pieces  of  quantitative  data  relevant  to  the 
assessment of the work and benefits of the ECOC in each city; 
Interviews  with  managing  teams:  those  responsible  for  the  direct  operational 
design and management of the ECOC projects were interviewed both during 2016 
and  again  in  2017  (12  interviews  of  the  Aarhus  team  and  11  interviews  for  the 
Pafos team). Almost all of the key positions linked to the delivery agencies were 
involved  in  the  interviews,  including  those  linked  to  strategic  development, 

- 

11   See 

previous 

reports 
europe/actions/capitals-culture_en (table at the bottom of the document). 

evaluation 

available 

at: 

https://ec.europa.eu/programmes/creative-

8 

 
                                                           
- 

and 

and 

implementation 

communication,  project 

marketing 
management; 
Interviews with key stakeholders12: interviews were undertaken with stakeholders 
both  directly  or  indirectly  involved  in  the  planning,  management  or  delivery  of 
the ECOC along with those more widely linked to the cultural, social, economic 
or political agenda of the host cities. Stakeholders included personnel working in 
cultural  organizations,  in  city/region/State  level  administrations,  in  tourism  and 
visitor  agencies,  in  media  organizations  as  well  as  in  voluntary  and  community 
organizations; 

financial 

-  An open public consultation13, as announced in the road-map14: launched on 11 
March  2018  and  closed  on  21  July  2018,  it  was  open  to  all  people  and 
organisations. However,  some questions ideally  required the respondent  to have 
visited the ECOC city in order to give a response. 

The  final  report  of  the  contract  supporting  the  evaluation  provides  a  detailed 
understanding of the 2017 ECOC Action and within this an assessment of the work and 
progress of Pafos and Aarhus. There are however issues to consider when assessing the 
strengths of the evidence base used for this study: 

-  Although both cities have undertaken some form of evaluation work themselves, 
part  of  the  results  of  these  studies  were  not  yet  available  at  the  time  of  the 
evaluation.  The  European  evaluation  of  the  ECOC  Action  has  used  as  much  of 
this secondary information as possible, but could not benefit from its final results, 
especially for what concerns quantitative data; 

-  As  already  mentioned  in  previous  evaluations,  an  ideal  way  of  conducting  this 
evaluation would entail a before ('baseline') study and an after-picture ('ex-post') 
study, instead of carrying out the latter alone; the impossibility of comparing the 
two studies affects the accuracy of the evaluation. Moreover, since the evaluation 
was  undertaken  during  and  shortly  after  the  end  of  the  ECOC  time  framework 
some  of  the  effects  of  the  programme  had  not  manifested  themselves  entirely. 
Many stakeholders involved in the evaluation commented that the real impact of 
the ECOC on the city and its residents would take time to filter through. In this 
respect, the Commission once again highlights that budget15 and timing16 only 
allow an ex-post evaluation to take place and therefore only an after picture has 
been studied; 

-  The  public  consultation  undertaken  to  gather  wider  views  from  individuals  and 
organisations  on  the  ECOC  acts  as  an  open,  inclusive  and  accessible  route 
through  which  people  can  feed  in  their  thoughts  and  opinions  into  the  ECOC 
evaluation.  The consultation was self-selective meaning anyone could  feed their 

12   A total of 37 stakeholders for Pafos and 30 for Aarhus were interviewed. 

13   The  OPC  received  a  total  of  76  responses  from  across  Europe.  A  number  of  respondents  attended  both  the 
European Capital of Culture (ECOC) in Pafos and Aarhus, which explains why the 56 respondents for Pafos and 
the 31 respondents for Aarhus exceeds the total number of respondents. 

14    https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-2639855_en. 

15   The budget allocated to the evaluation work (75 000 €) is proportionate to the low level of EU funding directly 
provided to the ECOC (i.e. an award based on a recommendation of the panel after the final monitoring meeting 
in the form of the €1.5m Melina Mercouri Prize). 

16   Decision No 1622/2006/EC requires that the Commission conducts the evaluation immediately after the title year. 

9 

 
                                                           
opinions into the evaluation without having to be asked. However, a non-selective 
research method such as the open survey means that anyone (including those with 
a vested interest) can take part and influence the outcome of the survey relatively 
easily.  In  addition,  as  a  consultation  tool,  the  public  consultation  provided  a 
relatively narrow set of findings as only 76 responses were received overall. This 
is  perhaps  understandable  as  completing  the  survey  required  the  respondent  to 
know (and ideally to have attended) ECOC activities in the cities. For this reason, 
the  results  of  the  survey  have  often  been  used  as  supportive  and  sometimes 
contextual  and  have  only  been  used  in  conjunction  with  other  sources  of 
evidence; 

-  An ulterior consequence of the modest yearly budget allocated to the evaluation is 
the fact that the primary evidence data gathering tends to be more of qualitative 
than  quantitative  nature;  while  qualitative  data  still  holds  a  great  importance  in 
the  evaluation,  the  lack  of  diversity  of  data  sources  translates  into  a  lesser 
dependability, for instance, in the process of proving the objective outcomes and 
impacts of ECOC on widening participation in culture. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

This part of the report presents a diagnosis of the implementation of the ECOC Action in 
2017. It draws together the results relating to the two ECOC 2017 and to a lesser extent 
the findings from previous ECOC evaluations. 

All  figures  appearing  in  the  following  sub-sections  have  been  directly  drawn  from  the 
report of the external ex-post evaluation of the two ECOC 2017. The reader will find in 
the  full  text  of  the  document17  many  more  examples  illustrating  the  conclusions 
presented below. 

Very often, examples and figures can illustrate different points made under "relevance", 
"efficiency",  "effectiveness",  "sustainability",  "EU  added  value"  and  "coherence",  and 
are  therefore  sometimes  repeated  in  various  sub-sections.  However,  for  the  ease  of  the 
reading,  repetitions  have  been  limited  to  the  minimum  and  the  reader  is  invited  to 
correlate some conclusions presented below with examples or figures that may have been 
given in previous paragraphs or may be given in following sub-sections. 

5.1.  Relevance 

Findings from the 2017 ECOC evaluation show that the two title-holders have developed 
and  implemented  a  cultural  programme  that  is  consistent  with  the  EU  Treaty  and  in 
particular  Article  16718.  Indeed,  in  2017  the  ECOC  Action  continued  to  help  EU 
Member  States  promote  cultural  diversity  as  well  as  highlight  common  elements,  and 
added  to  the  "flowering"  of  their  –  and  European  –  cultural  scene.  Indeed,  as  will  be 
shown in following sub-sections, in the two cities, thousands of events took place on top 
of  the  usual  cultural  offer,  covering  all  types  of  cultural  and  artistic  disciplines  and 
presenting  various  aspects  of  the  European  cultural  diversity  notably  through  co-
operations with partners from other (European) countries. 

Although  both  cities  clearly  understood  the  strategic  objectives  of  the  ECOC,  the 
programme in Aarhus was more relevant to the EU Action objectives with an articulated 

17   See footnote 5. 

18   Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A12008E167.  

10 

 
                                                           
and  comprehensive  set  of  objectives  that  directly  related  to  all  the  objectives  of  the 
Decision  while  Pafos  objectives  (to  re-connect  the  Greek  and  Turkish  Cypriot 
communities, re-connect the separate parts of the city, strengthen citizen’s identification 
with  the  city  and  enter  into  a  more  frequent  cultural  exchange  with  Europe  and  other 
regions) were narrower in scope. 

The European dimension in Pafos 

Pafos encapsulated well enough a European dimension at project level both in terms of 
highlighting Cypriot culture to Europe and also in terms of bringing culture from other 
European  countries  to  Cyprus.  Under  its  motto  “Linking  Continents  –  Bridging 
Cultures”, Pafos 2017 had three main themes with a clear European resonance: 1) Myth 
and Religion  (Pafos  cultural  heritage and historical  background  as the place of birth of 
civilisation  and  its  evolution  linked  back  to  the  European  civilisation),  2)  World 
Travellers  (in  reference  to  the  interconnections  of  people,  intercultural  dialogue, 
influences  and  interactivity  that  characterises  Pafos  and  Cyprus  as  a  link  between  East 
and  West)  and  3)  Stages  of  the  future  (which  was  about  the  problems  of  the  modern 
world, technology, dreams and hopes, originality and initiatives).  

The cultural programme had a series of projects that showed the European dimension  –  
in terms both of highlighting Cypriot culture to Europe and bringing cultures from other 
European countries to Cyprus. Examples include the International Sculture Symposium, 
the  visit  by  the  Berlin  Philarmonic  or  the  European  Music  Forum,  which  all  proved 
instrumental  in  prompting  debates  and  encounters  between  local  people  and  European 
and international  artists.  An aspect  of the European dimension covered in the bid-book 
but  that  was  less  obvious  in  the  actual  delivery  of  the  cultural  programme  was  around 
bringing  eastern  and  western  Europe  together.  The  bid-book  mentioned  a  number  of 
times about  the unique geographical  position of Cyprus and Pafos between eastern and 
western  Europe  and  the  city’s  proximity  to  places  such  as  the  Lebanon,  Turkey,  Syria, 
Egypt, Palestine and Israel19. In reality, the cultural programme that was delivered had 
less emphasis on this aspect  – although there were a few examples of individual artists 
coming from these countries. 

The “City and Citizens” dimension in Pafos  

Such  dimension  was  reflected  in  activities  that  took  place  across  the  city,  with  an 
emphasis  on  using  open  spaces.  This  was  a  strong  contributing  factor  in  mobilising  all 
citizens  –  both  local  residents  and  the  international  expat  community  –  to  both  re-
discover the cultural heritage of their city; and connect them with the communities living 
in these different spaces. According to the Cultural Programme of Events documentation, 
around  70%  of  projects  were  delivered  outside.  To  accommodate  for  this,  the 
Municipality invested heavily (around 14 million euros) in re-developing key areas of the 
old city centre. These significant changes made to their city in the years leading up to the 
ECOC helped citizens of Pafos embrace the ECOC project from the start.  

Evidence  of  this  can  be  found  in  the  survey  of  local  residents  carried  out  by  the  local 
evaluation.  It  reveals that the respondents’ perceptions on the value and contribution of 
the ECOC to the city of Pafos was high from the outset, and increased as the ECOC year 
progressed. This  survey  showed that 59.5% of respondents  who attended ECOC events 
felt that the added value of the ECOC to the city was very positive and 32.7% thought it 
was positive. The same survey showed that even those local residents who did not attend 

19   https://issuu.com/pafos2017/docs/pafos2017_bidbook. 

11 

 
                                                           
ECOC events also thought the ECOC had high levels of added value with a total of 84% 
of responses saying the effects were either very positive or positive. 

The European dimension in Aarhus 

The objectives of Aarhus 2017 were reflected in a set of “core values”, “concepts” and 
“motivations”  that  determined  the  overall  narrative  and  programming  of  the  title  year. 
One  –  important  –motivation  was  the  “European  dimension”,  which  was  reflected  in 
different ways.  

Firstly,  there  were  performances  by  a  diversity  of  European  and  international  artists  as 
well as exhibitions of international works, exploration of European cultural diversity and 
common history, and connections with European networks. 

Secondly,  to  showcase  the  European  cultural  diversity  and  common  history,  the 
programme  included  events  that  addressed  contemporary  issues,  such  as  the  European 
migration  crisis  and  Brexit.  Key  events  included  for  example  the  Eutopia  International 
Festival, which attracted artists from 23 different countries who performed a diversity of 
works  from  across  Europe  and  beyond  or  the  “Music  Unites  Europe”  festival,  which 
involved  numerous  intellectually-challenged  artists  from  several  different  European 
countries. 

The “City and Citizens” dimension in Aarhus 

Aarhus2017 has a clear “City and citizens” dimension articulated under three of the four 
Motivations of their Strategic Business plan, namely ‘Cultural infrastructure’, ‘Soft City’ 
and ‘City and Citizens’. In practice, the programme of Aarhus2017 used  the city space 
(and  spaces  across  the  region)  in  new  ways,  included  using  the  newly-developed 
waterfront area in Aarhus in ways that were not previously possible. 

It  created  new  opportunities  for  citizens  (with  a  special  focus  on  children  and  young 
people) to participate in culture through different approaches from working with existing 
community  groups  and  schools,  to  engaging  individuals  through  local  advertising  and 
social  media.  Opportunities  were  also  created  for  citizens  to  participate  as  performers. 
Finally,  it  had  an  important  regional  dimension,  with  some  200  events  taking  place 
throughout the Region of Central Denmark. 

5.2.  Efficiency 

Overall,  the  report  of  the  contractor  concludes  that  the  ECOC  remains  an  efficient  EU 
Action, providing good levels of benefits for the EU for relatively little EU investment. 
Indeed, the only direct contribution from the European Union, in the form of the €1,5m 
Melina Mercouri prize awarded to each ECOC, is dwarfed by the total amount of money 
invested by the host cities in designing and delivering the ECOC (the operational budgets 
were approximately €66.7m for Aarhus and € 8.5m for Pafos). 

Without  the  initial  EU  impetus  to  support  the  ECOC,  it  is  unlikely  that  the  host  cities 
would have invested anywhere near the amount of funding they did in  connection with 
the ECOC title. The possibility of securing the title typically20 stimulates cities – but also 
their respective regional and national public authorities – to invest much more heavily in 

20   More information on the Cities objectives when applying can be found in Chapter 3.3 of a European 

Parliament study available at: 

www.europarl.europa.eu/RegData/etudes/etudes/join/2013/513985/IPOL-
CULT_ET(2013)513985_EN.pdf. 

12 

 
                                                           
their  cultural  offer  than  they  would  in  the  absence  of  the  ECOC  –  both  in  terms  of 
infrastructure  (this  was  more  the  case  in  Aarhus  than  in  Pafos)  and  expenditure  on 
cultural events and operations. The independent evaluation concludes that this means the 
Action remains highly efficient in terms of returns from the Melina Mercouri prize.  

On  the  other  hand,  the  final  report  of  the  contract  supporting  the  evaluation  also 
highlights  that  delivering  an  ECOC  is  still  very  challenging  at  the  city  level  from  an 
efficiency  point  of  view,  due  to  the  need  to  build  cross-party  political  support,  secure 
funding  from  diverse  sources  (including  national  government),  build  capacity  for 
implementation and ensure artistic. In this respect, it is worth mentioning that Pafos had 
difficulties in securing the budget initially sought for. On average, in the delivery plan up 
to 60% of the projects’ budget was cut compared to the estimated budgets from the bid-
book  

Notwithstanding the impact of the reduced budget, the working arrangement of the Pafos 
2017  team  was  efficient  and  seen  by  the  external  evaluators  as  a  strong  point  of  the 
programme. All staff members tended to have multiple responsibilities and roles and the 
team was successful in generating a lot of interest from volunteers who played a critical 
role  in  the  programme's  delivery.  The  small  size  of  Pafos  city  meant  that  local  people 
tended to feel responsible for its success and were prouder of it in a way that people in 
other  larger  cities  with  ECOC  status  might  not  experience  to  the  same  extent.  This  is 
evidenced by the high levels of participation in the cultural events by local residents as 
well as the high number (350) of citizens who decided to enrol in the ECOC Volunteer 
Programme.  However, the communications activities of Pafos did not prove effective in 
reaching a wide international audience beyond those international visitors and expatriates 
already present in the city. Pafos implemented a local evaluation, which provided useful 
results but was quite limited in scope and made no use of big data. 

In  Aarhus,  the  arrangements  for  governance  and  implementation  proved  efficient, 
drawing on the strong political support both in the city and across the region and despite 
difficulties  arising  from  the  departure  of  senior  staff  during  the  development  phase21. 
This  is  reflected  in  the  fact  that  the  Foundation  and  its  partners  delivered  a  cultural 
programme of the intended scope, scale and quality, raised the necessary  resources and 
generally  fulfilled  the  expectations  set  for  the  ECOC.  Moreover,  the  regional 
collaboration  also  proved  effective,  given  that  the  ECOC  was  the  first  time  that  the 
different municipalities  of the CDR  had co-operated in  the field  of culture in  this way. 
Indeed, it appears from the evaluation that the  local and regional collaboration partners 
were  actively  involved  throughout  the  planning  process,  provided  important  financial 
backing  and  hosted  many  events  that  showcased  the  varied  culture  and  heritage  of  the 
region. A key factor here included the Regional Steering Group, which brought together 
the Heads of Culture from each municipality and the CDR (and has been continued after 
the  ECOC  year).  Aarhus  also  implemented  a  comprehensive  and  effective  local 
programme of evaluation and research, although the use of big data was limited. 

Aarhus  proved  broadly  successful  in  raising  the  resources  promised  in  the  application. 
Whilst  the  sum  raised  from  some  sources,  e.g.  in-kind  sponsorship,  was  less  than 
proposed, the shortfall was offset by increased from elsewhere. Overall, the award of the 
title  led  to  considerable  additional  investment  in  culture  of  around  €66.7m  during  the 
years 2013-18. Whilst the Melina Mercouri Prize added value, Aarhus has shown that, it 

21  As  expressed  by 

the  panel  during 

the 

first  monitoring  meeting  – 

see 

report  at 

https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/files/ecoc-2017-aarhus-
monitoring_en.pdf. 

13 

 
                                                           
is  the  award  of  the  title  rather  than  the  additional  EU  funding,  which  is  the  trigger  for 
such investment in large or medium-size cities. 

Regarding  the  efficiency  of  the  selection,  monitoring  and  EU  co-financing  procedures, 
the experience of the 2017 has demonstrated some strengths and weaknesses. On the one 
hand, the selection and monitoring procedures, as well as the informal meetings with the 
panel  prove  valuable  in  giving  impartial  advice  and  support  to  the  ECOC  from  highly 
experienced  experts,  some  of  whom  have  implemented  previous  ECOC.  Moreover,  the 
formal  meetings  provide  an  opportunity  for  the  panel  to  hold  the  cities  to  account,  in 
terms of respecting the promises made in their respective applications. 

At the same time, the experience of Pafos shows the limits to the procedures set out in 
the 2006 Decision. More specifically, the procedures have not proved sufficiently robust 
to  prevent  a  reduction  of  60%  in  the  funding  committed  to  the  Pafos  ECOC.  For  that 
reason, the changes introduced by the 2014 Decision (to be applied to the 2020-33 titles) 
are to be welcomed. In particular, it is encouraging to note that, for the 2020-33 titles, the 
Melina Mercouri Prize will only be awarded if the title-holders honour the commitments 
made  at  the  application  stage,  including  maintaining  the  budget  at  a  level  capable  of 
delivering a high-quality cultural programme in line with the application and the criteria.  

5.3.  Effectiveness 

Ultimately, the effectiveness of the ECOC in achieving the objectives set for them (at EU 
level and local level) can only be determined in the long-run. That being said, the final 
report  of  the  contract  supporting  the  evaluation  however  shows  that  both  cities  have 
proved  successful  in  enhancing  the  range,  diversity  and  European  dimension  of  the 
cultural offer in their respective cities during 2017.  

Both  presented  cultural  programmes  that  were  more  extensive,  diverse,  innovative  and 
international  compared  to  the  cultural  baseline  offering  in  previous  years.  In  total,  the 
ECOC cultural programme in Pafos involved 168 separate projects, which emerged from 
both  a  bidding  phase  or  came  out  more  organically  from  national  and  international 
synergies and collaborations. In Aarhus, a total of 442 core projects were implemented, 
corresponding to 13,708 “event days”. Whilst some projects featured the development or 
repeat of activities and events in previous years, the majority of each cultural programme 
was  genuinely  additional.  More  European  and  other  international  artists  were  involved 
compared to  previous  years, some new works were created and performed or exhibited 
and spaces in both cities were used in new ways to host cultural events. 

Both  ECOC  widened  access  to  and  participation  in  culture  during  2017,  although  the 
evidence  is  stronger  in  Aarhus  than  in  Pafos.  Both  ECOC  have  helped  strengthen  the 
cultural  capacity  of  the  local  cultural  and  creative  sectors  and  their  links  with  other 
sectors. The ECOC has raised the international profile of Aarhus through culture, whilst 
in both cities the ECOC has helped make audiences for culture more international.   

Effectiveness in delivering a cultural programme with a European dimension 

The  Aarhus  cultural  programme  was  of  the  scale  and  quality  proposed  in  the  original 
application and had a  genuine European dimension, as shown in  the paragraphs below. 
According to the evaluation, Aarhus 2017 has been successful in contributing to making 
the cultural offering of Aarhus and the CDR more European.  

Indeed,  the  cultural  programme  featured  performances  by  a  diversity  of  international 
artists  and  exhibitions  of  international  works,  strengthened  European  networks  and 
connections, European co-productions and residencies by European artists. According to 

14 

 
data  provided  by  rethinkIMPACTS  2017,  1,200  international  artists  contributed  to  the 
programme.  Some  79%  of  projects  featured  an  international  partner  and/or  a  cultural 
exchange within Europe. 

The performance of Pafos against its objectives was limited by the reduction in its budget 
compared  to  the  figure  proposed  in  the  application.  Reflecting  this,  the  size  of  the 
cultural programme was far less than originally proposed, although more ambitious than 
the yearly annual offering in a city with a low cultural base before the ECOC-year. 

Another  element  of  the  Pafos  2017  European  dimension  was  the  ambition  to  promote 
togetherness (a European value) in order to bring north and south Cypriots together but 
the cultural programme was perhaps too small to effectively deal with this complex issue. 
It did  “start  a conversation” about  coexisting at the individual level but did not make a 
significant impression at a higher level. 

Effectiveness  in  reaching and engaging  with local  citizens, in  targeting  specific groups 
and in strengthening the local cultural and creative sectors 

Both  ECOC  widened  access  to  and  participation  in  culture  during  2017,  although  the 
evidence is stronger in Aarhus than in Pafos.  

Aarhus attracted audiences of 3.3m and ensured that the total audience for culture in the 
city and the region  were greater in  2017 than in  2016. The four MEGA  events  and the 
twelve  Full  Moon  events  attracted  total  audiences  of  around  1.35m.  These  audiences 
were additional to the baseline situation in previous years, since these events were new 
for  the  title-year.  Audiences  for  both  types  of  event  exceeded  the  targets:  for  MEGA 
events, the average audience was 196,722 (against a target of 60,000) while it was 46,847 
for the 12 Full Moon Events (target: 15,000). 

Second,  the  external  evaluation  shows  that  Aarhus  reached  a  high  proportion  of  local 
citizens, with 98% of the city’s inhabitants aware of the ECOC while 60% of people in 
the city and 40% in the region attended at least one event. This was no doubt facilitated 
by the fact that 52% of events were free to attend. Of the respondents to the European-
level public consultation, 68% reported that the Aarhus 2017 encouraged them to attend 
more  cultural  events.  Moreover,  65%  of  respondents  stated  that  the  ECOC  had 
encouraged them to attend a wider type or genre of cultural events in 2017. 

Third, Aarhus 2017 increased the number of citizens involved as creators, performers and 
audiences,  including  children  and  young  people  compared  to  previous  years.  Most 
notably, some 6,000 people actively contributed to the “People’s Opening”. In advance 
of the title-year, some 10,000 residents of the region were consulted.  

Finally, Aarhus 2017 involved many more citizens as volunteers in culture compared to 
previous  years.  The  “Rethinkers”  volunteer  programme  involved  4,535  volunteers  in 
some capacity during 2017. 

In  Pafos,  the  collection  of  audience  data  was  not  comprehensive,  making  it  difficult  to 
grasp the full extent to which access was widened.  

Despite  this  shortcoming,  it  nonetheless  appears  that  Pafos  2017  was  successful  in 
developing the audience for culture. According to the local evaluation, the total number 
of audiences reached 207,250 – a figure to see in the context of a city with a population 
of around 35,000 inhabitants – and around half of stakeholders interviewed reported that 
the city had “become hungry for culture”, which had never been the case before. As an 
example,  it  emerges  from  the  same  local  evaluation  that  58.9%  of  ECOC  participants 
attended more cultural events as the year progressed while for 30.2% their involvement 

15 

 
stayed more or less the same. The local evaluation also reveals there was a good age and 
gender  balance  across  the  audience,  a  mix  of  residents  from  different  parts  of  the  city, 
including  the  international  expatriate  community  living  in  Pafos  and  more  traditional 
“sunseekers”, as well as a high degree of diversity amongst audiences, despite the lack of 
focus on the participation of disadvantaged groups. 

Finally,  the  volunteer  programme  was  another  key  factor  in  mobilising  the  citizens  to 
actively  engage  with  the  programme:  more  than  350  local  and  international  volunteers 
were mobilised for the practical implementation of ECOC activities. 

Both ECOC also helped strengthen the cultural capacity of the local cultural and creative 
sectors and their links with other sectors. Aarhus 2017 developed the capacity of cultural 
operators in the city and the region through various ways. For example, according to the 
Foundation’s  “Welcome  Future”  report,  some  48%  of  projects  aimed  directly  at 
developing  skills  and  competences,  in  line  with  the  “Soft  city”  motivation.  There  were 
also  100  small-scale  cultural  projects  implemented  by  project  managers  aged  under  35 
years and with limited experience of implementing such projects22. It has also provided 
an  increased  sponsorship  of  culture  (the  total  additional  sponsorship  attracted  from 
corporate sponsors and foundations amounted to €14.9m) and brought positive economic 
benefits  to  the cultural  and creative sectors.  In Pafos, both  the stakeholders’ interviews 
and the Open public consultation reported that the title developed the skills and capacity 
of  local  cultural  operators.  Around  three  quarters  of  stakeholders'  interviewed  revealed 
that, for most local artists and project coordinators, the ECOC was their first opportunity 
to get involved in projects at such a large scale, and to have such a large and international 
audience for their artistic products. 

Effectiveness in reaching out to European and international audiences  

According to the evaluation, the ECOC raised the international profile of Aarhus through 
culture, whilst in Pafos, the ECOC helped make audiences for culture more international.  

Aarhus 2017 attracted considerable attention from  national  and international  media and 
from  social  media,  raised  the  on-line  international  profile  of  Aarhus  and  increased  the 
number of domestic and international visitors to the city and the CDR during 2017. Data 
from  the  Foundation  shows  that  there  were  27,723  media  mentions  in  2017,  of  which 
2,528  were  international  (with  some  95%  of  media  mentions  of  a  positive).  It  also 
appears that  Aarhus 2017 featured in  media articles in  70 countries. Furthermore, from 
2016  to  2017,  the  number  of  overnight  stays  in  the  CDR  increased  by  4.0%  while  the 
number  of  hotel  bed-nights  increased  by  10.9%  and  the  number  of  hotel  bed-nights 
accounted for by foreign visitors increased by more than 21,000, equivalent to 8.4% and 
there was an increase of 24.4% in the number of cruise ship visitors arriving in Aarhus in 
2017. 

The  efforts  to  promote  Pafos  2017  internationally  were  quite  limited  and,  as  a  result, 
awareness of the ECOC amongst visitors to Cyprus was very low. However, the ECOC 
was  successful  at  attracting  expatriates  and  foreign  tourists  once  they  had  arrived  in 
Pafos, with  non-Cypriots  accounting for almost  40% of audiences. This  is significantly 
higher than other ECOC, where foreign visitors typically make up less than 10% of total 
audiences. 

It  is  to  be  noted  that  Pafos  2017  was  not  so  much  about  increasing  the  international 
profile  of  the  city  (which  was  already  high  before  the  ECOC  year  with  many  foreign 

22   Welcome Future, Short-term impact of European Capital of Culture Aarhus 2017, Aarhus 2017 Foundation. 

16 

 
                                                           
tourists visiting the city or expats living in the city) as it was about changing it. This is 
evidenced  by  the  fact  that  there  were  no  press  releases  in  the  ECOC  city  or  the 
international  travel  press  or  advertising  in  overseas  locations  to  attract  more  overseas 
visitors  to  Pafos.  The  focus  of  Pafos  2017  was  more  about  encouraging  the  local 
population and tourists already in the city to take part in the ECOC and the wider purpose 
was to  increase the potential  of the old  city centre as a ‘cultural  vector’  for tourism, in 
addition to the more ‘traditional beach tourism’ which Pafos has always been known for.  

Effectiveness in ensuring sustainability 

The  two  2017  ECOC  cities  offer  very  different  potential  for  the  sustainability  of  their 
activities and of improved cultural governance.  

Pafos will see some long-term benefits but little has been done to ensure these benefits 
are  maximised.  Whilst  some  of  those  effects  might  endure  –  and  funding  has  been 
provided  in  both  cities  for  legacy/continuation  projects  –  this  will  rely  on  continued 
partnership working by the various stakeholders, since neither city is planning a specific 
legacy body.  

Although the Pafos ECOC year generated a lot of interest and excitement during 2017, 
this  was  already  diminishing  in  early  2018  as  staff  posts  and  projects  stopped  being 
funded.  Soon  after  December  2017,  projects  had  disappeared  and  momentum 
(particularly in terms of volunteers) was on a sharp decline. The lack of any attempt to 
have a legacy plan developed even during the year is a key weakness of the ECOC.  

Because the cultural programme was relatively small it needed to have a long-term plan 
in place to make a meaningful step change in the cultural offer of the city. If the ECOC 
had been  awarded to  another city with  a larger  existing cultural  infrastructure, then the 
level of impact in the city may well have been higher than it actually was in Pafos. 

On a more positive note, Pafos 2017 also served as a catalyst for cultural infrastructure 
investments, with a range of new infrastructure developments and renovations including 
the Ibrahim’s Kahn, the Attikon and the Markideion theatre, and these new venues will 
continue working in the future, providing Pafos 2017 with some legacy.  

For  what  concerns  Aarhus,  the  ECOC  project  generated  potential  for  long-term  impact 
through  the  skills  and  experience  gained  by  cultural  operators,  the  involvement  of 
citizens, increased audiences and greater international profile.  

Planning for the legacy of Aarhus 2017 began before the title-year, including through a 
consultation  process  that  took  place  from  May  2016  to  January  2017.  The  consultation 
consisted  of  meetings,  interviews  and  workshops  with  more  than  100  stakeholders.  It 
culminated  in  the  publication  of  a  legacy  document:  “Our  legacy:  a  new  beginning”, 
which  was  published  by  the  Foundation  Board  in  March  2017.  Given  the  early 
publication  of  this  document,  the  focus  was  mostly  on  those  effects  that  were  likely  to 
endure, rather than legacy plans per se. 

In  Aarhus,  several  indicators  based  on  data  from  the  Visit  Aarhus  Tourism  Barometer 
shows a slowing in growth in tourist visits and cultural audiences after the title-year, as is 
the  case  for  most  title-holders.  However,  tourist  visits  and  cultural  audiences  were  still 
higher in the six months after the title-year (Jan-Jun 2018) compared to the same period 
in the year before the ECOC (2016), which suggests the potential for long-term impact 
compared to the “baseline” situation prior to the ECOC. 

The  continuation  of  key  projects  after  the  end  of  the  ECOC  year,  such  as  the  ARoS 
Triennial  and  the  Children’s  International  Festival,  offers  the  potential  for  long-term 

17 

 
impact,  as  does  the  continuation  of  the  volunteer  programme.  Whilst  much  of  the 
experience and expertise will be retained in the city and the CDR, the lack of a specific 
legacy  body  and  an  overall  continuation  strategy  risks  a  loss  of  momentum.  More 
promisingly, the potential for continued regional collaboration in the field of culture look 
promising, given the new structures put in place. 

5.4.  Coherence  

The ECOC Action is coherent and complementary to the Creative Europe Programme in 
that it promotes the objectives of Creative Europe and is distinct from the other activities 
supported by the programme. Aarhus and Pafos also made use of funding from Creative 
Europe  to  support  projects  in  their  cultural  programmes.  Funding  from  the  Europe  for 
Citizens programme also helped Aarhus continue its volunteer programme in 2018. 

The ECOC Action is also coherent with and complementary to the European Structural 
and Investment Funds, depending on the context of each city holding the title. In the case 
of  Pafos,  ERDF  was  used  in  the  years  leading  up  to  2017  to  co-finance  essential 
investments in the refurbishment of the city centre and renovation of key venues, such as 
a theatre and a cinema. Those investments thus supported the successful implementation 
of the ECOC cultural programme, whilst the ECOC helped “valorise” the investments by 
ensuring that the venues were used and given greater profile. 

EU cross-programme funding also helped maximize the results of the ECOC Action. For 
instance, Aarhus 2017 featured some links to other EU programmes and initiatives, albeit 
modest. The Creative Europe (MEDIA sub-programme) co-funded the DICTE III crime 
and  drama  series,  which  is  set  in  Aarhus  and  featured  in  the  cultural  programme  of 
Aarhus  2017.23  The  European  Capital  of  Volunteering,  which  is  not  an  EU  title  but  is 
supported  by  funding  from  the  EU’s  Europe  for  Citizens  programme,  was  awarded  to 
Aarhus for 2018. This provided additional impetus and profile to the continuation of the 
ECOC volunteer programme. 

5.5.  EU added value 

As  already  mentioned  and  illustrated  above,  the  ECOC  Action  has  achieved  an  impact 
that would not have arisen through the actions of Member States alone. 

The  designation  of  Pafos  and  Aarhus  as  ECOC  has  attracted  benefits  that  would  have 
been  unlikely  to  arise  to  the  same  extent  in  the  absence  of  the  Action.  Indeed,  the 
evidence presented in the chapters here above suggest that the ECOC, as an EU initiative 
and  an  EU  branding,  provides  the  stimulus  for  stakeholders  to  commit  resources  and 
effort to a shared vision and collaborative programme at a much greater scale than would 
otherwise  happen.  The  EU  added  value  consists  to  a  great  extent  in  the  force  of  the 
'brand' itself to act as a significant generator of interest from stakeholders not only from 
the city and but also from far beyond. 

This is evidenced, for example, by the creation of dedicated organisations to implement 
the ECOC and the allocation of resources that are additional to the “mainstream” funding 
for cultural activities in the cities and regions concerned. 

Regarding the visibility of the EU, both title-holders gave full prominence to the title of 
“European Capital of Culture” in their communication and promotional materials. Aarhus 
also made extensive use of the EU logo and gave prominence to the fact that the ECOC is 
an EU Action. 

23   http://www.aarhus2017.dk/en/programme/film-and-animation/dicte-3/  

18 

 
                                                           
The  results  consistently  achieved  by  the  Action  through  the  years  establish  that  its 
models  and  mechanisms  –  for  example  in  terms  of  building  a  solid  governance  for  the 
delivery of the ECOC, increasing the capacity of local cultural organizations or attracting 
projects'  ideas  from  local  residents  –  can  be  applied  not  only  within  one  individual 
Member  State  that  has  its  administrative  and  cultural  specificities  but  in  any  Member 
States finally confirming its EU added value.  

6.  CONCLUSIONS 

The Commission concludes that the ECOC Action is highly valued by the hosting cities 
that can obtain positive impacts during the year, as well as during the preparation phase. 
The action also remains relevant at EU level. However, the long-term impacts of the two 
ECOC 2017 cannot be assessed at this stage, as it is too early after the implementation of 
the ECOC year. 

The Commission also concludes that the programmes implemented by the two 2017 title-
holders were consistent with the objectives of the ECOC Action: 

-  They  reflected  its  European  dimension:  especially  in  Aarhus  with  for  example 
79%  of  projects  featuring  an  international  partner  and/or  a  cultural  exchange 
within  Europe  while  Pafos  2017  was  successful  in  mobilising  its  European  and 
international expats and in reaching out its foreign visitors; 

-  They  involved  local  residents  and  stakeholders:  in  Aarhus,  the  volunteer 
programme  was  one  of  the  largest  of  any  ECOC  to  date  with  4  535  citizens 
volunteering for the year, Pafos did a good job in terms of involving the city and 
its residents in the actual delivery of the programme, with for example 59% of the 
audience coming from the city itself and local hotels donating 3,000 free nights to 
visiting artists; 

-  They  widened  access  to  and  participation  in  culture  during  2017,  although  the 
evidence is stronger in Aarhus than in Pafos; Both ECOC also helped strengthen 
the cultural capacity of the local cultural and creative sectors and their links with 
other  sectors:  almost  half  of  the  Aarhus  2017  projects  aimed  directly  at 
developing  skills  and  competences,  in  line  with  the  “Soft  city”  motivation.  In 
Pafos,  the  coordinators  of  individual  ECOC  projects  and  their  local  volunteers 
increased their skills as they had a lot more responsibilities when it came to the 
implementation of their activities compared with other ECOC; 

-  The  ECOC  raised  the  international  profile  of  Aarhus  through  culture,  whilst  in 
both cities the ECOC helped make audiences for culture more international; 
-  Both ECOC year may lead to some legacies both physical (in the form of new or 
refurbished  cultural  and  logistic infrastructures,  as it happened in  both  cities) or 
intangible (by using open spaces, as Pafos did) although a proper legacy planning 
is lacking in Pafos (the legacy issue is exacerbated in Pafos because the cultural 
sector does not have enough scale to automatically ‘take on’ new and additional 
cultural  projects  without  the  injection  of  funding  and  interest  an  ECOC  year 
brings).  Furthermore,  in  Aarhus,  the  ECOC  was  the  first  time  that  the  different 
municipalities of the CDR had co-operated in the field of culture in this way, and 
this collaboration is due to continue. 

The  Commission  also  considers  that  the  scope  and  length  of  the  yearly  evaluation  it 
carries out in accordance with  Article 12 of Decision No 1622/2006/EC  does not allow 
consideration  of  the  long-term  impact  of  ECOC.  It  is  therefore  willing  to  review  such 

19 

 
 
impact  as  part  of  the  evaluation  exercise  foreseen  in  Article 16  of  Decision 
No 2014/445/EU. This  evaluation  “shall focus on placing  all past  European Capitals  of 
Culture in a European context, allowing comparisons to be drawn and useful lessons to 
be  learned  for  future  European  Capitals  of  Culture,  as  well  as  for  all  European  cities”. 
Building on the results of the study published in December 2017 on "European Capitals 
of  Culture:  Success  Strategies  and  Long-Term  Effects"24  and  will  make  it  possible  to 
analyse longer-term impact. 

24   See: 

http://www.europarl.europa.eu/RegData/etudes/etudes/join/2013/513985/IPOL-CULT_ET(2013)513985_EN.pdf 

20 

 
                                                           
  
ANNEX 1: PROCEDURAL INFORMATION 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

The evaluation was led by Directorate General for Education, Youth, Sport and Culture 
(DG EAC). It is included in the Work Programme of Creative Europe for 2017 and in the 
Agenda Planning with the reference  EAC - PLAN/2017/1264. 

2.  ORGANISATION AND TIMING 

The evaluation was supported by an external and independent evaluator, under a service 
contract.  The  service  contract  was  implemented  via  a  Framework  Contract  with 
reopening  of  competition  and  in  accordance  to  the  Financial  Rules  Applicable  to  the 
General Budget of the Union25 and its Rules of Application26. 
The evaluation roadmap was adopted on 24 May 201727. 

According  to  the  roadmap,  a  Steering  Committee  including  staff  from  the  European 
Commission’s  Directorate  General  for  Education,  Youth,  Sport  and  Culture  and 
Secretariat  General  was  established  in  May  2017.  The  Steering  Committee  met  in  four 
occasions:  to  prepare  the  Terms  of  Reference  (ultimately  sent  to  contractors  in  July 
2017), to approve the Inception Report in January 2018, to discuss the draft final report 
in July 2018 and to approve the final report in October 2018. Extensive correspondence 
between the Steering Committee members was held in between the meetings to follow-
up on the evaluation. 

3.  EXCEPTION TO THE BETTER REGULATION GUIDELINES 

No exception. 

4.  EVIDENCE, SOURCES AND QUALITY 

The 2017 evaluation of the ECOC used a series of data sets to inform its findings. The 
main ones being: 

- 

Interviews  with  over  67  stakeholders  directly  and  indirectly  involved  in  the 
planning, operation and delivery of the two ECOC programmes; 

-  A literature review of ECOC and European level information on the two ECOC 
information,  EC  Committee  reports,  cultural 
including  application/  bid 
programme  brochures,  web  sites  and  news  articles.  The  ECOC's  own  external 
evaluations have also been used to inform the European evaluation process; 

-  An Open public consultation gathering the view from 76 respondents; 
-  Together,  the  above  evidence  base  provides  the  evaluation  with  a  valid  and 
rounded  set  of  data  to  inform  the  views  on  the  main  aspects  of  the  ECOC 
evaluation  including  efficiency,  effectiveness,  sustainability  and  relevance.  This 
view has been informed by: 

25   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R0966-20160101&from=EN 

26   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R1268-20160101&from=EN 

27   https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-2639855_en 

21 

 
 
                                                           
  The scale of the consultation exercise. Around 150 individuals have fed their 
views  and  opinions  into  the  evaluation  process  either  through  face  to  face 
interviews, telephone interviews or through; 

  The nature of the consultation exercise. The evaluators were keen to consult 
with  those  who  had  a  more  indirect  and  external  view  of  the  two  ECOC. 
These  stakeholders  including  journalists,  those  not  directly  benefitting  from 
the  ECOC  (e.g.  rejected  projects)  as  well  as  those  working  in  the  wider 
cultural policy agenda at regional and city level. This ensures the evaluation is 
not simply based on those who benefitted the most from the ECOC. 

22 

 
 
ANNEX 2: STAKEHOLDER CONSULTATION 

1. 

 METHODOLOGY USED  

The consultation activities described in this synopsis report were conducted in the context of 
the evaluation of the European Capital of Culture (ECOC) 2017. The consultations sought to 
collate 
the  evaluation  criteria  (relevance, 
effectiveness,  efficiency,  sustainability,  coherence  and  EU  added  value).  The  relevant 
stakeholders were mapped at the early stage of the evaluation and are described below. 

information  and  stakeholders’  views  on 

2. 

OVERVIEW OF CONSULTATION METHODS AND CONSULTEES 

The  stakeholder  consultation  methods  used  consisted  of  in-depth  interviews  and  an  open 
public  consultation  (OPC).  Evidence  from  the  consultations  was  triangulated  evidence  from 
other  sources,  including  documentation  and  data  linked  to  the  ECOC,  as  well  as  data  from 
other  sources,  such  as  tourist  bodies  and  national  statistics.  The  table  below  provides  an 
overview of the types of stakeholders concluded via each method. 

Type of 
consultation 

  In-depth 

interviews 

  Open  public 
consultation  

Type of consultee 

Timing 

  National authorities 
  Regional authorities 
  Local authorities 
  Foundations 
  NGOs 
  Companies (including SMEs) 
  Cultural bodies & institutions 
  Tourism bodies & agencies 
  Media 
  Academic experts 
  Other 
The  online  consultation  was  open 
to any interested party or individual 
over  a  period  of  12  weeks, 
available in English. 

Q4/2017 – Q2/2018 

Q1/2018 - Q2/2018 

IN-DEPTH INTERVIEWS 

Two main types of in-depth interviews were undertaken: 

 

Interviews with managing teams: those responsible for the day-to-day design and delivery 
of the ECOC were interviewed in each city during city visits in late 2016 (i.e. during the 
host  year)  and  in  spring  2017.  Almost  all  of  the  key  individuals  within  the  delivery 

1 

 
 
 
 
agencies were interviewed including those linked to strategic development, marketing and 
communication, project implementation and financial management.  

 

Interviews  with  key  stakeholders:  mainly  face-to-face  interviews  were  undertaken  with 
stakeholders both directly and indirectly involved in either the planning or delivery of the 
ECOC along with those more widely linked to the cultural, social, economic or political 
agenda  of  the  host  cities.  Stakeholders  included  those  working  in  cultural  organisations, 
city/regional/national  administrations,  tourism  and  visitor  agencies,  media  organisations 
as  well  as  voluntary  and  community  organisations.  Managers  of  individual  projects  and 
activities  supported  through  the  ECOC  Action  that  made  up  the  cultural  programme  of 
each city were also interviewed. 

The table below specifies the number of each type of interview. 

Type of in-depth 
interview 

Number of each type 

Timing 

  Aarhus 

  ECOC managing team: 12 

Q4/2017 – Q2/2018 

  Stakeholders: 18 

  Pafos 

  ECOC managing team: 11 

Q1/2018 - Q2/2018 

  Stakeholders: 26 

OPEN PUBLIC CONSULTATION 

The  OPC  was  launched  on  11  March  2018  and  closed  on  21  July  2018.  It  was  open  to  all 
people  and  organisations.  However,  some  questions  ideally  required  the  respondent  to  have 
visited the ECOC city in order to give a response. The OPC received a total of 76 responses 
from across Europe. A number of respondents attended both ECOC. 

The public consultation gathered a large number of responses from individuals, responding in 
their own personal capacity 84% (N=64), whilst 16% (N=12) responded in their professional 
capacity on behalf of an organisation. 

The largest group of respondents were local residents living in Pafos, 36% (N=23), whilst less 
than half of this number, 16% (N=10), were local residents living in Aarhus. A relatively high 
number were citizens  responding from  elsewhere in  Cyprus:  22% (N=14), in  comparison  to 
those  responding  from  elsewhere  in  Denmark:  2%  (N=1).  People  outside  of  Denmark  and 
Cyprus constituted 25% (N=16) of those who responded.  

Among  the  respondents  who  replied  in  their  professional  capacity,  the  largest  number  of 
respondents (N=4) represented their local/regional or national government in Denmark, whilst 
no one from Cyprus responded in this capacity. Representatives from a national organisation 

2 

 
 
in  Cyprus  and  a  local  organisation  in  Pafos  were  both  the  second  most  common  type  of 
respondent (N=2 for both). 

A large number of respondents (N=32) had heard of the ECOC Action, as well as Pafos 2017 
and Aarhus 2017. Some 15% (N=11) had only heard of the ECOC Action. 

Most respondents (N=38) attended a cultural activity in Pafos during the 2017 ECOC, whilst 
22% (N=17) attended an event in Aarhus. A number of respondents attended an activity both 
in  Pafos and in  Aarhus (N=11),  whilst  only 13% (N=10) of respondents  had not  attended  a 
cultural  event  in  either  city.  The  majority  of  respondents  (N=38)  attended  more  than  12 
activities during 2017, whilst only 17% (N=11) only attended 1-3 activities. 

3.   MAIN TOPICS AND VIEWPOINTS EMERGING 

3.1   RELEVANCE 

Neither  the  OPC  nor  the  in-depth  interviews  directly  requested  consultees  to  provide  an 
opinion on questions of relevance. 

3.2   EFFECTIVENESS 

Both the OPC and the in-depth interviews gathered views on the effectiveness of the ECOC 
against the four specific objectives (SO) of the ECOC Action 

Aarhus 

The  results  of  both  types  of  consultation  activity  were  broadly  similar  regarding  the 
effectiveness of Aarhus ECOC, although the in-depth interviewees tended to be slightly more 
positive than the respondents to the OPC were. 

SO1:  Enhance  the  range,  diversity  and  European  dimension  of  the  cultural  offer  in 
cities, including through transnational co-operation 

In Aarhus, the interviewees generally agreed that the ECOC cultural programme was of high 
quality and that it was of greater scale and had more of a European dimension compared to 
the  city’s  baseline  offering  in  previous  years.  This  finding  was  supported  by  the  OPC 
respondents.  Some  73%  felt  that  the  quality  of  the  cultural  programme  was  very  strong  or 
strong and only 3% felt it was weak or very weak. Some 66% reported that the number and 
distribution  of  cultural  projects  was  strong  or  very  strong,  whilst  only  3%  felt  it  was  weak. 
Some  64%  stated  that  the  type  and  scope  of  cultural  projects  was  strong  or  very  strong  in 
Aarhus and only 6% reported it was weak or very weak. 

There  was  a  consensus  amongst  the  in-depth  interviewees  that  the  cultural  offer  of  the  city 
and the region would be better in future years as a result the ECOC. This was supported by a 
majority of respondents  to  the OPC;  some 52%  reported that  the ECOC  had a high or very 
high impact on Aarhus in terms of increasing the range and diversity of the cultural offered by 
the city. Only 3% of OPC respondents stated that there was no impact. 

3 

 
SO2: Widen access to and participation in culture 

In Aarhus, the consultees agreed that the ECOC involved more citizens (including children) 
as  creators,  performers  and  audiences.  They  tended  to  report  that  the  ECOC  had  expanded 
existing audiences and created new ones. This finding was supported by the OPC respondents 
in  terms  of  their  own  attendance  at  cultural  events,  with  68%  reporting  that  the  ECOC 
encouraged  them  to  attend  more  cultural  events  in  2017.  A  similar  proportion  of  OPC 
respondents  65%  also  reported  that  the  ECOC  encouraged  them  to  attend  a  wider  type  or 
genre of culture in 2017 than in previous years. 

However,  the  OPC  respondents  were  slightly  less  positive  about  the  effectiveness  of  the 
Aarhus ECOC in increasing access to culture for people in general. Some 45% reported that 
the ECOC had a high or very high impact on the number of people accessing culture, whilst 
another 26%  reported a  moderate impact.  Similarly, 36% believed the ECOC had a high or 
very  high  impact  on  introducing  more  people  in  the  city  to  culture  who  do  not  normally 
access it, with another 32% reporting a moderate impact. 

SO3: Strengthen the capacity of the cultural and creative sector and its links with other 
sectors 

There  was  a  consensus  amongst  consultees  that  the  ECOC  helped  strengthen  the  cultural 
capacity of the local cultural and creative sectors and their links with other sectors. This was 
broadly supported by the OPC respondents: 55% reported that the ECOC had a high or very 
high impact in terms of strengthening the city’s capacity in the cultural sector, whilst another 
19% reported a moderate impact. Similarly, 63% of OPC respondents reported that the ECOC 
had  a  high  impact  on  helping  local  cultural  operators  to  co-operate  with  European  partners 
and only 3% felt it had no impact. 

SO4: Raise the international profile of cities through culture 

In Aarhus, consultees also reported that the ECOC raised the international profile of the city 
through  culture.  They  tended  to  highlight  the  increase  in  international  tourist  visits,  the 
attention given by the international media and the response from social media users. 

This finding was supported by the majority of respondents to the OPC consultation. In total, 
80%  of  respondents  believed  that  the  ECOC  had  a  moderate  to  a  very  high  impact  on  the 
number of international visitors (of which 29% reported a very high impact). Similarly, 71% 
of  respondents  saw  the  ECOC  as  having  a  moderate  to  very  high  impact  on  increasing  the 
number of Danish visitors to the city. Only 6% believed that the ECOC did not increase the 
number of visitors. 

The OPC provided a slightly contradictory finding on the question of whether the ECOC had 
encouraged them to visit Aarhus more often in 2017: only 48% reported that it had. However, 
this  finding  is  perhaps  not  reliable  as  includes  local  residents  as  well  as  residents  of  other 
parts of Denmark and of other countries. 

4 

 
 
Pafos 

The  results  of  Pafos  were  similar  across  the  different  consultation  methods  although  the 
results of the OPC were more positive.   

SO1:  Enhance  the  range,  diversity  and  European  dimension  of  the  cultural  offer  in 
cities, including through transnational co-operation 

Stakeholders  in  Pafos  general  felt  that  their  ECOC  increased  the  range  and  diversity  of  the 
cultural offer mainly because the cultural offer in the city was comparatively limited prior to 
the ECOC year. Stakeholders felt that the quality of the offer was particularly limited as there 
were very few high profile artists, venues or other infrastructure in place pre- 2017. Through 
additional funding, larger audiences and the introduction of new producers and artists (often 
from Nicosia and elsewhere in Cyprus) the artistic quality increased during the ECOC year. 
The OPC survey backed this up by saying that 58.9% of ECOC participants felt that cultural 
activity in the city had increased significantly as a consequence of the ECOC status. Further 
evidence from the open public consultation shows that 73% of respondents felt that the ECOC 
year  had  seen  an  increase  in  the  number  and  distribution  of  the  cultural  projects.  84%  of 
respondents to the same survey state that the ECOC had encouraged them to attend more and 
a wider type of culture in 2017 than before.         

Both  interviews  and  the  OPC  results  generally  supported  the  view  that  Pafos  had  a  strong 
European  dimension  although  the  OPC  results  were  generally  more  positive  in  this  respect. 
Stakeholders  highlighted  a  number  of  examples  of  projects  that  promoted  a  European 
dimension (e.g. the International Sculpture Symposium) and also highlighted ways in which 
the Foundation promoted the importance of projects having a European dimension. The OPC 
also  gave  further  and  very  positive  insight  on  people’s  opinions  around  the  European 
Dimension. 52% of respondents stated that they felt the ECOC had either had a very high or 
high impact on promoting European cultures compared to 20% for Aarhus. In addition, 59% 
of  respondents  felt  that  the  ECOC  had  helped  promote  local  cultures  from  the  host  city 
compared to 37% in Aarhus. 

SO2: Widen access to and participation in culture 

Stakeholders and the OPC were generally positive about the ECOC widening participation in 
culture  although  they  showed  little  knowledge  of  the  ECOC  specifically  targeting  harder  to 
reach groups who traditionally did not access culture. Some of the culture  took place in the 
open air which helped to open up access to all of the communities as there was no ticketing 
nor many event taking place ‘behind closed doors’.    

Results  from  the  public  consultation  show  that  the  ECOC  had  a  more  positive  effect  on 
encouraging people to attend more cultural events in 2017. 80% of respondents said that the 
ECOC  encouraged  them  to  attend  more  culture  (compared  to  68%  in  Aarhus).  In  addition, 
84% of respondents said that the ECOC encouraged them to attend a wider type or genre of 
cultural events in 2017 and 76% said that the ECOC encouraged them to visit the city more 
often.  Finally,  59%  of  respondents  to  the  open  survey  said  that  the  ECOC  in  Pafos  had 
increased the number of people accessing culture (by a ‘very high’ or ‘high’ impact).  

5 

 
SO3: Strengthen the capacity of the cultural and creative sector and its links with other 
sectors 

There  was a  consensus  amongst  all consultees that  the capacity of the sector had benefitted 
greatly from the ECOC. Developing skills and capacity came out strongly as a direct benefit 
of  the  ECOC.  The  ECOC  open  public  consultation  shows  that  87%  of  respondents  felt  that 
ECOC had had some positive effect on building the cultural capacity of the sector in the city 
with  60%  stating  that  the  impact  was  either  very  high  or  high.  Around  three  quarters  of 
stakeholders'  interviews  revealed  that,  for  most  local  artists  and  project  coordinators,  the 
ECOC was their first opportunity to get involved in projects at such a large scale, and to have 
such a large and international audience for their artistic products. 

SO4: Raise the international profile of cities through culture 

There was less agreement on the level to which the ECOC raised the international profile of 
Pafos. Stakeholders were generally in agreement with one another by saying that the focus of 
Pafos2017 was more about encouraging the local population and tourists already in the city to 
take part in  the ECOC.  They  said  that one of the key objectives of the  “Open Air  Factory” 
was  to  reconnect  the  local  citizens  with  the  cultural  and  historical  heritage  of  Pafos,  in 
particular in the old  city centre. According to all stakeholders linked to the visitor economy 
interviewed as part of this evaluation, the wider purpose of attracting more visitors to the old 
city  centre  was  to  increase  its  potential  as  a  ‘cultural  vector’  for  tourism,  in  addition  to  the 
more ‘traditional beach tourism’ which Pafos has always been known for. Therefore they felt 
that actual foreign visitor numbers had not increased due to the ECOC.   

Despite the above, the open public consultation  shows that a city having ECOC status  does 
sometimes affect the likelihood of a person wanting to visit the city. The survey showed that 
42% of respondents  said that Pafos having ECOC status  would mean they  would be ‘much 
more’ likely to visit the city and a further 45% said that they would be ‘slightly more’ likely 
to visit the city. Only 13% said that having ECOC status would make no difference in their 
decision to  visit  Pafos.  The public  consultation  also  found that 58% of respondents  thought 
that the ECOC had either a very high or a high impact on increasing international visitors to 
the  city.  The  respondents  to  the  public  consultation  were  generally  from  the  local  city  or 
region  of Pafos, which suggests  why their responses were  generally positive in  this respect. 
The  results  from  the  public  consultation  also  show  that  a  city  having  ECOC  status  can 
influence  tourism  numbers  in  a  city  as  it  becomes  a  contributing  factor  in  their  decision  on 
which destination to choose. 

3.3   EFFICIENCY 

the  ECOC.  The 

The  in-depth  interviewees  and  the  OPC  respondents  were  asked  to  comment  on  various 
interviewees  were  generally  very 
efficiency  aspects  of 
knowledgeable  about  the  implementation  arrangements,  given  that  they  included  the  main 
stakeholders,  including  several  that  were  independent  of  the  bodies  responsible  for 
implementing  the  ECOC  (e.g.  media).  It  is  not  known  how  familiar  the  OPC  respondents 
were regarding the implementation arrangements. 

in-depth 

6 

 
Aarhus 

There  was  a  consensus  amongst  the  in-depth  interviewees  that  the  arrangements  for 
governance and implementation proved efficient, drawing on the strong political support both 
in  Aarhus and across the region  and despite difficulties arising from  the  departure of senior 
staff  during  the  development  phase.  This  positive  finding  was  supported  by  the  OPC,  with 
57% of respondents for Aarhus stating that it performed strongly or very strongly. Only 15% 
responded  negatively  to  this  question.  Similarly,  the  marketing  information  for  Aarhus  was 
well received with 57% of respondents stating that the information shared was strong or very 
strong, although 16% felt it was weak or very weak. 

Representatives  of  the  Aarhus  ECOC  reported  that  the  EU-level  selection  and  monitoring 
procedures  and  the  informal  meetings  with  the  panel  proved  valuable  in  giving  impartial 
advice and support to the ECOC from highly experienced experts. (OPC consultees were not 
asked to comment on this question). 

Pafos 

The  interviewees  and  OPC  respondents  both  felt  that  the  Pafos  ECOC  was  very  efficient 
mainly because it delivered what they felt was a positive ECOC on a very small budget. They 
felt  that  the  number  of  cultural  activities,  the  quality  of  the  cultural  programme  and  the 
various  benefits  that  were  identified  showed  a  very  efficient  ECOC  when  the  size  of  the 
limited  budget  was  considered.  High  levels  of  efficiency  included  activities  being  delivered 
outside (meaning now venue costs), the use of volunteers and also having limited high profile 
artists all added up to an efficiently run ECOC.     

3.4  

SUSTAINABILITY 

Aarhus 

In  Aarhus,  consultees  supported  the  view  that  the  ECOC  had  generated  potential  for  long-
term  impact  through  the  skills  and  experience  gained  by  cultural  operators,  involvement  of 
citizens,  increased  audiences  and  greater  international  profile.  It  was  also  felt  that  the 
continuation of key projects offers the potential for long-term impact. The OPC respondents 
were  also  positive  on  this  point,  with  57%  reporting  that  efforts  to  make  the  benefits  and 
activities sustainable were strong or very strong and only 6% reporting them as weak or very 
weak. However, 20% did not know. 

Some  in-depth  interviewees  felt  that  the  lack  of  a  specific  legacy  body  and  an  overall 
continuation  strategy  risks  a  loss  of  momentum.  However,  there  was  a  consensus  that  the 
proposal  for  continued  regional  collaboration  in  the  field  of  culture  looks  promising.  (OPC 
respondents were not asked to comment on these questions). 

Pafos 

This  issue is  where the largest  disagreement  took place between the OPC and stakeholders, 
with the OPC being much more positive about sustainability. In general, around 40% of OPC 
respondents  were  positive  about  legacy  arrangements  whereas  very  few  stakeholders  (less 

7 

 
than 10%) were generally positive about ECOC sustainability. The stakeholders pointed to a 
range of  evidence including no sustainability plan, no contracts  of ECOC staff running into 
2018 and no planned projects in following years as the reasons why sustainability was poor. It 
is not known why respondents to the OPC were much more positive about this issue.   

3.5   COHERENCE 

Neither  the  OPC  nor  the  in-depth  interviews  directly  requested  consultees  to  provide  an 
opinion on questions of coherence. 

3.6   EU ADDED VALUE 

The OPC included some specific questions on the impact of the ECOC regarding awareness 
of other European cultures, European sentiment and the Commission’s oversight of the ECOC 
Action.  The  in-depth  interviews  did  not  require  the  consultees  to  comment  on  these  issues 
meaning this sub-section is focussed on the OPC only. 

Across all OPC respondents, 41% felt that the ECOC had a high or very impact on helping 
local people to be more aware of other European cultures outside their country, whilst another 
37% felt it had a moderate impact. Fewer than 3% felt it had no impact. More respondents in 
Pafos (52%) than in Aarhus (20%) felt this impact was high or very high. 

The majority (51%) of OPC respondents perceived the ECOC to have had high or very impact 
on helping promote local cultures to European visitors, whilst nearly a third (31%) reported a 
moderate impact. Only 1% reported no impact. Again, more respondents in Pafos (59%) than 
in Aarhus (37%) felt this impact was high or very high. 

Only  a  minority  of  respondents  (43%)  felt  that  the  ECOC  had  a  high  or  very  impact  on 
helping local people feel more European, although another 30% reported a moderate impact. 
Only 7% reported no impact.  Once again,  more respondents  in  Pafos (49%) than in  Aarhus 
(31%) felt this impact was high or very high. 

Across the OPC, 93% of respondents were aware that the ECOC is an EU action overseen by 
the  European  Commission.  This  is  perhaps  unsurprising,  as  the  OPC  was  hosted  on  the 
European  Commission’s  survey  tool.28  Some  86%  of  OPC  respondents  also  believed  that 
there is a benefit for the European Commission overseeing the ECOC action, whilst only 8% 
did not see such a benefit. The in-depth interviewees were not specifically asked to comment 
on  this  question,  though  none  reported  any  particular  dissatisfaction  regarding  the 
Commission’s oversight of the ECOC Action. 

3.7   USE OF THE RESULTS OF THE CONSULTATIONS 

The evidence from the OPC and in-depth interviews has been fully taken into account in the 
preparation of the final report. The in-depth interviews have helped provide the narrative for 
each  ECOC  by  describing  the  activities  that  took  place.  The  in-depth  interviews  have  also 

28 https://ec.europa.eu/eusurvey/ 

8 

 
                                                           
provided  evidence  against  the  evaluation  questions.  This  evidence  has  been  triangulated 
against evidence from other sources, including the OPC, documentation and data linked to the 
ECOC and data from other sources, such as tourist bodies and national statistics. 

The  OPC  has  provided  some  useful  quantitative  data  regarding  the  opinions  of  individuals 
and organisations on a range of questions. This data has helped confirm some of the findings 
supported  by  evidence  from  other  sources.  However,  the  results  of  the  OPC  have  had 
relatively limited usefulness, as first, the consultation was self-selective, and, second, only 76 
responses were received. For this reason, the OPC results of the survey have mostly been used 
as  supportive  and  sometimes  contextual  and  have  only  been  used  in  conjunction  with  other 
sources of evidence. 

9 

 
ANNEX 3: METHODS AND ANALYTICAL MODELS 

The figure below presents the overview of the methodology. A more detailed overview 
of the methodology and sources used for each of the evaluation questions is presented in 
the subsequent tables and text. 

1. 

 Overview of methodology and tasks of the evaluation: 

Inception phase 

Task 1 : Kick-off meeting  

Task 2 : Initial consultations - List of candidates for interviews - Notes of first interviews  

Task 3 : First visits to the cities ECOC  

Task 4 : Desk research  

Task 5 : Inception Report  

Main Research phase 

Task 6 : Public consultation  

Task 7 : Initial Bulletin  

Task 8 : Online survey of projects  

Task 9 : Second visits to the cities  

Task 10 : Additional interviews 

Analysis and reporting phase 

Task 11 : Information sharing and analysis  

Task 12 : Interim Report  

Task 13 : Factual check of the city reports  

Task 14 : Final Report  

10 

 
 
 
Table 1.1  Evaluation questions: Relevance, EU added value and coherence 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ1: To what extent are the objectives of the 
ECOC Action? 

What  was  the  main  motivation  behind  the  city 
bidding  to  become  a  European  Capital  of 
Culture? 

the  process  of  determining 
What  was 
objectives? 
there  a  process  of 
  Was 
consultation  in  each  city  to  define  aims  and 
objectives? 

What  were  the  objectives  of  the  city  in  being 
ECOC?  What  was  the  relative  importance  of 
each objective? 

To  what  extent  were  the  objectives  consistent 
with  the  Decision  and  with  the  ECOC's  own 
application?  (special  focus  on  the  European 
dimension)  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Have any specific objectives of the ECOC event 
been related to social impacts? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

In  this  connection,  did  the  objectives  of  the 
ECOC  event  include  reaching  out  to  all  groups 
excluded, 
of 
disadvantaged, disabled people and minorities? 

including 

society, 

the 

EQ2:  To  what  extent  were  the  ECOC's 
associated 
cultural 
activities relevant to their own objectives? 

programmes 

and 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

To  what  extent  were  the  activities  consistent 
with the ECOC's own objectives? (special focus 
on the European dimension) 

Y 

Y 

Y 

Y 

Y 

extent 

specific 
To  what 
themes/orientations  of  the  cultural  programme 
proved to be relevant to the objectives defined? 

have 

the 

Y 

Y 

Y 

Y 

How  was  the  European  dimension  reflected  by 
the themes put forward by the ECOC event and 

Y 

Y 

Y 

Y 

Y 

11 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

in terms of cooperation at European level? How 
did  the  Capitals  of  Culture  seek  to  make  the 
European dimension visible? To what extent did 
the two ECOC cooperate? 

Table 1.2 Evaluation questions: Effectiveness 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ3:  To  what  extent  were 
objectives achieved? 

the  EU-level 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Provide  typology  of  outputs,  results  and  possible 
impacts of the Action at different levels (European, 
national, regional etc.) 

the  ECOC  event  been 
To  what  extent  has 
successful in attaining the objectives of the Action 
(refer to list in the intervention logic)? 

Was the cultural programme perceived as being of 
high artistic quality? To what extent did the ECOC 
prove  successful  in  bringing  their  chosen  artistic 
themes/orientations to the fore? 

To what extent did the ECOC title contribute to an 
enhanced  cultural  offer  in  the  cities  holding  the 
title  (e.g.  in  terms  of  scope  and  scale)  with 
stronger European Dimension? 

To  what  extent  did  the  ECOC  implementation 
widen access to and participation in culture in the 
two cities? What actions were taking to include the 
elderly,  young  people,  people  with  special  needs 
in the cultural activities? How accessible were the 
activities carried out? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

did 

the  ECOC 

How 
help 
strengthening  the  capacity  of  the  cultural  and 
creative  sectors  and  its  links  with  other  sectors? 
Which  help  was  available  to  cultural  operators  to 

programmes 

Y 

Y 

Y 

Y 

Y 

Y 

12 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

extend  their  networks  and  work  transnationally 
and internationally? 

To  what  extent  did  the  Action  in  the  two  cities 
raise their international profile through culture? 

EQ4:  To  what  extent  were  the  cities's  own 
objectives achieved? 

What  quantitative  indicators  (number  of  visitors, 
overnight  stays,  cultural  participation  of  people, 
etc.)  of  the  social,  tourist  and  broader  economic 
impacts  of  the  event  have  been  gathered  by  the 
ECOC?  

To what extent did the ECOC achieve the outputs 
hoped  for  by  the  city  and  as  set  out  in  the 
application?  

To what extent have specific objectives related to 
social impacts been met? 

To  what  extent  were  the  objectives  related  to 
reaching out to all groups of society, including the 
excluded, disadvantaged, disabled and minorities, 
met? 

EQ5: To what extent has the Action resulted in 
unintended effects? 

Are  there  any  instances  where  the  ECOC  event 
has  exceeded  initial  expectations?  What  positive 
effects has this had?  

Where  expectations  have  not  been  met,  what 
factors  have  hindered  the  development  of  the 
Action? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Have  any  other  unintended  effects  been 
identified? 

Y 

Y 

Y 

Y 

Y 

EQ6: To what extent can the positive effects of 
the  ECOC  Action  be  considered 
to  be 
sustainable? 

Which  of  the  activities  or  elements  of  the  ECOC 
event are likely to continue and in which form once 
the ECOC-year is over? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

13 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Has  any  provision  been  made  to  continue  and 
follow  up  the  cultural  programme  of  the  ECOC 
event after the closure?  

How will the city continue to manage its long-term 
cultural development following the ECOC event? 

What  will  be  the  role  of  the  operational  structure 
after the end of the ECOC event and how will the 
organizational structure change? 

What  has  been  the  contribution  of  the  ECOC 
improved  management  of  cultural 
event 
development in the city? (in the medium-term) 

to 

What are the impacts of the ECOC event likely to 
be  on  the  long  term  cultural  development  of  the 
city?  
What are the impacts of the ECOC event likely to 
be  on  the  long  term  social  development  of  the 
city? 
What are the impacts of the ECOC event likely to 
be on  the  long term urban and broader economic 
development of the city? 

Table 1.3 Evaluation questions: Efficiency 

EQ7:  How  did  the  management  arrangements 
of each ECOC contribute to the achievement of 
outputs, results and impacts? 

Y 

Y 

Y 

Y 

Y 

Y  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

s
w
e
i
v
e
r

e
r
u
t
a
r
e
t
i
L

a
t
a
D

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a

h
t
i

w

s
w
e
i
v
r
e
t
n

I

Y 

s
m
a
e
t
y
r
e
v
i
l
e
d

s
n
o
i
t
a
t
l
u
s
n
o
c

l

r
e
d
o
h
e
k
a
t
S
Y 

s
t
c
e
j
o
r
p
f
o
y
e
v
r
u
S

s
w
e
i
v
r
e
t
n

i

t
c
e
j
o
r
P

s
i
s
y
l
a
n
A
Y 

How  have  the  organizational  models  of  the 
formal  governing  Board  and  operational 
structures played a role in the European Capital 
of  Culture?  What  role  have  the  Board  and 
operational  structures  played  in  the  ECOC 
event's  implementation?  At  what  stage  were 
these  structures  established?  How  did 
it 
improve  management  of  culture  in  the  city 
during the event? 

Who  chaired  the  Board  and  what  was  his/her 
experience?  What  were  the  key  success  and 
failure elements related to the work of the Board 

Y 

Y 

Y 

14 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
and  operational  structure  used  and  personnel 
involved?  

Has  an  artistic  director  been  included  into  the 
operational  structure  and  how  was  he/she 
appointed?  What  were  the  key  success  and 
failure  elements  related  to  the  work  of  the 
artistic director and personnel involved? 

Y 

Y 

Y 

What  was 
programme? 

the  process  of  designing 

the 

Y 

How were activities selected and implemented? 

How  did  the  delivery  mechanism  contribute  to 
the achievement of outputs? 

Y 

to 

been 

strategy 

To  what  extent  has  the  communication  and 
promotion 
successful 
the  promotion  of  city 
in/contributed 
image/profile,  promotion  of  the  ECOC  event, 
awareness-raising  of  the  European  dimension, 
promotion  of  all  events  and  attractions  in  the 
city? 

To  what  extent  has  the  communication  and 
promotion  strategy  including  the  use  of  social 
the 
media 
communication's 
local, 
regional,  national,  European  and  international 
levels? 

target  groups  at 

successfully 

reached 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ8:  To  what  extent  were  the  selection, 
co-financing 
monitoring 
by  Decision 
procedures, 
2006/1622/2006/EC efficient? 

and 
EU 
introduced 

the  Commission 

To  what  extent  have  the  mechanisms  applied 
by 
line  with  Decision 
in 
2006/1622/EC for the selection of the European 
the  subsequent 
Capitals  of  Culture  and 
implementation  and  monitoring  mechanisms 
influenced the results of the ECOC event?    

informal  meeting 
To  what  extent  has 
following  the  designation  as  well  as  other 

the 

15 

L

Y 

Y 

Y 

Y 

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

advice  offered  by 
Commission 
ECOC event?  

the  panel  and  by 
the  results  of 

influenced 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

the 
the 

How was the Melina Mercouri Prize used? 

EQ9:  To what  extent  did the  ECOC  manage 
to raise the necessary resources?  

Y 

What was the process of securing the financial 
inputs? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  was  the  total  amount  of  resources  used 
for  each  ECOC  event?  What  was  the  final 
financial outturn of the year?  

Y 

Y 

Y 

What  were  the  sources  of  financing  and  the 
respective  importance  of  their  contribution  to 
the  total?  How  much  came  from  the  European 
Union Structural Funds (e.g. ERDF - European 
Regional  Development  Fund,  ESF  –  European 
Social Fund) or other sources of EU funding? 

To  what  extent  did  the  ECOC  title  trigger 
complementary sponsorship? 

What  was  the  total  expenditure  strictly  for  the 
implementation  of  the  cultural  programme  of 
the  year  (operational  expenditure)?  What  was 
the proportion of the operational expenditure in 
the total expenditure for the ECOC event?  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  proportion  of  expenditure  was  used  for 
tourism 
infrastructure 
infrastructure, including renovation)? 

(cultural 

and 

Y 

Y 

Y 

Y 

Y 

EQ10: To what extent were the financial and 
human  resources  secured  by  each  ECOC 
appropriate and proportionate? 

16 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

Was  the  total  size  of  the  budget  sufficient  for 
reaching  a  critical  mass  in  terms  of  impacts? 
Could  the  same  results  have  been  achieved 
with less funding? Could the same results have 
been achieved if the structure of resources and 
their respective importance was different?  

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t

y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Y 

Y 

Y 

Y 

To  what  extent  have  the  human  resources 
deployed for preparation and implementation of 
the  ECOC  event  been  commensurate  with  its 
intended outputs and outcomes?  

Y 

Y 

be 

As a result, could the total budget for the ECOC 
and 
event 
proportional to what the each ECOC set out to 
achieve?  

appropriate 

considered 

Table 1.4 Evaluation Questions: Coherence 

Evaluation Question 

Y 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

Y 

Y 

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

h
t
i

w
s
w
e
i
v
r
e
t
n

I

EQ11:  To  what  extent  were 
complementary to other EU initiatives? 

the  ECOC 

Y 

Y 

Y 

Y 

As far as the conclusions made for the two cities 
allows it, to what extent has the Action proved to 
be  complementary  to  other  EU  initiatives  in  the 
field of culture? 

To what extent has each  ECOC been reinforced 
by and added  impetus to investments by the EU 
Structural Funds? 

two  ECOC 
To  what  extent  have 
complemented 
e.g. 
other  EU 
European  Youth  Capital,  European  Green 
Capital? 

initiatives, 

the 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

s
i
s
y
l
a
n
A

Y 

Y 

Y 

Y 

EQ12:  What  is  the  EU  added  value  and  the 
visibility of the ECOC Action? 

Y 

Y 

Y 

Y 

Y 

Y 

17 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

As far as the conclusions made for the two cities 
allow,  what  is  the  added  value  of  the  European 
Capital  of  Culture  being  an  EU 
initiative, 
compared to what could be achieved if the Action 
was a purely national or local action? 

Was the fact that this is a European Union action 
sufficiently communicated by the cities?  

To  what  extent  were  the  general  public  and  the 
cultural operators aware of this fact? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

2.  Details of the relevant evaluation tasks and methodologies:  

Inception phase 

The  aim  of  the  inception  phase  was  to  finalise  the  evaluation  framework  and  research 
tools, address the project steering group's comments regarding the proposed method and 
collect background information on Pafos and Aarhus. 

Initial consultations   

The task consisted in  identifying stakeholders  to consult  with,  targeting  people that are 
knowledgeable  about  the  two  cities,  their  cultural  programmes  and  the  achievements, 
lessons  learnt  and  impact  of  the  ECOC,  including  national,  regional  and  local 
stakeholders. Discussion at the kick off meeting identified a need for strong triangulation 
and to gain opinions from a broader set of stakeholders. 

First visits to the cities  

Initial face to face visits were made, to  Pafos on 28-29 November 2017  and Aarhus in 
December 2017.  

Those consulted with during the initial visit to Pafos were: 

  Christos Patsalides- Chairman of the Board of Directors for Pafos2017, 
  Georgia Doetzer- Artistic Programme Director for Pafos2017, 
  Terpsi Vasou- Financial and Administrative Director for Pafos2017. 

Those consulted in Aarhus were:  

  Rebecca Matthews, Chief Executive Officer Aarhus2017, 
  Juliana Engberg, Programme Director Aarhus2017, 
  Karin Buhl Slæggerup, Head of Partnerships & Development Aarhus2017, 
  Rina Valeur, Head of Strategy & Operations Aarhus2017, 

18 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  Bent Sørensen, Head of Communications Aarhus2017. 

Desk research including big data feasibility 

This involved collection and review of literature, primary data from the delivery agency 
and available secondary data, as well as web and social media statistics. The list of the 
literature that was collected at the European level which was reviewed and assessed for 
usefulness  for  the  sake  of  the  evaluation  can  be  found  in  the  Report29.  Much  of  the 
European  level  literature  is  helpful  to  either  understand  the  policy  drivers  at  EU  level 
which support the two 2017 ECOC or provide background on the bidding and application 
stages  for  both  2017  cities  (i.e.  little  of  it  provides  information  useful  for  the  content, 
delivery and impact  of the two programmes). More academic literature  at  the EU level 
linked  to  the  ECOC  was  also  searched  for.  Instead  of  looking  at  academic  literature 
generally on 'European culture' and also literature before 2017, the contractor supporting 
the  evaluation  specifically  looked  at  up  to  date  (i.e.  produced  in  2017)  pieces  which 
linked to the ECOC Action. This search did not prove fruitful, although some academic 
literature at the city level was found. 

To complement the above desk research task the contractor assessed the extent to which 
each  ECOC  had  used  big  data  as  well  as  analysed  web  and  social  media  to  increase 
visibility and interest in the ECOC among country residents and internationally. 

Open public consultation 

The  Open  public  consultation  proved  to  be  a  source  of  complementary  qualitative  data 
for both cities.  

Second visits to the cities 

With  a  clear  view  emerging  from  desk  research  and  project  survey  as  to  the  overall 
relevance, efficiency and effectiveness of each ECOC, the contractor undertook a second 
three day visit to each city. The visit focused on conducting in-depth interviews with the 
managing teams, their key stakeholders and cultural operators identified during the initial 
visits- please see the annex for more information.  

During the visits to the cities described below, the contractors supporting the evaluation 
also sourced stakeholders for consultation who did not directly benefit or take part in the 
ECOC programme and who they felt would have an honest, impartial and unbiased view 
of how the programme was developed or implemented. While the aim was not to look for 
purely  negative  views,  the  contractors  wanted  to  speak  with  stakeholder  who,  for 
example: 

  did not deliver an ECOC project or take part in the cultural programme attached 

to the ECOC; 

  were a member of the local and national press who had written either an honest or 
negative piece on the ECOC (as opposed to simply writing a press release which, 
for example, communicated the content and dates of the cultural programme); 
  had applied for ECOC funding (to be part of the ECOC cultural programme) and 

were rejected; 

29   See previous footnote.  

19 

 
                                                           
  were  involved  in  ECOC  activity  but  were  often  vocal  about  a  particular  aspect 
(e.g. the quality of culture on offer, the lack of audiences, the lack of diversity of 
cultural content);  

  were not  directly involved in  the cultural  policy agenda in  the cities but  instead 

had an economic, social or environmental view on the ECOC. 

The list of stakeholders consulted is provided in the annex to the Report30. This list was 
partly  developed  by  the  ECOC  delivery  teams  but  also  through  an  internet  search  for 
stakeholders who work in the above fields.  

With this in mind, in Pafos, the list of stakeholders consulted includes in Pafos:  

  Officials  from  the  City,  the  Chamber  of  Commerce,  the  Regional  Board  of 

Tourism and Tourism organisation, 

  Cultural operators from various sectors (theatre, music,…), 
  Professor at the University. 

In Aarhus, it included:  

  Sponsor companies, 
  Cultural operators, 
  Associate Professor, 
  Official from other municipalities. 

Building on the interviews undertaken in the first visits, the interviews with the delivery 
teams  served  four  purposes:  i)  gathering  further  factual  data  and  information  about  the 
cultural  programme  and  its  achievements;  ii)  identifying  the  "story"  of  the  ECOC 
throughout  its  lifecycle,  i.e.  conception,  application,  development,  delivery,  legacy;  iii) 
gaining a critical (albeit "insider") perspective on the relevance, efficiency, effectiveness 
and sustainability of the ECOC; iv) triangulating emerging findings, for example, those 
emerging from the desk research and project survey. 

Interviews  with  other  key  stakeholders  in  the  cities  (such  as  municipality,  Chamber  of 
Commerce,  commercial  sponsors,  media  representatives,  and  cultural  operators)  were 
also  essential  to  identifying  the  overall  "story"  of  the  ECOC  from  conception,  through 
application,  to  development  and  delivery.  These  individuals  offered  an  external, 
alternative and broader perspective on the ECOC. 

Additional project and stakeholder interviews 

These interviews sought to gather an in-depth understanding of the effect that ECOC had 
for the individuals involved, their organizations and the local culture sector as a whole. It 
also  allowed  to  ensure  that  findings  are  based  on  consensus  across  a  range  of  target 
groups as well as to identify key differences in the experience of different stakeholders. 
In  the  case  of  projects,  the  criteria  for  inclusion  focused  on  whether  it  represents  good 
practice in relation to at least one of the key dimensions of the evaluation, including (but 
not limited to) increased European cooperation, the effective targeting of key groups of 
citizens or neighbourhoods, audience development strategies, or legacy effects. 

30   hthttps://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/ecoc-2017-evaluation-

en.pdf 

20 

 
 
                                                           
Analysis and reporting phase 

Once  all  the  information  gathered,  the  experts  fulfilled  an  analysis  of  the  available 
information,  drawing  conclusions  and  triangulating  data  to  ensure  consistency  and 
accuracy. The limitations of the data gathered have been explained in  the report and in 
the  Staff  Working  Document.  After  the  first  drafting  exercise,  the  results  were  shared 
with the two ECOC for a factual check. Both the Pafos and Aarhus ECOC were asked to 
undertake  a  check  on  the  completeness  of  data  being  used  to  address  each  evaluation 
topic. Their comments,  where relevant, were included in the Final Report submitted by 
the contractor supporting the evaluation. 

21