COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex Post evaluation of the 2016 European Capitals of Culture (Donostia San Sebastian and Wroclaw)
Date: 2018-04-25
Institutions: Directorate-General for Education, Youth, Sport and Culture, European Commission
Subjects: EU action, European cultural event, European symbol, Poland, Spain, capital city

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 25.4.2018  SWD(2018) 140 final  COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex Post evaluation of the 2016 European Capitals of Culture (Donostia San Sebastian and Wroclaw) {COM(2018) 235 final}  
 
 
Table of contents 

1. 

2. 

3. 

INTRODUCTION ................................................................................................................................ 2 

BACKGROUND TO THE INTERVENTION ..................................................................................... 3 

IMPLEMENTATION / STATE OF PLAY .......................................................................................... 6 

4.  METHOD ........................................................................................................................................... 10 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS .......................................... 11 

6. 

CONCLUSIONS ................................................................................................................................. 22 

ANNEX 1: PROCEDURAL INFORMATION ............................................................................................ 23 

ANNEX 2: STAKEHOLDER CONSULTATION ....................................................................................... 25 

ANNEX 3: METHODS AND ANALYTICAL MODELS .......................................................................... 26 

1 

 
 
 
 
 
 
 
 
 
1. 

INTRODUCTION 

This  document  describes  the  methodology  and  findings  of  the  ex-post  evaluation  of  the 
European Capital of Culture (ECOC) Action for 2016.  
Decision No 1622/2006/EC1 requires, in Article 12, that the Commission ensures the external 
and independent evaluation of the results of the ECOC from the previous year. The purpose of 
the yearly evaluation, which the European Commission carries out since 2008, is to assess the 
achievements of the Action in the relevant year, to establish a comprehensive understanding 
of the overall performance and to put forward conclusions, recommendations and lessons for 
future ECOC title-holders, applicants and EU institutions.  

The aim of this evaluation exercise is not to lead to any change in the regulation governing 
the ECOC Action. Indeed those rules were changed in 20142 on the basis of an ad hoc interim 
evaluation of selection and monitoring procedures of European Capitals of Culture3, but the 
new rules only apply in the case of cities which are designated as ECOC for the years from 
2020  to  20334.  It  is  therefore  impossible  to  use  the  outcomes  of  the  evaluation  of  the  two 
ECOC 2016 to draw conclusions on the new legal basis as this new Decision doesn't apply to 
these two ECOC title-holders.  

This  working  document  summarizes  the  findings  of  the  external  evaluation  of  the 
implementation of the ECOC 2016, including the selection and monitoring procedures and the 
operational delivery  by the  two  hosting  cities,  namely  Donostia  San  Sebastian  in  Spain  and 
Wroclaw  in  Poland5.  The  external  evaluation  examined  how  the  two  cities  developed  their 
respective  applications  and  cultural  programmes,  how  they  delivered  the  planned  initiatives 
throughout the year, and any cultural, general and long-term impact generated by the Action.  

The  evaluation  constitutes  a  valuable  opportunity  to  critically  reconsider  the  past  year  with 
the intention of reshaping insights and broadening wisdom in the light  of the experiences of 
the host cities. 

1   Decision  No  1622/2006/EC  of  the  European  Parliament  and  of  the  Council  of  24  October  2006  establishing  a 
Community Action for the European Capital of Culture event for the years 2007 to 2019 (OJ L 304, 3.11.2006, p. 1), 
available at:  

- http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:304:0001:0006:EN:PDF. 

2   Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union action 
for the European Capitals of Culture for the years 2020 to 2033 and repealing Decision No 1622/2006/EC (OJ L 132, 
3.5. 2014, p. 1), available at: 

- http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2014.132.01.0001.01.ENG 

This  Decision,  which  covers  the  ECOC  titles  2020  to  2033,  retains  the  general  structure  and  main  elements  of  the 
previous Decision while introducing improvements to maximise the benefits of holding the title as well as taking part in 
the competitive process for all bidding cities and their citizens. Improvements include among others the introduction of 
more explicit and measureable criteria, the reinforcement of conditionality for the payment of the Melina Mercouri Prize 
and the obligation for the cities – instead of the Commission – to carry out the ex-post evaluation of the ECOC year. 

3   Available  at:  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/library/2011-capitals-culture-

assignment-report_en.pdf 

4   Article 17 of Decision No 445/2014/EU. 

5   Full document available at: https://ec.europa.eu/culture/evaluations_en.  

2 

 
 
                                                            
 
2.  BACKGROUND TO THE INTERVENTION 

2.1.  The European Capital of Culture Action 

The initial scheme of "the European City of Culture" was launched at intergovernmental level 
in  19856.  On  the  basis  of  this  experience,  Decision  No 1419/1999/EC7  established  a 
Community  Action  for  the  ECOC  event  for  the  years  2005  to  2019.  Member  States  were 
ranked  in  a  chronological  order  of  entitlement  to  host  the  event  each  year.  Decision 
No 1419/1999/EC was replaced by Decision No 1622/2006/EC which kept the principle of a 
chronological  order  of  Member  States  but  further  refined  the  objectives  of  the  Action  and 
introduced new selection and monitoring arrangements. As mentioned in previous paragraph, 
a new Decision was adopted by the European Parliament and the Council in April 2014, but 
cities  which  were  designated  as  European  Capitals  of  Culture  for  the  years  up  to  2019 
continue  to  be  regulated  by  former  Decision  No 1622/2006/EC.  The  2014  Decision  was 
amended in September 2017 to open the ECOC Action to European Free Trade Association 
countries which are parties to the Agreement on the European Economic Area8. 

2.2.  Objectives of the ECOC Action 

The ECOC Action aims to highlight the richness and diversity of European cultures and the 
features  they  share,  thereby  promoting  greater  mutual  understanding  among  European 
citizens, as well as to foster the contribution of culture to the long-term development of the 
cities. ECOC shall strive to foster cooperation between cultural operators, artists and cities in 
Europe,  foster  the  participation  in  cultural  activities  of  the  citizens  living  in  the  city  and 
surroundings while raising the interest of citizens from abroad, to be sustainable and to be an 
integral part of the long-term cultural and social development of the city. 

The hierarchy of objectives presented in the Table below is based on the objectives as stated 
in Decision No 1622/2006/EC, but it has been updated to reflect the content of the new legal 
basis  for  ECOC  post-2019.  The  general  and  strategic  objectives  are  taken  directly  from 
Article 2 of Decision No 445/2014/EU, with the operational objectives flowing logically from 

6   Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the 

annual event 'European City of Culture' (85/C 153/02), available at; 

- http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:1985:153:0002:0003:EN:PDF. 

7   Decision  1419/1999/EC  of  the  European  Parliament  and  of  the  Council  of  25  May  1999  establishing  a  Community 
action for the European Capital of Culture event for the years 2005 to 2019 (OJ L 166, 1.7.1999, p. 1). The Decision was 
amended by Decision No 649/2005/EC to integrate the Member States that joined the EU in 2004 (OJ L 117, 4.5.2005, 
p. 20). Both Decisions are available at: 

- http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A31999D1419 

- http://eur-lex.europa.eu/eli/dec/2005/649(1)/oj 

8   Decision (EU) 2017/1545 of the European Parliament and of the Council of 13 September 2017 amending Decision No 
445/2014/EU establishing a Union action for the European Capitals of Culture for the years 2020 to 2033 (OJ L 237, 
15.9.2017, p. 1). Text available at: 

- http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32017D1545  

3 

 
 
 
                                                            
these.  They  are  also  informed  by  the  selection  criteria  detailed  in  Article 5  of  the  2014 
Decision.

4 

 
 
Table on ECOC hierarchy of objectives 

General objective 

Safeguard and promote the diversity of cultures in Europe, highlight the common features they share, and foster the contribution of culture to the long-term development 
of cities 

Specific objectives (SO) 

SO1:  Enhance  the  range,  diversity  and  European 
dimension of the cultural offer in cities, including 
through transnational co-operation 

SO2:  Widen  access  to  and  participation  in 
culture 

SO3:  Strengthen  the  capacity  of  the 
cultural  and  creative  sector  and  its 
links with other sectors 

SO4: Raise the international 
profile  of  cities 
through 
culture 

Operational objectives 

Stimulate  a  diverse  range  of  cultural  activities  of 
high artistic quality  

Create new and sustainable opportunities for 
a  wide  range  of  citizens  to  attend  or 
participate in cultural events 

Implement  cultural  activities  promoting  cultural 
diversity, dialogue and mutual understanding 

Implement cultural activities highlighting (shared) 
European cultures and themes  

Involve  local  citizens,  artists  and  cultural 
organizations 
and 
implementation 

development 

in 

Improve cultural infrastructure 

Develop 
governance of the cultural sector 

skills,  capacity  or 

the 

Stimulate 
operation with other sectors 

partnership 

and 

co-

Attract  the  interest  of  a 
broad 
and 
European 
international public  

Involve  European  artists,  promote  cooperation 
transnational 
with 
partnerships  

countries 

different 

and 

Provide  opportunities  for  volunteering  and 
foster links with schools and other education 
providers 

Combine  traditional  art  forms  with 
new types of cultural expression 

5 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3. 

IMPLEMENTATION / STATE OF PLAY 

3.1.  The selection and monitoring of the European Capitals of Culture 2016 

In accordance with Decision No 1622/2006/EC, Spain and Poland were entitled to host 
the ECOC in 2016. The two-phased selection (shortlisting and nomination) was carried 
out  by  a  panel  of  thirteen  members  –  six  of  whom  nominated  by  the  Member  State 
concerned  and  the  other  seven  by  European  Union  institutions  and  bodies  (European 
Parliament,  Council,  Commission  and  Committee  of  the  Regions)  –  who  examined  the 
bids  from  candidate  cities  on  the  basis  of  the  objectives  and  criteria  laid  down  in  the 
Decision. 

The two Member States launched their respective competition in parallel in the summer 
2009.  Donostia  San  Sebastian  was  shortlisted  in  September  2010  against  14  other 
bidding cities in Spain, while Wroclaw was shortlisted in October 2010 against other 10 
competitors in Poland. In June 2011, both cities were recommended by the panel to host 
the 2016 title, and were nominated by the Council of the European Union in May 2012.  

After  their  nomination,  Donostia  San  Sebastian  and  Wroclaw  were  subjected  to 
monitoring  arrangements:  the  progress  in  the  cities'  preparations  was  monitored  and 
guided by a panel composed of the seven independent experts appointed by the European 
Union institutions and bodies, which also checked compliance with the programme and 
commitments on the basis of which the cities had been selected. Donostia San Sebastian 
and Wroclaw attended two formal monitoring meetings convened by the Commission, in 
October 2013 and March 2015. 

During this monitoring phase, the panel recommended both cities to adequately involve 
minorities  and  all  various  groups  of  citizens  and  to  address  the  requirement  of 
encompassing the EU dimension in their activities. The main difficulties encountered by 
Wroclaw concerned the initial securing of funds (at the time of the first monitoring report 
only 37% of the funds had already been secured), while in Donostia San Sebastian local 
political changes that happened between 2011, 2015 and 2016 (the former for local and 
province  elections and the latter  for the Basque Government and Spanish Government) 
affected a smooth development of the project. 

During the monitoring phase, the two cities introduced modifications into the programme 
described in their original applications, in response to a changing environment and to the 
recommendations made by the panel. The panel also visited the cities where it found it 
appropriate. 

The monitoring process culminated with the panel making a positive recommendation to 
the Commission on awarding a €1.5m prize in honour of Melina Mercouri to both cities 
after the last monitoring meeting. The Melina Mercouri prize was therefore awarded and 
paid by the Commission (under the EU Creative Europe programme9) to the two ECOC 
in the autumn 2015. 

9   Regulation (EU) No 1295/2013 of the European Parliament and of the Council of 11 December 2013 
establishing  the  Creative  Europe  Programme  (2014  to  2020)  and  repealing  Decisions  No 
1718/2006/EC, No 1855/2006/EC and No 1041/2009/EC (OJ L 347, 20.12.2013, p.221). 

6 

 
 
 
                                                            
The sub-sections below describe the main features of the ECOC programmes of Donostia 
San  Sebastian  and  Wroclaw.  The  final  report  of  the  contract  supporting  the  evaluation 
contains  further  information,  including  on  the  European  dimension  of  the  programmes, 
on  the  way  the  programmes  involved  citizens  as  well  as  on  the  development  of  the 
applications and the governance and funding structures.  

3.2.  Donostia San Sebastian 2016 

Donostia  San  Sebastian  is  a  city  of  about  186,000  people  and  the  capital  of  Gipuzkoa 
Province in the Basque Autonomous Community of Spain, close to the French border. A 
unique  feature  of  the  city  is  its  very  rich  cultural  offer,  with  the  number  of  cultural 
projects taking place in it during a 'non-ECOC' year, estimated between 1,000-1,500 (i.e. 
performances, exhibitions etc.), attracting  well  over  1 million visitors per  year. Despite 
having been a popular tourist destination ever since the beginning of the 20th century, the 
city  has  been  deeply  scarred  by  the  ETA's  terrorist  activities  (in  Euskara:  Euskadi  Ta 
Askatasuna, Basque Homeland and Freedom). ETA was created in 1959 as a group for 
the  promotion  of  the  Basque  Culture,  and  later  developed  as  a  paramilitary  group  with 
the objective of independence of the Basque Country. 

Bearing  such  background  in  mind,  Donostia  San  Sebastian's  application  was  entitled 
"Culture  for  coexistence";  it  aimed  to  help  overcome  the  past  violence  engraved  in  the 
identity of the city and to stimulate a culture of respectful tolerance between people who 
"share places where multiple identities, values and objectives cohabit and constantly keep 
redefining one another"10. 

The concept of coexistence was translated into three  main "lighthouses of change"  that 
structured  the  whole  cultural  programme  of  the  ECOC  year  and  aimed  at  establishing 
sustainable cooperation. The three "lighthouses" were as follows: 

- 

- 

- 

"Lighthouse  of  Peace"  explored  a  very  relevant  topic  for  the  Basque  country, 
which faced decades of terrorism. In addition, this topic was particularly relevant 
in  the  European  context  where  the  question  of  integration  and  coexistence  is 
becoming  more  and  more  urgent.  The  concept  behind  this  lighthouse  was  to 
support new forms of governance, promoting and improving dignity and human 
rights; 
"Lighthouse of Life" dealt with the concept of human existence, being a member 
of society and a part of the environment. This path included topics such as health, 
work and the environment. One focus was to allow citizens to reflect on how they 
are in relation with each other and the environment; 
"Lighthouse of Voices" was seen as a way to reach coexistence between people. 
The  idea  was  to  use  effective  communication  as  a  virtual  mechanism  enabling 
people from different societies to interact among themselves. This lighthouse was 
primarily for activities related to communication with each other (i.e. 'voices') and 
therefore  the  main  difference  in  emphasis  from  the  lighthouse  of  peace  was 
around the importance of talking and working with one another. Communication 
was  through  culture  and  included  art  expression,  such  as  music,  literature  and 
visual  art  as  they  are  one  of  the  ingredients  for  a  more  pluralistic  society.  It 
represented the  concept  of  cultural  and  linguistic  minorities, a  topic  particularly 
relevant for the Basque Country. 

10   Selection of the European Capital of Culture for 2016 in Spain, Final selection17/28 June 2011, p.5, available at: 
https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/files/ECOC-2016-panel-spain_en.pdf.  

7 

 
                                                            
Transversal lines (called the 'quays') were  developed to  allow a  more in-depth  view on 
the  cultural  activities  that  were  proposed  by  the  three  lighthouses.  These  lines  were  in 
fact  five  methodologies  –  in  the  fields  of  sustainability  of  linguistic  diversity,  critical 
thought through contemporary art, citizens' engagement, connection between culture and 
technology, and the development of interactive situations in cultural spaces – which were 
meant to help all individual projects to achieve their own objectives. 

The  city  of  Donostia  San  Sebastian  was  the  main  driver  behind  the  initial  ECOC  bid, 
although the regional government were also very much involved in the initial discussions 
and  the  eventual  bid.  The  three  public  administrations  of  the  Basque  Country 
(Municipality  Council,  Territorial  Administration  and  the  Basque  Government)  came 
together to form a 'consortium' that ensured a joint ownership of key decisions (the ones 
that were linked to priorities, themes and major projects). 

The main focus of the governance arrangements was a Foundation that was specifically 
set  up  to  design,  deliver  and  manage  the  entire  ECOC  programme.  This  was  an  arm's 
length  organization  that  was  partly  set  up  to  ensure  a  quick  and  'flexible'  response  to 
various issues and was meant to be largely free of political influence although it had to 
deal with a comparatively complex and layered political structure and was dependant on 
it for some key decisions. This foundation was effectively at the heart of the ECOC and 
was  responsible  for  all  its  financial,  marketing,  communication  and  cultural  work  on  a 
day to day basis. 

The total planned budget for the ECOC as set out in San Sebastian's original bid was €89 
million. The actual budget during 2016 was €49.6 million, of which 27% came from the 
City,  26%  from  the  Province,  25%  from  the  Basque  Government  and  10%  from  the 
Spanish Ministry of Culture. The overall budget figure was 44% less than planned.  The 
main reasons for the difference between the planned and actual income were highlighted 
as very tight pressure on public sector budgets, political pressure to prioritise budgets on 
other issues of public interest and the Foundation's perceived or actual lack of capacity to 
seek out private sector funding coupled with poor economic conditions.   

3.3.  Wroclaw 2016 

Wroclaw is Poland's fourth largest city with a population of 636,000 inhabitants, situated 
in the South-West of the country (Lower Silesia), not far from the Czech Republic. The 
city  has  had  an  eventful  history,  changing  hands  between  Bohemia,  Poland,  Hungary, 
Austria, Prussia, Germany and modern-day Poland. Until the Second World War, the city 
was known as "Breslau" and was part of Germany. After the war, the city became part of 
Poland  under  the  terms  of  the  Potsdam  Agreement.  Most  of  the  remaining  German 
population  of  190,000  fled  or  were  forcibly  removed,  to  be  replaced  by  ethnic  Poles, 
most of whom were resettled from parts of eastern Poland that were incorporated into the 
Soviet Union. At that point, the city was (re-) named "Wroclaw". During the communist 
period  (1945-89),  Wroclaw  became  one  of  the  centres  of  opposition  to  the  communist 
regime. Since the end of the communist period, Wroclaw has developed a reputation as 
one of the most liberal cities in Poland, embracing for example public and street art. 

This  diversity  of influences  has  given the city a varied architectural  heritage, including 
Silesian  Gothic  and  Baroque  but  also  Modernism,  and  a  strong  multicultural  character, 
notably in its religious diversity as well as a very rich cultural life.  

Wroclaw's ECOC application emerged from the wider strategy of the City of Wroclaw, 
which  emphasised  the  potential  for  urban  development  to  be  supported  by  prioritizing 
investments in cultural and sporting events and infrastructures.  

8 

 
"Metamorphoses  of  Culture"  was  the  overall  concept  adopted,  which  served  as  a 
metaphor for both the historical transformation of the city (i.e. the upheaval at the end of 
the  Second  World  War  when  the  German  population  was  replaced  by  the  ancestors  of 
today's  inhabitants)  and  for  contemporary  processes  of  cultural  and  social  change 
(including  globalisation,  immigration,  EU  enlargement  and  the  growing  role  of  digital 
communications).  The  Wroclaw  2016  application  presented  the  city  as  a  "space  for 
beauty", and the intention was to "create spaces within which to restore the presence of 
beauty in public life and in daily habits", reflecting also the multicultural character of the 
city. 

The programme was divided into eight artistic domains  – Architecture, Film, Literature 
(connected with  the  UNESCO  World  Book  Capital  2016), Music,  Opera,  Performance, 
Theatre,  and  Visual  Arts  –  each  of  them  led  by  a  curator  and  featuring  a  number  of 
events and projects.  

The programme had four different stages: 

-  The "Wroclaw stage", focusing on dialogue between the city and its inhabitants; 
-  The "Lower Silesia stage", promoting regional participation in the project; 
-  The "Polish stage", a platform for collaborative artistic activities from around the 

country; and 

-  The "European and World stage", showcasing international cooperation between 
artists  and  creators  of  culture,  highlighting  the  city's  role  in  the  European  and 
world culture. 

Wroclaw envisioned very clear goals, especially for the longer term, which ranged from 
increasing awareness and recognition of the Wroclaw/Lower Silesia's cultural identity, to 
creating public spaces for social activities and shaping of civic attitudes, to increasing the 
number of visiting tourists11. 

Responsibility  for  the  organization  of  the  ECOC  was  entrusted  to  "IMPART  2016 
Festival  Office"  ("Impart"),  a  new  cultural  institution  of  Wroclaw  Municipality  formed 
from  the  merger  of  "Wroclaw  2016"  (the  institution  responsible  for  preparing  the 
application for the title) with "IMPART Art Centre" (a cultural management institution 
with  production  resources).  Impart  was  responsible  for  the  overall  organization  and 
coordination  of  the  ECOC  programme,  acted  as  the  main  contact  point  and  developed 
promotional and marketing campaigns. The leadership of the ECOC was entrusted to the 
Director-General of Impart, whilst the developing process of the cultural programme was 
delegated to a Board of Curators, all of whom were recognized cultural operators in their 
respective artistic domains. 

The eventual operating income of Wroclaw 2016 was €86.4m (as against €78.6m in the 
original bid), of which €34.4m (40%) was provided by the City of Wroclaw and €31.7m 
(37%) by the National Government. Around €7m was received from corporate sponsors, 
including the main sponsor KGHM, the Polish multinational corporation specialising in 
copper and silver production, or in the form of in-kind income. 

Unlike  some  other  ECOC,  the  body  responsible  for  the  ECOC  cultural  programme 
remained  in  existence  beyond  the  title-year  and  still  continues  to  play  a  key  role  in 
coordinating  cultural  activities  in  the  city:  some  exhibitions,  artistic  residencies  and 
micro-grants  schemes  initiated  in  the  frame  of  the  ECOC  2016  were  continued  after 
2016.  

11   All the detailed long-term goals of Wroclaw2016 are available at: www.wroclaw2016.pl/about-ECOC.  

9 

 
                                                            
4.  METHOD 

In order for results to be comparable with previous evaluations, the methodology for this 
evaluation  closely  followed  the  approach  adopted  in  previous  assessments  of  the 
Action12,  with  focus  being  placed  on  research  at  city  level  and,  in  particular,  on  the 
gathering  of  data  and  stakeholders'  views  from  both  Donostia  San  Sebastian  and 
Wroclaw. The main evaluation sources can be identified as follows: 

-  EU  level  literature:  higher  level  EU  policy  and  legislative  briefings,  papers, 
decisions and other documents relating to ECOC – such as reports of the selection 
and  monitoring  panels  –  as  well  as  academic  research  concerning  the  ECOC 
Action and the role of culture in the development of cities; 

-  ECOC  level  literature  from  Donostia  San  Sebastian  and  Wroclaw:  original  bids 
and  applications,  internal  reports  linked  to  the  application,  monitoring  or 
evaluation  processes,  as  well  as  other  pieces  of  literature  analysing  the  cultural 
programme itself; 

- 

-  Quantitative data: where available, evidence linked to each ECOC was collected 
in relation to budgets and expenditures, projects' numbers and types, participation 
and  audience  figures  as  well  as  other  pieces  of  quantitative  data  relevant  to  the 
assessment of the work and benefits of the ECOC in each city; 
Interviews  with  managing  teams:  those  responsible  for  the  direct  operational 
design and management of the ECOC projects were interviewed both during 2016 
and again in 2017. Almost all of the key positions linked to the delivery agencies 
were involved in the interviews, including those linked to strategic development, 
financial 
marketing 
management; 
Interviews with key stakeholders13: interviews were undertaken with stakeholders 
both  directly  or  indirectly  involved  in  the  planning,  management  or  delivery  of 
the ECOC along with those more widely linked to the cultural, social, economic 
or political agenda of the host cities. Stakeholders included personnel working in 
cultural  organizations,  in  city/region/State  level  administrations,  in  tourism  and 
visitor  agencies,  in  media  organizations  as  well  as  in  voluntary  and  community 
organizations; 

communication,  project 

implementation 

and 

and 

- 

-  Surveys  of  ECOC  projects:  a  specific  survey  of  projects  was  undertaken  by  the 
evaluators  in  Wroclaw,  whilst  in  Donostia  San  Sebastian  a  survey  already 
undertaken by the Donostia San Sebastian Foundation (the body responsible for 
the ECOC delivery) was used to gain further insight of project managers' views 
on a variety of different issues linked to the design, delivery, benefits and legacy 
of the ECOC14; 

-  The evaluation does not include a wider public consultation. As explained in the 
roadmap15,  the  Action  is  considered  to  be  local.  International  participation  is 
scattered  within  and  outside  Europe  and  is  difficult  to  reach.  On  the  other  hand 

12   See 

previous 

reports 
europe/actions/capitals-culture_en (table at the bottom of the document). 

evaluation 

available 

at: 

https://ec.europa.eu/programmes/creative-

13   A total of 29 stakeholders for Donostia San Sebastian and 30 for Wroclaw were interviewed. 

14   29 projects responded to the survey in Wroclaw while 52 projects responded to the project survey carried out by 

the Donostia San Sebastian 2016 Foundation. 

15   Available at: http://ec.europa.eu/smart-regulation/roadmaps/docs/2016_eac_015_evaluation_ecoc_2016.pdf. 

10 

 
                                                            
the opinions would be based on attendance to specific events and would not give 
useful insights for the evaluation of the ECOC Action as a whole. 

The  final  report  of  the  contract  supporting  the  evaluation  provides  a  detailed 
understanding of the 2016 ECOC Action and within this an assessment of the work and 
progress of Donostia San Sebastian and Wroclaw. There are however issues to consider 
when assessing the strengths of the evidence base used for this study: 

-  Although both cities have undertaken some form of evaluation work themselves, 
part  of  the  results  of  these  studies  were  not  yet  available  at  the  time  of  the 
evaluation.  The  European  evaluation  of  the  ECOC  Action  has  used  as  much  of 
this secondary information as possible, but could not benefit from its final results, 
especially for what concerns quantitative data; 

-  As  already  mentioned  in  previous  evaluations,  an  ideal  way  of  conducting  this 
evaluation would entail a before ('baseline') study and an after-picture ('ex-post') 
study, instead of carrying out the latter alone; the impossibility of comparing the 
two studies affects the accuracy of the evaluation. Moreover, since the evaluation 
was  undertaken  during  and  shortly  after  the  end  of  the  ECOC  time  framework 
some  of  the  effects  of  the  programme  had  not  manifested  themselves  entirely. 
Many stakeholders involved in the evaluation commented that the real impact of 
the ECOC on the city and its residents would take time to filter through. In this 
respect,  the  Commission  once  again  highlights  that  budget16  and  timing17  only 
allow an ex-post evaluation to take place and therefore only an after picture has 
been studied;  

-  An  ulterior  consequence  of  the  modest  budget  allocated  is  the  fact  that  the 
primary evidence data gathering tends to be more of qualitative than quantitative 
nature; while qualitative data still holds a great importance in the evaluation, lack 
of  quantitative  data  translates  into  a  lesser  dependability,  for  instance,  in  the 
process  of  proving  the  objective  outcomes  and  impacts  of  ECOC  on  widening 
participation in culture. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

This part of the report presents a diagnosis of the implementation of the ECOC Action in 
2016. It draws together the results relating to the two ECOC 2016 and to a lesser extent 
the findings from previous from ECOC evaluations.  

All  figures  appearing  in  the  following  sub-sections  have  been  directly  drawn  from  the 
report of the external ex-post evaluation of the two ECOC 2016. The reader will find in 
the  full  text  of  the  document18  many  more  examples  illustrating  the  conclusions 
presented below. 

Very often, examples and figures can illustrate different points made under "relevance", 
"efficiency",  "effectiveness",  "sustainability",  "EU  added  value"  and  "coherence",  and 
are  therefore  sometimes  repeated  in  various  sub-sections.  However,  for  the  ease  of  the 
reading,  repetitions  have  been  limited  to  the  minimum  and  the  reader  is  invited  to 

16   The  budget  allocated  to  the  evaluation  work  (approximately  70  000  €)  is  proportionate  to  the  low  level  of  EU 
funding  directly  provided  to  the  ECOC  (i.e.  an  award  based  on  a  recommendation  of  the  panel  after  the  final 
monitoring meeting in the form of the €1.5m Melina Mercouri Prize). 

17   Decision No 1622/2006/EC requires that the Commission conducts the evaluation immediately after the title year. 

18   See footnote 5. 

11 

 
                                                            
correlate some conclusions presented below with examples or figures that may have been 
given in previous paragraphs or may be given in following sub-sections. 

5.1.  Relevance 

Findings from the 2016 ECOC evaluation show that the two title-holders have developed 
and  implemented  a  cultural  programme  that  is  consistent  with  the  EU  Treaty  and  in 
particular Article 16719. Indeed, in 2016 the ECOC Action continued to help EU Member 
States promote cultural diversity as well as highlight common elements, and added to the 
"flowering"  of  their  –  and  European  –  cultural  scene.  Indeed,  as  will  be  shown  in 
following  sub-sections,  in  the  two  cities,  thousands  of  events  took  place  on  top  of  the 
usual cultural offer, covering all types of cultural and artistic disciplines and presenting 
various  aspects  of  the  European  cultural  diversity  notably  through  co-operations  with 
partners from other (European) countries. 

Although  both  cities  clearly  understood  the  strategic  objectives  of  the  ECOC,  the 
programme in Wroclaw was more relevant to the EU Action objectives compared to that 
of Donostia San Sebastian.  

Donostia San Sebastian encapsulated well enough a European dimension at project level, 
with  its three key themes, namely multilingualism (with projects on European  minority 
languages,  such  as  Basque,  Welsh,  Cornish  and  Gaelic),  cohabitation  (with  projects  on  
European conflicts and how to avoid and overcome violence), and multiculturalism and 
migration (with projects focused on human rights, the roles of women or the coexistence 
with several religious identities often being found in one place). According to the survey 
of  participants,  30%  of  the  projects  specifically  promoted  a  sense  of  "European 
belonging"  with  a  further  24%  stating  that  they  collaborated  with  European  partners 
whilst developing and delivering their projects.  

The  project  "Europa  Transit"  is  emblematic  in  this  sense;  this  travelling  "embassy" 
visited ten cities that had lived through wars and conflicts: Ceuta (Spain), Belfast (UK), 
Dresden  (Germany),  Wroclaw  (Poland),  Sarajevo  (Bosnia-Herzegovina),  Moscow 
(Russia),  Paphos  (Cyprus),  Plovdiv  (Bulgaria),  Pristina  (Kosovo)  and  Thessaloniki 
(Greece). The work acted as a spotlight to share how people overcome conflicts and find 
mutual understanding with people who have lived through a similar history. The project 
also  included  a  "pan-European  Mashup"  which  presented  stories,  pictures  and  videos 
from the locals of the places visited on topics such as discrimination, injustice, solidarity, 
dignity and fear.  

Another project worth mentioning in this respect is 'Peace Treaty'. With an international 
aspect, it included a series of exhibitions, contemporary artistic productions, publications, 
seminars and conferences on the representation of peace in the history of art, culture and 
law.  In  this  way,  it  used  peace  as  the  main  theme  and  used  culture  to  communicate 
moments  of  Western  Society  that  included  peace  treaties  that  configured  to  Basque 
Society.  Part  of  this  looked  at  reasons  why  peace  is  important,  the  reasons  why  peace 
does not exist and the challenges of not having peace. 

However,  as  highlighted  by  a  majority  of  the  stakeholders  taking  part  in  the  European 
ECOC  evaluation,  the  global  objective  of  Donostia  San  Sebastian  2016  was  more 
focussed  on  local  issues  and  priorities  –  through  a  relatively  large  number  of  smaller 
rather  intimate  cultural  activities  found  throughout  the  year  instead  of  a  range  of  high 
profile and 'big hitting' events such as large concerts, exhibitions and festivals. This was 

19   Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A12008E167.  

12 

 
                                                            
very much linked to the needs and context of the city (i.e. dealing with its recent past of 
violence and reconciling its communities) and the strength and international connections 
of its existing cultural offer prior to the ECOC year (and the correlated absence of need 
to seek for much improvement in this respect). In this sense, it shows that all ECOC will 
also reflect local circumstances and ambitions, despite the ECOC being an EU title.  

Donostia San Sebastian focussed on using the ECOC to strengthen the cultural software 
rather than hardware of the city, the latter being already quite developed in the city prior 
to the ECOC year. Rather than developing new buildings or facilities like Wroclaw did, 
the city used the ECOC to put on a sensitive cultural programme that partly helped the 
citizens  of  the  city  overcome  some  of  the  difficulties  of  the  past  terrorist  activities  of 
ETA – thus meaning it was very relevant and unique to the city and its past.  

On  the  other  hand,  it  appears  from  the  final  report  of  the  contract  supporting  the 
evaluation that Wroclaw used the ECOC to further strengthen the European character of 
the city, raise its international profile and, crucially, explore some of the more difficult 
elements of its recent history. It also implemented the ECOC as part of a wider strategy 
to develop the city on the basis of culture, tourism and sport. 

Indeed,  several  themes  of  European  significance  were  presented  and  explored  by  the 
cultural  programme.  These  were  diverse  and  included  issues  such  as  Polish-German 
reconciliation, cultures from different European countries and review of trends emerging 
in the work of young artists from across Europe.  

The  majority  of  respondents  to  the  survey  of  organizations  that  implemented  projects 
reported that their projects had a European dimension: 50% said that their projects sought 
to  increase  awareness  of  cultural  diversity,  50%  that  their  projects  highlighted 
intercultural  dialogue  and  mutual  understanding  and  45%  that  their  projects  included 
themes of relevance or significance across Europe with only 30% reported no European 
dimension.  

In this respect,  "Migrations and Metamorphoses" was a particularly important theme in 
the promotion of the ECOC European dimension; it was dedicated to the cultural effects 
of the population transfer that took place in the city at the end of World War II20, but also 
on  the  modern  phenomenon  of  migration  into  and  across  Europe.  Among  the  many 
relevant  projects  to  this  theme  there  was  "Unfinished  Palace.  About  people,  migration, 
and  borders",  that  consisted  in  a  series  of  multimedia  events  at  the  former  Świebodzki 
Railway  Station  over  two  weeks,  including  exhibitions,  concerts,  readings  and 
performances, as well as "Open Call Europe", a long-term and interdisciplinary artist-in-
residence  programme  which  invited  artists,  art curators,  cultural  activists  and  managers 
from the 27 EU Member  States, as well as Albania,  Bosnia and Herzegovina,  Georgia, 
Iceland,  Israel,  Kosovo,  Liechtenstein,  FYROM,  Moldova,  Montenegro,  Morocco, 
Norway, Serbia, Switzerland, Turkey and Ukraine.  

Wroclaw 2016 also featured numerous collaborations with cultural operators from other 
European  countries  and  elsewhere  in  the  world.  For  example,  the  "European  Student 
Symphony  Orchestra",  which  created  an  orchestra  with  young  musicians  from  selected 

20   During the Second World War, Jewish and Polish citizens were persecuted by the Third Reich, although the city 
largely escaped damage until the final few months of the war. The advancing Soviet armies surrounded the city 
during the three-month-long “Battle of Breslau” which culminated in considerable destruction, loss of life and the 
eventual  surrender  by  German  forces  on  6  May  1945.  After  the  war,  the  city  became  part  of  Poland  under  the 
terms  of  the  Potsdam  Agreement.  Most  of  the  remaining  German  population  of  190,000  fled  or  were  forcibly 
removed,  to  be  replaced  by  ethnic  Poles,  most  of  whom  were  resettled  from  parts  of  eastern  Poland  that  were 
incorporated into the Soviet Union. 

13 

 
                                                            
European  schools,  which  then  performed  in  Wroclaw  and  elsewhere  in  Poland  and 
internationally.  

Furthermore, with the view to connecting with audiences from abroad and further profile 
Wroclaw in other cities, events were also organised in other countries, such as a major 
exhibition on "The Wild West. A History of Wroclaw's Avant-Garde" in Warsaw, as well 
as in Bochum (Germany), Budapest (Hungary), Kosice (Slovakia) and Zagreb (Croatia). 

Of those organizations responding to the project survey, 81% reported that their project 
involved cultural organisations or artists from other countries in some way (presence of 
performers from other countries, exhibition of works from other countries, international 
cultural exchanges, performance of Polish performers abroad or collaboration with non-
cultural  organisations  or  people  from  abroad).  As  a  whole,  projects  reported 
collaborations with organisations or artists from 42 other countries, mostly in Europe. 

5.2.  Efficiency 

Overall, and despite the absence of a strong costs-benefits analysis, the report concludes 
that the ECOC remains an efficient EU Action, providing good levels of benefits for the 
EU  for  relatively  little  EU  investment.  Indeed,  the  only  direct  contribution  from  the 
European  Union,  in  the  form  of  the  €1,5m  Melina  Mercouri  prize  awarded  to  each 
ECOC, is dwarfed by the total amount of money invested by the host cities in designing 
and  delivering  the  ECOC  (the  operational  budgets  were  approximately  €86,4m  for 
Wroclaw and €49,6m for Donostia San Sebastian).  

Without the initial EU impetus to support the ECOC then it is unlikely that the host cities 
would have invested anywhere near the amount of funding they did in connection with 
the ECOC title. The possibility of securing the title typically stimulates cities – but also 
their respective regional and national public authorities – to invest much more heavily in 
their  cultural  offer  than  they  would  in  the  absence  of  the  ECOC  –  both  in  terms  of 
infrastructure (this  was more the case in  Wroclaw than in Donostia San Sebastian) and 
expenditure on cultural events and operations. The independent evaluation concludes that 
this  means  the  Action  remains  highly  efficient  in  terms  of  returns  from  the  Melina 
Mercouri prize.  

On  the  other  hand,  the  final  report  of  the  contract  supporting  the  evaluation  also 
highlights  that  delivering  an  ECOC  is  still  very  challenging  at  the  city  level  from  an 
efficiency  point  of  view.  Any  city  considering  applying  for  ECOC  status  needs  to 
recognise that it will need to  generate significant  levels of funding from various public 
and private stakeholders if it wants  to deliver a strong cultural programme and that the 
returns on the investment are sometimes intangible (e.g. better access to culture).  

In  this  respect,  it  is  worth  mentioning  that  Donostia  San  Sebastian  had  difficulties  in 
securing the budget initially sought for. Ultimately, it had to deliver a programme with a 
budget  that  was  44%  lower  than  in  its  original  planning.  The  Foundation  itself  was 
relatively  poorly  resourced  in  terms  of  staff  before  2016.  Just  three  years  before  the 
ECOC  year  it  had  less  than  5  staff  members  and  by  2014  staff  numbers  were  still 
relatively modest (13). Despite this, looking at the visitors' numbers, the audience figures 
(1,08  million,  despite  the  quasi-absence  of  'big  ticket'  events,  which  are  expensive  to 
produce) and the level of cultural activity (3,475) it is clear that high levels of efficiency 
were  achieved.  As  far  as  Wroclaw  2016  is  concerned,  it  was  on  the  contrary  able  to 
attract  a  very  high  level  of  financial  contribution  from  the  State  (€31.7m,  representing 
37%  of  the  whole  operational  budget)  as  well  as  of  corporate  sponsorship  in  culture 
(€7m).  

14 

 
Like  in  previous  years,  the  ECOC  in  2016  has  stimulated  many  thousands  of  'extra' 
cultural  activities  and  the  cities  have  both  gone  out  their  way  to  generate  much  more 
depth  and  volume  of  cultural  activity  compared  to  their  usual  cultural  offer  – 
approximately 2,000 more activities than the standard figure in Donostia San Sebastian; 
and 2,000 artistic events as a whole in Wroclaw, many of them new and in new settings 
such  as  urban  spaces  (see  following  sub-section).  New  activities,  new  resources,  new 
audiences and new cultural infrastructure (as illustrated in following sub-sections) have 
all been generated across 2016 as a direct consequence of the ECOC. 

The  delivery  mechanisms  established  in  both  San  Sebastian  and  Wroclaw  were  strong 
and  successful.  Donostia  San  Sebastian  delivered  its  programme  in  relatively  difficult 
circumstances including political changes at various levels of territorial governance (City 
in  2011  and  2015,  but  also  Province,  Region  and  State  in  2015  and  2016)  and 
comparatively  high  levels  of  political  influence,  making  'culture  to  overcome  violence' 
even more of a 'brave' subject to engage with, when it would have been easier to deliver 
'more  of  the  same'  type  of  culture.  This  higher  than  normal  (in  comparison  with  other 
ECOC)  political  involvement  brought  about  a  range  of  issues  that  were  often  seen  as 
adding  an  extra  level  of  complexity  to  the  governance  structures  of  the  ECOC,  in 
particular  because  it  meant  that  some  key  decisions  (for  example,  on  what  the  legacy 
plans of the programme should be) were delayed or sometimes not taken at all.  

The  governance  arrangements  in  Wroclaw  were  generally  stable  throughout  the 
development  and  implementation  stages,  albeit  with  a  very  important  change  in  the 
artistic  direction  (i.e.  the  move  from  a  single  Artistic  Director  to  a  Board  of  Curators). 
The  consensus  amongst  stakeholders  was  also  that  Impart  (the  delivery  body)  and  the 
Curators  enjoyed  a  high  degree  of  artistic  independence  and  freedom  from  political 
control  at  local  and  national  level  (within  the  usual  constraints  associated  with 
accountability for public expenditure) and operated efficiently. 

5.3.  Effectiveness 

Ultimately, the effectiveness of the ECOC in achieving the objectives set for them (at EU 
level and local level) can only be determined in the long-run. That being said, the final 
report  of  the  contract  supporting  the  evaluation  however  shows  that  both  cities  have 
proved successful in fulfilling the short-term objectives, most notably the implementation 
of  extensive,  innovative  cultural  programmes,  which  had  a  European  dimension  and 
involved local citizens (both title-holders) and international visitors (Wroclaw more than 
Donostia  San Sebastian).  It appears from the final  report of the contract supporting the 
evaluation  that  none  of  the  stakeholders  in  either  of  the  two  cities  felt  that  more  could 
have been done and that 2016 was ineffective, a waste of resources or should have been 
done significantly differently. The final report also mentioned that the overriding 'feeling' 
on  the  effectiveness  of  their  ECOC  was  one  of  positivity  and  supportiveness  towards 
their  title  year,  even  for  those  stakeholders  who  were  interviewed  specifically  because 
they were initially against the ECOC. 

Effectiveness in delivering a cultural programme with a European dimension 

Donostia  San  Sebastian  succeeded  in  delivering  a  cultural  programme  that  was  very 
wide,  varied  and  more  extensive  than  in  in  a  'non-ECOC'  year,  for  an  expenditure  of 
almost €28m (62% of the total expenditure of the ECOC year). The programme consisted 
of 3,475 activities (as against 1,500 in a normal year), with theatrical activities making up 
33%  of  the  programme,  musical  activities  25%,  audio  visual  20%,  literacy  7%  and  the 
remainder  being  a  mix  of  other  forms  of  art  (i.e.  visual  arts  etc.).  Those  designing  the 
original cultural programme were keen to not simply put on 'more of the same' in relation 

15 

 
to delivering or repeating more 'film festivals, jazz and high-quality art exhibitions' that 
the  city  already  had  and  which  local  people  were  used  to.  According  to  the  local 
evaluation that was carried out, 70% of the projects were specifically put on for 2016 and 
did not exist prior to the year. Furthermore, 65% of the projects said that they would not 
have gone ahead without the stimuli (the funding) of the ECOC. 

The Donostia San Sebastian's programme didn't just amplify the city's cultural offer, but 
it clearly differentiated it from the pre-existing cultural activity, using culture as a vehicle 
for healing wounds inherited from the past.  

However, although the ECOC in San Sebastian included themes and projects which had a 
European dimension some of the strategic stakeholders taking part in the evaluation felt 
that  the  overall  programme  generally  had  a  firm  local  rather  than  European  focus. 
Despite  some  of  the  topics  being  relevant  at  the  European  level,  like  terrorism, 
cohabitation and multilingualism – which were strongly present in the programme, some 
stakeholders felt that the ECOC targeted and supported local citizens and culture rather 
than European ones. This has to be read keeping in mind that the content of the cultural 
offer  of  the  city  before  and  outside  of  the  ECOC  was  already  very  European  and  to  a 
large extent already had aspects with a strong European dimension – for instance, the San 
Sebastian International Film Festival and the San Sebastian Jazz Festival.  

In  Wroclaw,  the  ECOC  cultural  programme  was  more  extensive,  innovative  and 
European in nature compared to the city's cultural "baseline" offering in previous years. 
It should be noted that the city's cultural offering had been steadily expanding in scope 
and diversity over the ten or more years preceding 2016, reflecting the long-term strategy 
of the City of Wroclaw. However, the title-year saw a further, marked increase in both 
the scale and the diversity. As a whole, 424 projects totaling around 2,000 artistic events 
took  place.  They  were  articulated  in  the  eight  disciplinary  sub-programmes  indicated 
above and represented an expenditure of almost €27m (47.4% of the total expenditure of 
the  ECOC  year).  Of  the  organizations  that  responded  to  the  project  survey  on  cultural 
capacity, 30% reported that they implemented entirely new activities, 39% said that their 
activities were operated at greater scale in 2016 while 30% reported that their activities 
were undertaken at the same scale in 2016 as in previous years.  

The  eight  curators  of  the  eight  disciplinary  sub-programmes  were  provided  with  the 
opportunity  and  the  resources  to  test  new  ideas.  Some  new  activities  with  a  European 
dimension  were  introduced  into  Wroclaw's  cultural  offering,  including  a  programme  of 
artistic  residences  for  artists  from  other  European  countries.  Of  the  organisations  that 
responded to the project survey, 72% believed that the Wroclaw2016 cultural programme 
featured  a  good  balance  of  traditional  and  avant-garde  cultural  expressions  and  80% 
believed that it was of high artistic quality. Furthermore, among the projects responding 
to  the  survey,  81%  had  involved  cultural  organisations  or  artists  from  other  countries. 
Just  to  give  a  few  examples,  Wroclaw  hosted  the  "Theatre  Olympics",  an  international 
theatre festival, which presented the achievements of some of the most prominent theatre 
practitioners  from  around  the  world,  and  its  New  Horizons  International  Festival  was 
extended to include masterclasses as well as a presentation of Basque cinema as a way to 
showcase the other ECOC 2016. 

Effectiveness  in  reaching  and  engaging  with  local  citizens  and  in  targeting  specific 
groups  

As already mentioned, Donostia San Sebastian used its ECOC to tackle a complex and 
sensitive issue, namely past violence and the differences that the city's communities have 
with  one  another,  and  the  focus  was  perhaps  more  than  in  any  other  ECOC  on  local 

16 

 
citizens. It is difficult to understand the effectiveness that the ECOC had on this central 
issue  in  a  quantitative  or  'empirical'  way.  Despite  this,  the  report  supporting  the 
evaluation concludes that the ECOC went hand-in-hand with a very solid and substantial 
local  involvement  and  was  effective  in  helping  people  open  up  about  their  differences 
and also talk about the past issues that faced the city.  

According to the local evaluation, 60% of the projects involved local people in some way 
(undertaking research with local residents and organizations, undertaking in-depth work 
and co-designing cultural activities with them or involving them in the actual delivery of 
cultural activities). The project "Forum Theatre: what about you" is a perfect illustration 
of  how  citizens  were  pro-actively 
involved.  This  project  addressed  feelings, 
contradictions and archetypes provoked by violence and social unrest. It used the model 
of  the  "Theatre  of  the  Oppressed",  giving  the  audience  the  power  to  decide  the 
development  of  the  narration.  40  sessions  of  this  project  successfully  took  place  with 
good  audience  throughout  the  year.  Moreover,  a  volunteer  programme  run  centrally  by 
the Foundation counted up to 2,373 volunteers in 2016 (with 41% of them not having any 
volunteering experience before the ECOC year), amounting to 10,493 hours of volunteer 
time.  Citizens'  participation  was  further  stimulated  by  the  "Waves  of  energy"  project, 
which  gave local  individuals  and  non-profit  organizations opportunities  to  present their 
cultural  projects  for  development,  of  which  sixty-six  were  financed  with  60  grants  of 
€2,000 and six of €20,000. Some of these projects were really intimate and personal, and 
were meant to help people deal with the violence they had experienced in their own lives. 
Although  staff  in  the  Foundation  was  involved  in  facilitating  the  project,  local  people 
made all the key decisions in terms of who and what themes benefitted from the funding. 

Against  this  backdrop,  there  is  little  evidence  of  the  ECOC  in  Donostia  San  Sebastian 
targeting  groups  in  the  city  with  specific  activities  of  the  cultural  programme  –  for 
instance, designing projects specifically for older residents or the disabled. Having said 
this, audience figures show that 58% of the audience of ECOC projects were female and 
that  15%  were  under  25  and  nearly  half  were  under  39  years  old.  More  generally,  the 
baseline survey undertaken by the local evaluators shows that the number of local people 
that 'actively participated in culture on a regular basis' (twice a month) grew slightly from 
60% in 2015 (a high figure already) to 64% in 2016. However, the report also highlights 
that  the  absence  of  'mega  events'  negatively  impacted  on  local  people  attending  ECOC 
projects.  

As  far  as  Wroclaw  2016  is  concerned,  it  also  involved  a  large  numbers  of  citizens, 
including  children  and  young  people:  while  around  2,000  people  served  as  volunteers, 
numerous projects and events enabled citizens to participate as creators and performers. 
In total, more than 170,000 people actively participated in projects, of which 50,000 were 
children  and  young  people  (e.g.  15,000  local  school  pupils  participating  in  the 
educational  programme  related  to  the  Film  sub-programme  with  the  support  of  the 
educational and academic community).  

For  instance,  the  "Backyards"  project  sought  to  initiate  a  creative  process  based  on 
collaboration  between  artists  (most  coming  from  outside  Wroclaw),  local  residents, 
property owners and other local institutions. The artists were invited to select one of the 
many backyards in the city and generated ideas for a creative activity. Once the backyard 
had  been  selected,  the  project  co-ordinators  would  then  identify  the  owners  of  the 
backyard and thus seek the relevant permissions to operate. The artists would then begin 
the  fieldwork  and  the  consultation  with  residents.  Over  a  two-year  period  (2015-16),  a 
total  of  32  projects  took  place  in  42  backyards,  7  estates  and  5  backyard  walks.  The 
artistic  activities  varied  and  included  lighting  installations,  workshops  for  children, 

17 

 
exhibition of residents’ objects, collecting residents’ stories and making them available in 
audio form, involving residents in one block writing letters to residents in another block 
etc. 

Contrary  to  what  happened  in  Donostia  San  Sebastian,  many  of  the  activities  to 
encourage the participation of citizens were aimed at specific groups. The respondents to 
the  survey  of  organizations  that  implemented  projects  particularly  highlighted  children 
(75%  of  projects),  older  people  (70%),  people  with  disabilities  (50%),  poor  or 
disadvantaged communities (40%) and minority ethnic groups (30%). As an example, the 
intention  behind  the  music  project  "Yiddish  and  Ladino"  aimed  at  engaging  the  many 
societies and artistic groups associated with ethnic groups living in Wroclaw, in this case 
by opening up the city for a presentation of the Jewish culture. Another example was the 
"Adapter – Cinema Without Barriers" project, targeting at people with hearing and visual 
impairments,  which  included  educational  activities  and  a  video-on-demand  portal  of 
more than 100 film with audio-description and subtitles. There was also a special focus 
on involving foreigners, for example through the project "Polish Cinema for Beginners", 
a series of screenings with English subtitles targeted at foreigners living in Wroclaw. 

The  most  prominent  project  in  this  aspect  was  "MikroGRANTY"  ("micro-grants").  To 
effectively build local cultural capacity and facilitate the participation of local groups and 
citizens,  this  programme  was  open  to  applications  from  private  individuals,  informal 
groups,  NGOs,  home-owner  associations,  social  cooperatives,  church  institutions, 
religious  organizations  and  other  similar  groups.  Applicants  could  submit  projects 
relating to any artistic discipline, but their artistic expressions had to relate to the overall 
ECOC concept of "Spaces for Beauty", chosen slogan for the ECOC programme whose 
intention was "to create spaces within which to restore the presence of beauty in public 
life and in daily habits". In total, 115 projects were supported, with a grant support of up 
to  about  €1,175.  Taking  only  those  projects  implemented  in  2016,  some  190  partner 
bodies were involved, as well as 319 people (including 179 volunteers) and 3,690 active 
participants were reached with a total of 12,000 "likes" on Facebook. 

Effectiveness in reaching out to European and international audiences  

The  final  report  of  the  contract  supporting  the  evaluation  highlights  that  the  local 
orientation of the ECOC in Donostia San Sebastian made fruition of the ECOC activities 
more difficult for non-locals. As an example, although some of the literature associated 
with  the  ECOC  was  found  in  different  languages  (e.g.  the  monthly  'What's  on' 
publication was  40%  in  Basque,  40%  in  Spanish,  10%  in  English and  10%  in  French), 
the proportion of literature translated into English was relatively low compared to other 
ECOC. 

Against  this  backdrop,  it  however  appears  from  the  report  that  7.6%  of  the  ECOC 
audiences were from another European country – this is in line with the average ECOC 
level, which is estimated between 5-10% – and that out of the 1.08 million people who 
attended  ECOC  events,  84%  came  from  Donostia  San  Sebastian  or  its  close 
surroundings. Furthermore, figures from the tourist agency show that the city saw a slight 
uplift  in  foreign  visitors  by  1.5%  in  2016  compared  to  2015,  but  it  is  not  possible  to 
directly connect this increase with the ECOC. 

In  Wroclaw, one  of the explicit goals of the ECOC was to  increase tourism and  attract 
international  visitors.  In  spite  of  a  very  modest  budget,  Wroclaw  run  an  effective 
marketing and communication campaign targeting among others Europe and the world, 
which  put  a  considerable  emphasis  on  co-operation  with  other  agencies  (for  joint 
representation  at  promotional  events  and  tourism  fairs)  as  well  as  on  attracting 

18 

 
international  journalists.  In  total,  Impart  organized  visits  by  153  journalists  from  26 
countries and welcomed about another 100 who organized their own visits.  

Given  Wroclaw's  history  and  location,  there  was  a  particular  focus  on  Germany.  For 
example,  with  the  co-operation  of  the  relevant  transport  bodies,  a  special  direct  train 
service from Berlin was established: the "Culture Train", operating at the weekends from 
30 April 2016 onwards. The "Culture Train" itself presented a range of culture, including 
literature,  music,  theatre  and  exhibitions  and  more  than  40  artists  participated  in  the 
programme. In total, nearly 23 000 passengers made use of the train in 2016, and given 
these passenger numbers, it was decided that the service would continue at least until the 
end of 2018. 

According  to  research  undertaken  on  behalf  of  the  City  of  Wroclaw,  around  5  million 
tourists visited  Wroclaw in 2016, of which 1.6m came from other countries, marking a 
substantial increase in international tourist stays during the year. Data from the Central 
Statistical  Office  of  Poland  shows  that  2016  was  a  year  of  growth  in  tourist  stays  in 
Wroclaw  (+5,2%  compared  with  2015,  after  already  two  years  of  sharp  increases)  and 
that  that  an  extra  50,000  international  tourists  stayed  in  the  city's  hotels  in  2016 
compared to 2015. 

5.4. 

Sustainability 

The  two  2016  ECOC  cities  offer  very  different  potential  for  the  sustainability  of  their 
activities and of improved cultural governance.  

For  what  concerns  Donostia  San  Sebastian,  the  main  finding  of  the  evaluation  is  that 
legacy  planning  of  the  ECOC  has  been  generally  limited,  with  a  lack  of  a  clear  and 
formal  strategy  from  the  outset:  compared  to  other  ECOC  bid  books,  there  was  in 
Donostia San Sebastian's bidding document very little detail on the planned activities to 
sustain engagement with the themes and projects of the ECOC year. One driver behind 
this shortcoming was identified through consultations with stakeholders as the deficiency 
of joint understanding by the political parties on what an ECOC legacy should look like. 
The process of shaping this perspective fell under the responsibilities of the ECOC Board 
(rather than the Foundation). 

Although the cultural operators were all generally still in place and still delivering strong 
cultural content beyond 2016, this was more due to the strength of the existing cultural 
infrastructure  found  in  the  city,  rather  than  any  specific  legacy  work  from  the  ECOC 
programme. While there hasn't been an outstanding increase in cultural capacity – which 
was already extremely well developed in the city prior to the ECOC year in the city and 
still  remains  of  an  excellent  standard  –  the  ECOC  year  highly  motivated  stronger 
linkages between local players: almost 80% of organizations delivering an ECOC project 
worked  in  cooperation  with  other  actors  in  order  to  deliver  their  projects  and  67%  of 
them were new relationships that had previously not existed prior to 2016.  

Another aspect of the legacy issue in Donostia San Sebastian is linked to a lack of new 
cultural  buildings:  as  stated  earlier,  the  ECOC  was  generally  focused  on  cultural 
content/events  rather  than  buildings/infrastructures,  making  the  lasting  legacy  of  the 
programme even more difficult to grasp. 

As  in  many  ways  the  goals  of  the  ECOC  in  San  Sebastian,  its  results  and  focus  were 
socially and politically oriented with a strong focus on reconciliation, its legacy is more 
of a "soft" nature, and articulating its impact is therefore harder. Even stakeholders taking 
part  in  the  evaluation  found  it  hard  to  express  the  legacy  of  a  cultural  theme  aimed  at 
'peace' or community cohesion compared to describing the lasting legacy and impact of a 

19 

 
new theatre or opera house. Moreover, unlike Wroclaw, Donostia San Sebastian did not 
have  a  strong  ambition  to  improve  tourism  by  linking  it  to  the  ECOC  Action,  thus 
making yet another indicator of objective success of the Action unavailable.  

Despite the difficulties outlined above, a key result of the programme overall was often 
recognized  by  stakeholders  taking  part  in  the  European  evaluation  as  being  one  which 
brought the political parties closer together to actively work on a 'city level' project and 
on an activity that was genuinely 'positive', therefore influencing in the longer-term the 
way the city operates internally. 

On the other hand, legacy planning was much developed and coordinated in Wroclaw as 
the ECOC was implemented as an integral part of a long-term development strategy with 
strong and linear political support at local level, and therefore offers greater potential for 
long-term  sustainability  of  the  ECOC  impacts.  This  strategy  is  also  being  informed  by 
the various research and evaluation activities undertaken by the University of Wroclaw, 
which  included  recommendations  for  enhancing  the  "mikroGRANTY"  programme 
beyond 2016: for the first time, a dedicated funding stream will be available to nurture 
grass-roots  cultural  initiatives,  not  only  through  finance  but  also  through  the  technical 
and administrative support offered by Impart. 

On the long-term scheme, Impart together with the City's Departments for Social Affairs 
and for Culture have put forward a proposal for a strategy for 2017-2020 and beyond – 
"Culture  –  The  Current!"  ("Kultura  –  Obecna!")  –,  building  among  others  on  the 
experience  of  the  ECOC.  The  strategy  lists  four  strategic  objectives  (and  31 
recommended actions):  

-  Maintaining the prestige of Wroclaw culture;  
-  Developing the cultural infrastructure of Wroclaw;  
-  Ensuring conditions for the development of culture; and  
-  Ensuring an increase in participation in culture in Wroclaw.  

Additionally,  Wroclaw  has  the  potential  to  be  sustained  by  the  continued  operation  of 
Impart from its premises in the city centre and with many of the key staff members still 
in post. Impart will therefore continue to play a key role in coordinating cultural activities 
in the city, as well as in implementing specific projects. Furthermore, other major events 
such as the World Games 2017 (an international multi-sport event involving sports that 
are not contested in the Olympic Games) already built – or will build – on the capacity 
and profile generated by the ECOC.  

As is often the case with ECOC, Wroclaw 2016 will also have soft legacies in terms of 
stronger  skills  for  its  cultural  sector,  stronger  relationships  (in  particular  at  European 
level),  a  stronger  and  more  vibrant  cultural  life  (that  is  the  opinion  of  74%  of  those 
organisations that responded to the projects survey) and – more widely – a higher profile 
for culture in the city (according to 87% of firms in the cultural and creative industries 
surveyed by the University of Wroclaw). 

Finally, the ECOC in Wroclaw was supported by and provided an impetus to investments 
in  the  cultural  infrastructures  of  the  city,  for  instance  with  the  opening  of  the  National 
Forum of Music concert venue. Various locations in the city were also used in new ways 
as  venues  for  cultural  events,  such  as  rivers  and  riverbanks  within  the  city  centre.  The 
(re)opening  of  several  new  (or  refurbished)  venues  and  the  general  modus  operandi  of 
taking  culture  out  of  the  main  venues  in  the  city  centre  and  "spread"  it  around  many 
other,  often  new  or  unusual  locations  will  also  provide  with  a  tangible  kind  of  legacy. 
Reflecting  this,  83%  of  respondents  to  the  project  survey  reported  that  the  ECOC  had 
improved local cultural infrastructure.  

20 

 
5.5.  Coherence  

The  ECOC  has  stimulated  the  two  cities  to  use  European  Regional  Development  Fund 
(ERDF)  for  investments  in  cultural  infrastructure  and  has  given  greater  impetus  to  the 
completion of those investments in time for the title-year. 

In particular in Wroclaw, there was a wider strategy for the development of culture and 
tourism  from  the  early  2010s  that  foresaw  investments  in  physical  infrastructure  and 
facilities. 

It  included  improvements  in  the  rail,  road  and  air  transport  infrastructure  of  Wroclaw, 
some  of  which  was  funded  by  the  ERDF.  Funding  from  the  ERDF  also  supported  the 
creation or renovation of cultural infrastructures in the city – for instance, nearly €41m of 
ERDF  funding  was  invested  in  the  new  venue  of  the  National  Forum  for  Music  which 
opened  in  September  2015.  In  turn,  the  ECOC  helped  raise  awareness  of  this  new 
cultural  facility  and  contributed  to  the  NFM's  considerable  success  in  attracting 
audiences  in  its  first  full  year  of  operation  (more  than  500,000)  as  much  of  the  music 
programme  of  the  ECOC  took  place  at  the  new  NFM  with  for  example  a  series  of  14 
concerts exploring "1000 Years of Music in Wroclaw" all along the year.  

EU cross-programme funding has also helped maximize the results of the ECOC Action. 
For instance, Donostia San Sebastian's project "With Corners of Europe", that connected 
six  Basque artists  with  another  60 European  ones,  was  co-funded  by  the  EU's  Creative 
Europe programme and produced activities in Umea and Stockholm (Sweden), Ljubljana 
(Slovenia),  Prizren  (Kosovo),  Gdansk  (Poland),  Zagreb  and  Rijeka  (Croatia),  Donostia 
San Sebastian (Spain), as well as Belfast, Northumberland and Middlesbrough (UK). 

The ECOC  Action is therefore relevant  and complementary to  a variety of EU policies 
and  programmes,  impacting  not  just  cultural  stakeholders  but  also  those  related  to 
employment  (with  its  impact  on  capacity  building  for  example),  enterprise  (as  an 
instance, 14% of firms from the cultural and creatives sectors in Wroclaw were involved 
in the ECOC, i.e. about 450 out of +3,000 firms, while 52% of them felt that they had 
derived commercial benefits from the ECOC and 40.7% reported an increase in turnover 
during 2016) or tourism as illustrated in the various sub-sections above. 

5.6.  EU added value 

As  already  mentioned  and  illustrated  above,  the  ECOC  Action  has  achieved  an  impact 
that would not have arisen through the actions of Member States alone.  

The designation of Donostia San Sebastian and Wroclaw as ECOC has attracted benefits 
that would have been unlikely to arise to the same extent in the absence of the Action. 
The  'label'  itself  is  a  key  aspect  of  the  EU  added  value  of  this  Action  as  it  acts  as  a 
significant  generator  of  interest  from  stakeholders  not  only  from  the  city  and  but  also 
from  far  beyond;  for  example,  Wroclaw's  growth  in  terms  of  corporate  sponsorship, 
cultural participation, visibility and local/international tourism was fairly consistent with 
and directly connected to the fact that the city was European Capital of Culture in 2016. 

Moreover,  for  both  cities  there  was  great  scope  for  European  cooperation  in  terms  of 
partnership  and  transfer  of  good  practices,  both  from  one  ECOC  to  the  other  and  in 
connection with other European organizations. 

The  results  consistently  achieved  by  the  Action  through  the  years  establish  that  its 
models  and  mechanisms  –  for  example  in  terms  of  building  a  solid  governance  for  the 
delivery of the ECOC, increasing the capacity of local cultural organizations or attracting 

21 

 
projects'  ideas  from  local  residents  –  can  be  applied  not  only  regionally  or  within  an 
individual Member State but also EU-widely, thus finally confirming its EU added value.  

6.  CONCLUSIONS 

The lack of baseline data to be integrated in a comparative study of the city prior to the 
win  of  the  title,  start  of  title  year  and  after  the  implementation  of  the  ECOC  year  is  a 
limitation  to  the  study  carried  out  by  the  contractors.  These  data  are  considered  of 
paramount importance in order to achieve a balanced perspective, supported by a cogent 
and ample data basis, of the actual impact of the Action on a city. However the budget 
allocated to the evaluation work (approximately 70 000 € each year) is proportionate to 
the  low  level  of  EU  funding  directly  provided  to  the  ECOC  (€1.5m  Melina  Mercouri 
Prize) and doesn't make it possible to have a before ('baseline') study and an after-picture 
('ex-post')  study.  An  ulterior  consequence  of  the  modest  budget  is  that  the  primary 
evidence  data  gathering  tends  to  be  more  of  qualitative  than  quantitative  nature;  while 
qualitative data still holds a great importance in the evaluation, lack of quantitative data 
translates into a lesser dependability, for instance, in the process of proving the objective 
outcomes and impacts of ECOC on widening participation. 

The  following  conclusions  are  therefore  substantiated  by  an  ample  basis  of  qualitative 
and "soft" data (e.g. the views and opinions of various types of stakeholders) more than 
by  a  comprehensive  quantitative  set  of  data.  The  Commission  is  fully  aware  of  –  and 
accepts – those limitations. 

The provisions of Decision 445/2014/EU, which will cover the ECOC titles from 2020 to 
2033, foresee that the cities (starting with the ECOC 2020)  shall be responsible for the 
evaluation  of  the  results.  The  requirement  to  have  a  monitoring  and  evaluation  plan  is 
therefore part of the criteria used to assess the applications from bidding cities and should 
allow  for  the  collection  of  data  at  early  stages  so  as  to  establish  baselines.  This 
requirement  is  already  an  element  of  the  monitoring  process  cities  designated  for  the 
ECOC titles from 2020. As specified in Article 14 of Decision 445/2014/EU, the Melina 
Mercouri Prize money will be paid by the end of March of the year of the title, provided 
that the designated  city concerned continues to honour  the commitments it made  at the 
application stage, complies with the criteria and takes into account the recommendations 
contained in the selection and monitoring reports. To give evidence that the commitments 
they  made  at  selection  phase  are  honoured,  cities  have  among  others  to  show  that  "the 
plans for the monitoring and evaluation of the impact of the title on the designated city 
are in place" at the end of the monitoring phase.  

On top of that, according to Article 16 of Decision 445/2014/EU, the Commission will 
ensure  external  and  independent  evaluations  that  "shall  focus  on  placing  all  past 
European Capitals of Culture in a European context, allowing comparisons to be drawn 
and useful lessons to be learned for future European Capitals of Culture, as well as for all 
European cities". Those evaluations will "include an assessment of the action as a whole, 
including  the  efficiency of the processes involved  in  running it, its impact and ways in 
which it could be improved." The first of these evaluations is due in late 2024. 

Against this backdrop, the Commission concludes that the ECOC Action is highly valued 
by the hosting cities that can obtain positive impacts during the year, as well as during 
the preparation phase. The action also remains relevant at EU level. However, the long-
term  impacts  of  the  two  ECOC  2016  cannot  be  assessed  at  this  stage  as  it  is  too  early 
after the implementation of the ECOC year. 

22 

 
 
The Commission also concludes that the programmes implemented by the two 2016 title-
holders  were  innovative  and  consistent  with  the  objectives  of  the  ECOC  Action;  they 
reflected its European dimension (in particular Wroclaw, to a lesser extent Donostia San 
Sebastian),  involved  local  residents  and  stakeholders  (in  San  Sebastian  with  60%  of 
projects involved local people in some way according to the local evaluation and 10,493 
hours  of  volunteer  time  while  Wroclaw  designed  cultural  activities  aimed  at  specific 
groups), brought culture to new audiences through specific strategies (such as "Waves of 
Energy in Donostia San Sebastian and "MikroGRANTY" in Wroclaw) and may lead to 
some  legacies  both  physical  (in  the  form  of  new  or  refurbished  cultural  and  logistic 
infrastructures, as it happened in Wroclaw) or intangible (by creating a space and vehicle 
that helped communities talk about, understand and come to terms with past differences 
and  helping  the  city  and  its  residents  coexist  with  one  another  in  the  future,  as  it 
happened  in  Donostia  San  Sebastian)  although  a  proper  legacy  planning  is  lacking  in 
Donostia San Sebastian.  

23 

 
 
 
ANNEX 1: PROCEDURAL INFORMATION 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

The evaluation was led by Directorate General for Education, Youth, Sport and Culture 
(DG EAC). It is included in the Work Programme of Creative Europe for 2016 and in the 
Agenda Planning with the reference 2016/EAC/026. 

2.  ORGANISATION AND TIMING 

The evaluation was supported by an external and independent evaluator, under a service 
contract.  The  service  contract  was  implemented  via  a  Framework  Contract  with 
reopening  of  competition  and  in  accordance  to  the  Financial  Rules  Applicable  to  the 
General Budget of the Union21 and its Rules of Application22. 

The evaluation Roadmap was adopted on 13 June 201623. 

According  to  the  Roadmap,  a  Steering  Committee  including  staff  from  DG  EAC  and 
from the Secretariat General was established in April 2016. The Steering Committee met 
in  four  occasions:  to  prepare  the  Terms  of  Reference  (ultimately  sent  to  contractors  in 
July  2016),  to  approve  the  Inception  Report  in  January  2017,  to  discuss  the  draft  final 
report  in  June  2017  and  to  approve  the  final  report  in  November  2017.  Extensive 
correspondence  between  the  Steering  Committee  members  was  held  in  between  the 
meetings to follow-up on the evaluation. 

The evaluation initial schedule foresaw a final report in the third quarter of 2017. Due to 
the  late  availability  of  data  used  for  the  evaluation  (and  gathered  by  the  cities)  it  was 
agreed to delay the submission of the final report to the fourth quarter of 2017. 

The evaluation did not need to be submitted to the Regulatory Scrutiny Board, as it is not 
considered  a  major  evaluation,  does  not  contain  an  impact  assessment  and  does  not 
constitute a fitness check for legislation. 

3.  EXCEPTION TO THE BETTER REGULATION GUIDELINES 

The only exception is that the evaluation does not include a wider public consultation. As 
explained  in  the  Roadmap,  the  ECOC  Action  is  considered  to  be  local.  International 
participation is scattered within and outside Europe and is difficult to reach. On the other 
hand the opinions would be based on attendance  to  specific events and would not  give 
useful insights for the evaluation of the ECOC Action as a whole. 

21   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R0966-20160101&from=EN 

22   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:02012R1268-20160101&from=EN 

23   http://ec.europa.eu/smart-regulation/roadmaps/docs/2016_eac_015_evaluation_ecoc_2016.pdf 

24 

 
 
                                                            
4.  EVIDENCE, SOURCES AND QUALITY 

The 2016 evaluation of the ECOC used a series of data sets to inform its findings. The 
main ones being: 

- 

Interviews  with  over  59  stakeholders  directly  and  indirectly  involved  in  the 
planning, operation and delivery of the two ECOC programmes; 

-  Two  surveys  of  projects  that  were  part  of  the  two  ECOC  programmes  who 

delivered the cultural programme attached to the intervention; 

-  A literature review of ECOC and European level information on the two ECOC 
including  application/  bid 
information,  EC  Committee  reports,  cultural 
programme  brochures,  web  sites  and  news  articles.  The  ECOC's  own  external 
evaluations have also been used to inform the European evaluation process. 

-  Together,  the  above  evidence  base  provides  the  evaluation  with  a  valid  and 
rounded  set  of  data  to  inform  the  views  on  the  main  aspects  of  the  ECOC 
evaluation  including  efficiency,  effectiveness,  sustainability  and  relevance.  This 
view has been informed by: 

-  The scale of the consultation exercise. Over 150 individuals have fed their views 
and  opinions  into  the  evaluation  process  either  through  face  to  face  interviews, 
telephone interviews or through the project level survey.; 

-  The nature of the consultation exercise. The evaluators were keen to consult with 
those  who  had  a  more  indirect  and  external  view  of  the  two  ECOC.  These 
stakeholders including journalists, those not directly benefitting from the ECOC 
(e.g.  rejected  projects)  as  well  as  those  working  in  the  wider  cultural  policy 
agenda at regional and city level. This ensures the evaluation is not simply based 
on those who benefitted the most from the ECOC. 

25 

 
 
ANNEX 2: STAKEHOLDER CONSULTATION 

The  stakeholders  were  consulted  via  targeted  consultations  (interviews  or  phone 
interviews)  and  via  an  on-line  survey,  mainly  on  projects  that  had  participated  in  the 
ECOC year or had submitted a proposal that was rejected. For Donostia San Sebastian, 
the  on-line  survey  of  projects  (survey  questions  below)  was  implemented  by  the 
contractor  supporting  the  evaluation,  while  for  Wroclaw  the  contractor  relied  on  the 
survey of projects that was implemented Impart, the ECOC implementing body itself and 
which questions aligned with the information sought.  

The consultations included the team responsible for the implementation of the ECOC, the 
political statements involved in the project, the projects participating in the programme or 
having submitted proposals to participate that were rejected, and personalities attached to 
the cultural tissue in both cities. The objective of the consultations was to have evidence 
supporting  the  findings  and  conclusions  of  the  evaluation.  The  questions  used  for  the 
survey of projects in Donostia San Sebastian can be found in the report produced by the 
experts assisting the Commission: 

-  https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/files/ecoc-
2016-evaluation-en_0.pdf 

For the reasons already mentioned in Chapter 4 of the Staff Working Document and in 
Annex 1, no open public consultation was conducted in the framework of the evaluation 
exercise. The consultation was restricted to relevant stakeholders in the two cities hosting 
the title. The characteristics of the Action and the scope of the evaluation do not make it 
necessary  to  extend  the  consultation  to  a  wider  public  (as  indicated  in  the  roadmap 
published). 

Annex  3  details  the  evaluation  questions  for  which  the  stakeholders'  consultation  was 
used.  These  regard  mainly  the  questions  which  answers  are  not  based  on  factual  data. 
The  stakeholders'  consultation  was  particularly  useful  to  find  information  about  the 
impact of the ECOC in the cultural offer of the city, the participation of citizens and local 
cultural  operators,  the  building  of  capacity  for  local  cultural  operators  and  legacy 
prospects.  

26 

 
 
 
ANNEX 3: METHODS AND ANALYTICAL MODELS 

The figure below presents the overview of the methodology. A more detailed overview 
of the methodology and sources used for each of the evaluation questions is presented in 
the subsequent tables and text. 

1. 

 Overview of methodology and tasks of the evaluation: 

27 

Inception PhaseAnalysis and Reporting PhaseTask One: Kick Off MeetingTask Two: Initial ConsultationsTask Three: First Visits to CitiesTask Eleven: Interim Report and MeetingTask Twelve: Factual Check of the City ReportsTask Thirteen: Final Report and MeetingTask Five: Inception Report and MeetingMain Research PhaseTask Six: Initial Bulletin of ResultsTask Eight: Second Visits to CitiesTask Nine: Additional InterviewsTask Ten: Information sharing and analysisTask Four: Desk ResearchTask Seven: Online Survey of Projects 
 
 
Table 1.1  Evaluation questions: Relevance, EU added value and coherence 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

EQ1: To what extent are the objectives of the 
ECOC Action? 

What  was  the  main  motivation  behind  the  city 
bidding  to  become  a  European  Capital  of 
Culture? 

the  process  of  determining 
What  was 
objectives? 
there  a  process  of 
  Was 
consultation  in  each  city  to  define  aims  and 
objectives? 

What  were  the  objectives  of  the  city  in  being 
ECOC?  What  was  the  relative  importance  of 
each objective? 

To  what  extent  were  the  objectives  consistent 
with  the  Decision  and  with  the  ECOC's  own 
application?  (special  focus  on  the  European 
dimension)  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Have any specific objectives of the ECOC event 
been related to social impacts? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

In  this  connection,  did  the  objectives  of  the 
ECOC  event  include  reaching  out  to  all  groups 
excluded, 
of 
disadvantaged, disabled people and minorities? 

including 

society, 

the 

EQ2:  To  what  extent  were  the  ECOC's 
associated 
cultural 
activities relevant to their own objectives? 

programmes 

and 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

To  what  extent  were  the  activities  consistent 
with the ECOC's own objectives? (special focus 
on the European dimension) 

Y 

Y 

Y 

Y 

Y 

extent 

specific 
To  what 
themes/orientations  of  the  cultural  programme 
proved to be relevant to the objectives defined? 

have 

the 

Y 

Y 

Y 

Y 

How  was  the  European  dimension  reflected  by 
the themes put forward by the ECOC event and 

Y 

Y 

Y 

Y 

Y 

28 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

in terms of cooperation at European level? How 
did  the  Capitals  of  Culture  seek  to  make  the 
European dimension visible? To what extent did 
the two ECOC cooperate? 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

did 

How 

EQ3: 
the  management 
arrangements  of  each  ECOC  contribute  to 
the  achievement  of  outputs,  results  and 
impacts? 

How  have  the  organizational  models  of  the 
formal  governing  Board  and  operational 
structures played a role in the European Capital 
of  Culture?  What  role  have  the  Board  and 
operational  structures  played  in  the  ECOC 
event's  implementation?  At  what  stage  were 
it 
these  structures  established?  How  did 
improve  management  of  culture  in  the  city 
during the event? 

Who  chaired  the  Board  and  what  was  his/her 
experience?  What  were  the  key  success  and 
failure elements related to the work of the Board 
and  operational  structure  used  and  personnel 
involved?  

Has  an  artistic  director  been  included  into  the 
operational  structure  and  how  was  he/she 
appointed?  What  were  the  key  success  and 
failure  elements  related  to  the  work  of  the 
artistic director and personnel involved? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  was 
programme? 

the  process  of  designing 

the 

Y 

How were activities selected and implemented? 

How  did  the  delivery  mechanism  contribute  to 
the achievement of outputs? 

Y 

to 

been 

strategy 

To  what  extent  has  the  communication  and 
successful 
promotion 
in/contributed 
the  promotion  of  city 
image/profile,  promotion  of  the  ECOC  event, 
awareness-raising  of  the  European  dimension, 
promotion  of  all  events  and  attractions  in  the 
city? 

29 

Y 

Y 

Y 

Y 

Y 

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

To  what  extent  has  the  communication  and 
promotion  strategy  including  the  use  of  social 
the 
media 
communication's 
local, 
regional,  national,  European  and  international 
levels? 

reached 
target  groups  at 

successfully 

Table 1.2  Evaluation questions: Efficiency 

EQ4:  To  what  extent  were  the  selection, 
monitoring and EU co-financing procedures, 
introduced  by  Decision  2006/1622/2006/EC 
efficient? 

the  Commission 

To  what  extent  have  the  mechanisms  applied 
by 
line  with  Decision 
in 
2006/1622/EC for the selection of the European 
the  subsequent 
Capitals  of  Culture  and 
implementation  and  monitoring  mechanisms 
influenced the results of the ECOC event?    

the 

informal  meeting 
To  what  extent  has 
following  the  designation  as  well  as  other 
advice  offered  by 
the 
Commission influenced the results of the ECOC 
event?  

the  panel  and  by 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

Y 

Y 

s
i
s
y
l
a
n
A

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

How was the Melina Mercouri Prize used? 

EQ5:  To  what  extent  did  the  ECOC  manage 
to raise the necessary resources?  

Y 

What was  the  process  of securing  the financial 
inputs? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  was  the  total  amount  of  resources  used 
for  each  ECOC  event?  What  was  the  final 
financial outturn of the year?  

Y 

Y 

Y 

What  were  the  sources  of  financing  and  the 
respective importance of their contribution to the 
total?  How  much  came  from  the  European 
Union Structural Funds  (e.g. ERDF  - European 
Regional  Development  Fund,  ESF  –  European 
Social Fund) or other sources of EU funding? 

Y 

Y 

Y 

Y 

Y 

Y 

30 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

To  what  extent  did  the  ECOC  title  trigger 
complementary sponsorship? 

What  was  the  total  expenditure  strictly  for  the 
implementation of the cultural programme of the 
year  (operational  expenditure)?  What  was  the 
proportion of the operational expenditure in the 
total expenditure for the ECOC event?  

Y 

Y 

Y 

Y 

What  proportion  of  expenditure  was  used  for 
tourism 
infrastructure 
infrastructure, including renovation)? 

(cultural 

and 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

EQ6:  To  what  extent  were  the  financial  and 
human  resources  secured  by  each  ECOC 
appropriate and proportionate? 

Was  the  total  size  of  the  budget  sufficient  for 
reaching  a  critical  mass  in  terms  of  impacts? 
Could  the  same  results  have  been  achieved 
with less funding? Could the same results have 
been achieved if the structure of resources and 
their respective importance was different?  

Y 

Y 

Y 

Y 

To  what  extent  have  the  human  resources 
deployed for preparation and implementation of 
the  ECOC  event  been  commensurate  with  its 
intended outputs and outcomes?  

Y 

Y 

be 

As a result, could the total budget for the ECOC 
event 
and 
proportional  to  what  the  each  ECOC  set  out  to 
achieve?  

appropriate 

considered 

Table 1.3 Evaluation questions: Effectiveness 

Evaluation Question 

Y 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

Y 

Y 

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

h
t
i

w
s
w
e
i
v
r
e
t
n

I

EQ6:  How  did  the  management  arrangements 
of each ECOC contribute to the achievement of 
outputs, results and impacts? 

Y 

Y 

Y 

Y 

Y 

Y 

31 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

How have the organizational models of the formal 
governing Board and operational structures played 
a  role  in  the  European  Capital  of  Culture?  What 
role  have  the  Board  and  operational  structures 
played  in  the  ECOC  event's  implementation?  At 
what stage were these structures established? 

Who  were 
the  key  external  partners  and 
stakeholders  in  the  project,  and  how  were  they 
involved in governance structures? 

Who  chaired  the  Board  and  what  was  his/her 
experience?  What  were  the  key  success  and 
failure  elements  related  to  the  work  of  the  Board 
and  operational  structure  used  and  personnel 
involved? 

Has  an  artistic  director  been  included  into  the 
operational  structure  and  how  was  he/she 
appointed? What were the key success and failure 
elements related to the work of the artistic director 
and personnel involved? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

What  was 
programme? 

the  process  of  designing 

the 

Y 

How were activities selected and implemented? 

Y 

What  cultural  activities 
development phase / before the title year? 

took  place 

in 

the 

Y 

EQ7:  To  what  extent  were 
objectives achieved? 

the  EU-level 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Provide  typology  of  outputs,  results  and  possible 
impacts of the Action at different levels (European, 
national, regional etc.) 

 To  what  extent  has  the  ECOC  event  been 
successful in attaining the objectives of the Action 
(refer to list in the intervention logic)? 

Was the cultural programme perceived as being of 
high artistic quality? To what extent did the ECOC 
prove  successful  in  bringing  their  chosen  artistic 
themes/orientations to the fore? 

32 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

To what extent did the ECOC title contribute to an 
enhanced  cultural  offer  in  the  cities  holding  the 
title  (e.g.  in  terms  of  scope  and  scale)  with 
stronger European Dimension? 

To  what  extent  did  the  ECOC  implementation 
widen access to and participation in culture in the 
two cities? What actions were taking to include the 
elderly,  young  people,  people  with  special  needs 
in the cultural activities? How accessible were the 
activities carried out? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

did 

programmes 

the  ECOC 

How 
help 
strengthening  the  capacity  of  the  cultural  and 
creative  sectors  and  its  links  with  other  sectors? 
Which  help  was  available  to  cultural  operators  to 
extend  their  networks  and  work  transnationally 
and internationally? 

To  what  extent  did  the  Action  in  the  two  cities 
raise their international profile through culture? 

EQ8:  To  what  extent  were  the  ECOC's  own 
objectives achieved? 

What  quantitative  indicators  (number  of  visitors, 
overnight  stays,  cultural  participation  of  people, 
etc.)  of  the  social,  tourist  and  broader  economic 
impacts  of  the  event  have  been  gathered  by  the 
ECOC?  

To what extent did the ECOC achieve the outputs 
hoped  for  by  the  city  and  as  set  out  in  the 
application?  

To what extent have specific objectives related to 
social impacts been met? 

To  what  extent  were  the  objectives  related  to 
reaching out to all groups of society, including the 
excluded, disadvantaged, disabled and minorities, 
met? 

EQ9: To what extent has the Action resulted in 
unintended effects? 

33 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

s
w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Are  there  any  instances  where  the  ECOC  event 
has  exceeded  initial  expectations?  What  positive 
effects has this had?  

Where  expectations  have  not  been  met,  what 
factors  have  hindered  the  development  of  the 
Action? 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Have  any  other  unintended  effects  been 
identified? 

Y 

Y 

Y 

Y 

Y 

Table 1.4 Evaluation Questions: Sustainability 

Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

EQ10: To what extent can the positive effects 
of  the  ECOC  Action  be  considered  to  be 
sustainable? 

Which  of  the  activities  or  elements  of  the  ECOC 
event  are  likely  to  continue  and  in  which  form 
once the ECOC-year is over? 

Has  any  provision  been  made  to  continue  and 
follow  up  the  cultural  programme  of  the  ECOC 
event after the closure?  

How will the city continue to manage its long-term 
cultural development following the ECOC event? 

What will be the role of the operational structure 
after the end of the ECOC event and how will the 
organizational structure change? 

What  has  been  the  contribution  of  the  ECOC 
event 
improved  management  of  cultural 
development in the city? (in the medium-term) 

to 

What are the impacts of the ECOC event likely to 
be  on  the  long  term  cultural  development  of  the 
city?  

34 

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y  

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation Question 

e
r
u
t
a
r
e
t
i
L

w
e
i
v
e
r

e
v
i
t
a
t
i
t
n
a
u
Q

s
i
s
y
l
a
n
a
a
t
a
D

h
t
i

w
s
w
e
i
v
r
e
t
n

I

s
m
a
e
t
y
r
e
v
i
l
e
d

l

r
e
d
o
h
e
k
a
t
S

s
n
o
i
t
a
t
l
u
s
n
o
c

f
o
y
e
v
r
u
S

s
t
c
e
j
o
r
p

t
c
e
j
o
r
P

s
w
e
i
v
r
e
t
n

i

s
i
s
y
l
a
n
A

What are the impacts of the ECOC event likely to 
be  on  the  long  term  social  development  of  the 
city? 

What are the impacts of the ECOC event likely to 
be on the long term urban and broader economic 
development of the city? 

EQ11:  What  is  the  EU  added  value  and  the 
visibility of the ECOC Action? 

As far as the conclusions made for the two cities 
allow,  what  is  the  added  value  of  the  European 
Capital  of  Culture  being  an  EU 
initiative, 
compared to what could be achieved if the Action 
was a purely national or local action? 

Was the fact that this is a European Union action 
sufficiently communicated by the cities?  

To  what  extent  were  the  general  public  and  the 
cultural operators aware of this fact? 

EQ12:  To  what  extent  were 
complementary to other EU initiatives? 

the  ECOC 

As far as the conclusions made for the two cities 
allows it, to what extent has the Action proved to 
be  complementary  to  other  EU  initiatives  in  the 
field of culture? 

To what extent has each ECOC been reinforced 
by and added impetus to investments by the EU 
Structural Funds? 

two  ECOC 
To  what  extent  have 
e.g. 
other  EU 
complemented 
European  Youth  Capital,  European  Green 
Capital? 

initiatives, 

the 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

2.  Details of the relevant evaluation tasks and methodologies:  

Inception phase 

The  aim  of  the  inception  phase  was  to  finalise  the  evaluation  framework  and  research 
tools, address the project steering group's comments regarding the proposed method and 
collect background information on Donostia San Sebastian and Wroclaw. 

35 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Initial consultations   

The task consisted in  identifying stakeholders to consult  with, targeting people that are 
knowledgeable  about  the  two  cities,  their  cultural  programmes  and  the  achievements, 
lessons  learnt  and  impact  of  the  ECOC,  including  national,  regional  and  local 
stakeholders. Discussion at the kick off meeting identified a need for strong triangulation 
and to gain opinions from a broader set of stakeholders. 

During the initial first visit to the cities described below, the contractors supporting the 
evaluation also sourced stakeholders for consultation who did not directly benefit or take 
part  in  the  ECOC  programme  and  who  they  felt  would  have  an  honest,  impartial  and 
unbiased view of how the programme was developed or implemented. While the aim was 
not to look for purely negative views, the contractors wanted to speak with stakeholder 
who, for example: 

  did not deliver an ECOC project or take part in the cultural programme attached 

to the ECOC; 

  were a member of the local and national press who had written either an honest or 
negative piece on the ECOC (as opposed to simply writing a press release which, 
for example, communicated the content and dates of the cultural programme); 
  had applied for ECOC funding (to be part of the ECOC cultural programme) and 

were rejected; 

  were  involved  in  ECOC  activity  but  were  often  vocal  about  a  particular  aspect 
(e.g. the quality of culture on offer, the lack of audiences, the lack of diversity of 
cultural content);  

  were  not directly involved in the cultural policy agenda in the cities but instead 

had an economic, social or environmental view on the ECOC. 

The list of stakeholders consulted is provided in the annex to the Report24. This list was 
partly  developed  by  the  ECOC  delivery  teams  but  also  through  an  internet  search  for 
stakeholders  who  work  in  the  above  fields.  The  list  was  further  expanded  during  the 
second visits themselves (through a snowballing technique - i.e. asking stakeholders for 
other stakeholders who could be useful "external" individuals to consult).  

With this in mind, in Donostia San Sebastian, the list of stakeholders consulted includes:  

  A  journalist  for  the  local  newspaper  who  had  written  independent  reviews  of 
various cultural projects as well as the overall delivery of the ECOC programme; 
  A  cultural  operator  that  applied  for  ECOC  funding  (to  be  part  of  the  ECOC 

cultural programme) but was rejected; 

  A cultural operator that was involved in the cultural scene of the city but did not 
directly  benefit  from  the  ECOC  programme  and  also  who  were  vocal  about  a 
particular aspect; and 

  Local stakeholders not directly involved in the cultural policy agenda in the cities 

but instead had an economic, social or environmental view on the ECOC. 

In Wroclaw, it included:  

24   https://ec.europa.eu/programmes/creative-europe/sites/creative-europe/files/files/ecoc-2016-

evaluation-en_0.pdf 

36 

 
 
                                                            
  An external evaluator from local university; 
  A former Minister of Culture and National Heritage; 
  An independent radio journalist; 
  A television journalist; and 
  The Head of Polish-German Chamber of Commerce. 

First visits to the cities  

Initial  face  to  face  visits  were  made,  to  Donostia  San  Sebastian  on  28  November  2016 
and Wroclaw on the 7 to 9 December 2016.  

Those consulted with during the initial visit to Donostia San Sebastian were: 

  Pablo Berástegui (Managing Director of the DSS2016); 
  Xabier Paya (Cultural director of DSS2016); 
  Garbiñe Muñoa (Financial Director of DSS2016); 
  Enara  García  DSS2016  (Director  of  participation  and  evaluation  of 

DSS2016); and 

  Fernando Alvarez (Communication Manager of DSS2016). 

Those consulted in Wroclaw were:  

  Dominika Kawalerowicz (Head of Programme Team, Wroclaw2016); 
  Katarzyna Mlynczak-Sachs (International Relations Manager, Wroclaw2016); 
  Olga Nowakowska (Chief Financial Officer, Wroclaw2016); 
  Jaroslaw Fret (Theatre Curator, Wroclaw2016 and Director of the Grotowski     

Institute); and 

  Katarzyna Kajdanek (Associate Professor, University of Wroclaw, evaluator). 

Desk research including big data feasibility 

This involved collection and review of literature, primary data from the delivery agency 
and available secondary data, as well as web and social media statistics. The list of the 
literature that was collected at the European level which was reviewed and assessed for 
usefulness  for  the  sake  of  the  evaluation  can  be  found  in  the  Report25.  Much  of  the 
European  level  literature  is  helpful  to  either  understand  the  policy  drivers  at  EU  level 
which support the two 2016 ECOC or provide background on the bidding and application 
stages  for  both  2016  cities  (i.e.  little  of  it  provides  information  useful  for  the  content, 
delivery and impact of the two programmes). More academic literature at the  EU level 
linked  to  the  ECOC  was  also  searched  for.  Instead  of  looking  at  academic  literature 
generally on 'European culture' and also literature before 2016, the contractor supporting 
the  evaluation  specifically  looked  at  up  to  date  (i.e.  produced  in  2016)  pieces  which 
linked to the ECOC Action. This search did not prove fruitful, although some academic 
literature at the city level was found. 

To complement the above desk research task the contractor assessed the extent to which 
each  ECOC  had  used  big  data  as  well  as  analysed  web  and  social  media  to  increase 
visibility and interest in the ECOC among country residents and internationally. 

25   See previous footnote.  

37 

 
                                                            
Online survey of projects 

The survey of projects proved to be an important source of complementary quantitative 
data for previous evaluations, and was done for 2016, supported by additional qualitative 
interviews  of  projects.  The  survey  was  performed  by  the  contractor  supporting  the 
evaluation  for  Donostia  San  Sebastian,  while  for  Wroclaw  they  relied  on  the  survey 
performed by the implementing body itself. 

Second visits to the cities 

With  a  clear  view  emerging  from  desk  research  and  project  survey  as  to  the  overall 
relevance, efficiency and effectiveness of each ECOC, the contractor undertook a second 
three day visit to each city. The visit focused on conducting in-depth interviews with the 
managing teams, their key stakeholders and cultural operators identified during the initial 
visits- please see the annex for more information.  

Building on the interviews undertaken in the first visits, the interviews with the delivery 
teams  served  four  purposes:  i)  gathering  further  factual  data  and  information  about  the 
cultural  programme  and  its  achievements;  ii)  identifying  the  "story"  of  the  ECOC 
throughout  its  lifecycle,  i.e.  conception,  application,  development,  delivery,  legacy;  iii) 
gaining a critical (albeit "insider") perspective on the relevance, efficiency, effectiveness 
and sustainability of the ECOC; iv) triangulating emerging findings, for example, those 
emerging from the desk research and project survey. 

Interviews  with  other  key  stakeholders  in  the  city  (e.g.  municipality,  chamber  of 
commerce,  commercial  sponsors,  media  representatives,  and  cultural  operators)  were 
also  essential  to  identifying  the  overall  "story"  of  the  ECOC  from  conception,  through 
application,  to  development  and  delivery.  These  individuals  offered  an  external, 
alternative and broader perspective on the ECOC. 

Additional project and stakeholder interviews 

These interviews sought to gather an in-depth understanding of the effect that ECOC had 
for the individuals involved, their organizations and the local culture sector as a whole. It 
also  allowed  to  ensure  that  findings  are  based  on  consensus  across  a  range  of  target 
groups as well as to identify key differences in the experience of different stakeholders. 
In  the  case  of  projects,  the  criteria  for  inclusion  focused  on  whether  it  represents  good 
practice in relation to at least one of the key dimensions of the evaluation, including (but 
not limited to) increased European cooperation, the effective targeting of key groups of 
citizens or neighbourhoods, audience development strategies, or legacy effects. 

Analysis and reporting phase 

Once  all  the  information  gathered,  the  experts  fulfilled  an  analysis  of  the  available 
information,  drawing  conclusions  and  triangulating  data  to  ensure  consistency  and 
accuracy. The limitations of the data gathered have been explained in the report and in 
the  Staff  Working  Document.  After  the  first  drafting  exercise,  the  results  were  shared 
with the two ECOC for a factual check. Both the Donostia San Sebastian and Wroclaw 
ECOC  were  asked  to  undertake  a  check  on  the  completeness  of  data  being  used  to 
address  each  evaluation  topic.  Their  comments,  where  relevant,  were  included  in  the 
Final Report submitted by the contractor supporting the evaluation. 

38