COMMISSION STAFF WORKING DOCUMENT EVALUATION of the Commission Recommendation of 27 October 2011 on the digitisation and online accessibility of cultural material and digital preservation
Date: 2021-01-29
Institutions: Directorate-General for Communications Networks, Content and Technology, European Commission
Subjects: copyright, cultural cooperation, cultural heritage, cultural transmission, digital archiving, digital technology, digitisation, heritage protection, virtual library

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 29.1.2021  SWD(2021) 15 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION of the Commission Recommendation of 27 October 2011 on the  digitisation and online accessibility of cultural material and digital preservation {SWD(2021) 16 final}  
 
 
Table of contents 

1. 

INTRODUCTION ................................................................................................................... 3 

2.  BACKGROUND OF THE INTERVENTION ........................................................................ 5 

3. 

IMPLEMENTATION / STATE OF PLAY .......................................................................... 10 

4.  METHOD .............................................................................................................................. 14 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS............................. 16 

6.  CONCLUSIONS ................................................................................................................... 47 

7.  ANNEXES……………………………………………… .............. …………….…………. 50 

1 

 
 
 
 
 
 
Glossary 

Term or acronym 

Meaning or definition 

AI /AR/VR 

Emerging technologies such as artificial intelligence/augmented 
reality/virtual reality  

CC 

CCI 

CHIs 

Creative commons, a set of copyright licenses developed and offered free of 
charge by the American non-profit organisation of the same name. 

Cultural and creative industries 

Cultural heritage institutions. This concept covers film institutes, audiovisual 
/ broadcasting archives, institutions for performing arts, national libraries, 
institutions for monument care, museums of natural history or natural 
science, museums of ethnography or anthropology, museums of science or 
technology, national archives, special or other types of libraries, higher 
education libraries, museums of art, public libraries, archive / records offices, 
museums of archaeology or history. 

Cultural material/cultural 
assets 

Print (books, journals and newspapers), photographs, museum objects, 
archival documents, sound and audiovisual material, monuments and 
archaeological sites 

DCHE 

Commission Expert Group on Digital Cultural Heritage and Europeana. The 
group  is  composed  of  representatives  of  the  Member  States  and  observers 
from  Member  States, 
and 
European/international organisations. A sub-group has been set up and has 12 
members. More details are provided by the Register of the Expert Groups. 

countries, 

candidate 

countries 

third 

EEA 

European Economic Area 

Europeana (or Europeana 
initiative) 

Europeana  is  an  initiative,  built  through  a  series  of  EU-funded  projects, 
that has created a platform for unique access to Europe’s digitised cultural 
heritage resources, developed common standards and solutions to achieve 
data interoperability and accessibility, and nurtured cross-border visibility 
and use of cultural resources.  

Europeana network (or 
ecosystem)  

The  network  of  cultural  heritage 
aggregators and professionals contributing to Europeana.  

institutions,  domain  and  national 

GLAM 

ISG 

Metadata 

PPP(s) 

Galleries, libraries, archives and museums 

Interservice Group 

Set of text data that describes the cultural objects. 

Public- private partnership(s) 

the Recommendation 

Commission Recommendation on the digitisation and online accessibility of 
cultural material and digital preservation (2011/711/EU)   

SWD 

Staff working document 

2 

 
 
 
1.  INTRODUCTION 

1.1. Purpose and scope 

In  an  age  of  rapid  technological  progress,  digital  technologies  bring  unprecedented 
opportunities to the cultural heritage sector. They provide more effective tools to digitise 
cultural  heritage  assets,  for  diverse  purposes  such  as  preservation,  conservation, 
restoration,  reproduction,  research,  education,  as  well  as  for  a  broader,  more  democratic 
online access and reuse in key ecosystems such as sustainable tourism and the cultural and 
creative  industries.  They  provide  the  public  with  numerous  ways  to  access,  discover, 
explore and enjoy  cultural  material  while cultural  heritage institutions can reach broader 
audiences,  and  engage  them  in  innovative  ways  by  offering  immersive,  creative  and 
accessible content.  

Cultural  heritage  is  an  important  contributor  to  the  European  economy,  fostering 
innovation,  creativity  and  economic  growth.  For  example,  cultural  tourism  accounts  for 
40% of all tourism in Europe1, and cultural heritage is an essential part of cultural tourism. 
There  are  places,  regions  and  even  some  countries  where  the  local  economy  relies 
extensively  or  even  completely  on  the  revenue  that  cultural  heritage  sites  and  venues 
generate by attracting visitors to those areas.  

Digitisation of cultural heritage and the reuse of such content can  generate new jobs not 
only  in  the  cultural  heritage  sector,  but  also  in  other  key  areas  such  as  the  creative 
industries  (e.g.  the  video  game  industry,  film  industry).  Such  processes  also  provide 
incentives  for  companies  to  develop  more  efficient  technologies,  and  new  business 
models.  

However,  cultural  heritage,  including  monuments,  archaeological  landscapes,  buildings 
and  artefacts,  continues  to  face  significant  threats  of  deterioration  or  partial  or  full 
destruction  or  disappearance.  Some  threats  are  human-derived,  e.g.  mass  tourism, 
terrorism  and  vandalism,  while  others  originate  in  nature,  e.g.  deterioration  over  time, 
pollution, flooding, wind erosion. There have been regular, tragic reminders (e.g. fires at 
the  Nantes  Cathedral  in  2020,  at  the  Notre  Dame  in  Paris  in  2019,  at  the  Cavallerizza 
Reale in Turin in 2019, the theft from the Green Vault at Dresden’s Royal Palace in 2019) 
of the need to leverage digital technologies to avoid the complete loss of the memory of 
our cultural heritage assets and to enable their reconstruction and reproduction.  

The unprecedented challenges of the COVID-19 pandemic, with its significant social and 
economic consequences, have heightened even more the need for and the urgency of using 
digital  technologies.  Many  cultural  heritage  institutions  (‘CHIs’)  had  to  close  and  thus 
incurred major financial loss. However, despite the loss of income, reaching 75-80%2 for 

1 https://ec.europa.eu/growth/sectors/tourism/offer/cultural_en 
2See report by Network of European Museums Organisation (NEMO)  

3 

 
 
                                                           
some  CHIs,  many  of  them  have  managed  to  bounce  back3  and  keep  or  extend  their 
audiences.  There  has  been  an  increase  in  the  digital  services  they  offer4  (e.g.  engaging 
with audiences, sharing collections, offering digital tools), demonstrating once more their 
high value to society. 

The Commission Recommendation on the digitisation and online accessibility of cultural 
material  and  digital  preservation  (2011/711/EU)5  (“the  Recommendation”),  is  the 
Commission’s main policy tool for digital cultural heritage. Its aim is to optimise the use 
of information and communication technologies to unlock the full economic and cultural 
potential of Europe’s cultural heritage.  

The  Recommendation  dates  back  to  October  2011.  Some  of  the  challenges  facing  the 
cultural heritage sector at that time, i.e. an urgent need to protect and preserve European 
cultural  heritage  at  risk  are  still  present  today,  but  others  have  arisen  since.  Major 
advances in digital technologies for example have created new opportunities that did not 
exist  when  the  Recommendation  was  adopted.  Therefore,  the  Recommendation  needs  to 
be assessed in order to decide whether or not it still meets the needs and expectations of 
the cultural heritage sector, of the users, of the creative sectors, of society in general.  In 
this  context  of  major  digital  developments,  a  clear,  evidence-based  assessment  of  the 
Recommendation  will  help  the  Commission  to  decide  on  the  follow-up  steps,  such  as  a 
revision of the Recommendation. 

In  line  with  the  Better  Regulation  Guidelines6,  this  evaluation  assesses  how  the 
Recommendation  has  been  working  and  actually  performing  compared  to  initial 
expectations  by  examining  its  effectiveness,  efficiency,  relevance,  coherence  and  EU 
added value.  

This  staff  working  document  (‘SWD’)  addresses  the  five  areas  in  the  Recommendation, 
i.e. organisation and funding of digitisation; digitisation and online accessibility of public 
domain  material;  digitisation  and  accessibility  of  in-copyright  material;  Europeana; 
digital preservation, taking into account the digital lifecycle approach to cultural material. 

This  evaluation  is  supported  by  an  externally  contracted  study7  (’evaluation  study’),  by 
various  meetings  and  workshops  with  Member  States,  stakeholders  and  experts  in  the 
field, and by targeted and online public consultations. 

3 According to the report prepared by NEMO (see footnote 2), 4 out of 5 museums have increased their 
digital services to reach their audiences. 
4   See also the Art Fund survey on COVID-19 Impact: Museum sector research report on the challenges 
facing museums and galleries, which highlights that the vast majority (86%) of organisations have increased 
their online presence. 
5 OJ L 283, 29.10.2011, p. 39–45 
6 https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-
how/better-regulation-guidelines-and-toolbox_en 
7 Study to support the evaluation and possible revision of the Commission Recommendation (2011/711/EU) 
on the digitisation and online accessibility of cultural material and digital preservation, 
SMART 2019/0027. 

4 

 
                                                           
The evaluation covers the whole period since the Recommendation’s adoption in 2011 to 
date.  It  covers  the  Member  States,  the  European  Economic  Area  countries  (‘EEA’), 
Norway, Iceland, Lichtenstein, as well as the UK.  

2.  BACKGROUND OF THE INTERVENTION 

2.1. Description of the intervention and its objectives 

The legal basis for the Recommendation is Article 292 of the Treaty on the Functioning of 
the  European  Union,  which  lays  down  powers  for  the  European  Commission  to  adopt 
recommendations in specific cases provided for by the Treaties. 

At the time of its adoption, the Recommendation was an important action for one of the 
key  areas  tackled  by  the  Digital  Agenda  for  Europe,  a  flagship  initiative  in  the  Europe 
2020  strategy8  of  March  2010.  It  built  on  the  work  done  through  the  digital  libraries 
initiative, with a view to making Europe’s cultural and scientific heritage accessible to all 
online. The launch of Europeana in 2008, Europe’s digital platform for cultural heritage, 
was one of the most important stepping stones for digital cultural heritage.  

The  Recommendation  built  on  the  2006  Recommendation  on  the  digitisation  and  online 
accessibility of cultural material and digital preservation (2006/585/EC)9, which had been 
an  important  means  of  supporting  and  encouraging  the  work  of  the  Member  States  and 
CHIs in the field. 

The  Recommendation  reflected  the  necessary  changes  in  the  policy  context  and  in  the 
cultural  heritage  sector  since  2006,  such  as  the  establishment  of  Europeana,  the 
recommendations  stemming from  'the New Renaissance' report, prepared in  2011 by the 
Comité  des  Sages10,  and  the  European  Commission's  proposal  for  an  Orphan  Works 
Directive in 2011. 

the  Member  States‘ 

Moreover, 
the  2006 
reports  on 
Recommendation for the reporting cycle covering 2008 to 2010 showed progress, but this 
was inconsistent and uneven for the different provisions of the 2006 Recommendation.  

implementation  of 

the 

A modern and more effective set of measures for digitising and bringing cultural heritage 
online was needed to reflect the emerging new challenges and opportunities for the sector 
at  that  time.  In  particular,  the  Recommendation  sought  to  address  Europe’s  untapped 
economic  and  cultural  potential.  Increasing  digitisation  and  developing  and  applying 
advanced technologies to the sector simulates growth and job  creation, as the process of 
digitisation  is  labour  intensive  and  generates  new  jobs.  Digitised  cultural  heritage 
resources are long-term economic assets which, through the data attached to them, create 

8  Communication  from  the  Commission  “Europe  2020  -A  strategy  for  smart,  sustainable  and  inclusive 
growth”, COM(2010) 2020 final, 3.3.2010. 
9 OJ L 236, 31.8.2006, p. 28–30, OJ L 118M , 8.5.2007, p. 1279 
10 See “The new renaissance -Report of the Comité des Sages on bringing Europe's cultural heritage online”, 
E. Niggemann, J. De Decker, M.Lévy, 2011 

5 

 
 
                                                           
value  and  contribute  to  economic  growth,  with  spillover  effects  into  tourism,  education, 
research, etc. 

The  Recommendation  sought  to  address  the  need  to  better  harmonise  national 
approaches  across  Member  States  to  the  digitisation  and  preservation  of  cultural 
heritage. Considering the resources required for  digitisation, a concerted  action from  the 
Member  States  was  necessary  to  ensure  more  coherence  in  the  selection  of  cultural 
material, better optimisation in the use of digitisation capacity and achieving economies of 
scale, as well as a more secure and stimulating climate for companies willing to invest in 
digitisation  technologies.  Stimulating  innovative  ways  to  make  cultural  and  creative 
content  available  across  the  Member  States  results  in  more  possibilities  for  users  to 
explore and enjoy European cultural heritage treasures.  

The Recommendation sought to address the need for increased digitisation, online access 
and  preservation  to  avoid  losing  our  cultural  heritage  assets  and  their  memory.  The 
increasing  natural  or  human  derived  threats  to  historical  buildings,  museums  and  other 
cultural  heritage  assets  also  made  it  clear  that,  without  digitisation  and  preservation  of 
cultural heritage treasures, society would run the risk of losing its very ‘memory’ (digital 
and physical) and knowledge in the event of destruction or disappearance of those assets. 

The  ultimate  impacts  expected  from  the  Recommendation  were  a  better  protection  of 
Europe's cultural  heritage, higher public awareness  and appreciation  of Europe's cultural 
heritage, increased availability of open data and exchange of knowledge.  

The intervention logic of the EU action, in  the  form  of  a diagram  summarising  how the 
EU  action  was  originally  expected  to  work  at  the  time  of  adoption/implementation,  is 
provided below:  

6 

 
The Recommendation had the following main overall objectives:  

  optimise the cultural and economic potential of Europe’s cultural heritage 
  ensure Europe’s position as a leading international player in the field of culture 

and creative content; and 

  unlock  investments  from  Member  States  and  the  private  sector  for  digital 

cultural heritage initiatives  

The main measures promoted in the Recommendation are: 

1. 
Digitisation-organisation  and  funding:  Member  States  are  recommended  to 
further develop their planning and monitoring of cultural material (targets and overviews), 
to encourage partnerships between cultural institutions and private sector, make use  of the 
EU’s Structural Funds to co-finance digitisation activities, consider ways to optimise the 
use of digitisation capacity and achieve economies of scale; 
2. 
Digitisation  and  online  accessibility  of  public  domain  material:  Member  States 
are  recommended  to  improve  the  access  to  and  use  of  digitised  cultural  material  in  the 
public domain; 
3. 
for the digitisation and online accessibility of in-copyright material; 
4. 
 Europeana: contribute to its further development; 
5. 
Digital preservation: reinforce national strategies for the long-term preservation of 
digital material, update action plans implementing the strategies and exchange information 
with each other on the strategies and action plans; provide in their legislation for multiple 
copying  and  migration  of  digital  cultural  material  for  preservation  purposes;  make  the 
necessary arrangements for the deposit of the material created in digital format. 

 Digitisation and online accessibility of in-copyright material: improve conditions 

7 

 
 
Point  12  of  the  Recommendation  encourages  Member  States  to  inform  the  Commission 
every  2  years  of  their  implementation  actions.  The  Commission  monitors  the  progress 
made  by  the  Member  States  in  implementing  the  Recommendation  with  the  help  of  the 
Expert Group on Digital Cultural Heritage and Europeana. 

2.2. Baseline and points of comparison  

2.2.1. Digitisation: organisation and funding 

According  to  the  evaluation  study,  when  the  Recommendation  was  adopted,  the  state  of 
digitisation varied significantly depending on the type of institution and cultural material. 
While most CHIs had a digital collection or were involved in some digitisation activities 
(83%),  only  an  estimated  20  %  of  all  collections  that  needed  to  be  digitised  had  in  fact 
been  digitised.  Moreover,  only  one  third  of  the  institutions  were  part  of  a  national 
digitisation strategy and only 34 % had a written digitisation strategy. Art museums were 
the  most  digitised  (at  42%),  while  national  libraries  had  digitised  only  4%  of  their 
collections.  Photographs  were  the  most  commonly  digitised  type  of  cultural  material, 
followed by archival material.  

Moreover,  the  Commission’s  Second  progress  report  on  the  digitisation  and  online 
accessibility of cultural material and on digital preservation in the European Union from 
201011,  noted  that,  despite  the  important  progress  brought  about  by  the  2006 
Recommendation, the work carried out at the time to digitise cultural heritage content was 
still in its early stages. In particular, the report indicated that the vast majority of cultural 
objects worth digitising had not yet been digitised (e.g. for national libraries, only 3.5 % of 
relevant  cultural  objects  had  been  digitised).  While  the  national  reports  indicated  that 
inventories,  overviews  and  databases  were  increasingly  being  set  up,  to  ensure  a  cross-
sectoral  and  national  dimension,  several  Member  States  had  still  not  developed  any 
national  overviews  of  digitised  collections  The  Commission’s  report  pointed  to  the  fact 
that progress in digitisation activities depended on the long-term financing available, but 
very often a national budget had not been assigned to it and it was mainly the CHIs’ own 
resources  that  were  financing  digitisation  (followed  by  project  based  financing  with 
national  and  EU  funds).  Likewise,  according  to  the  report,  some  countries  provided 
information  about  the  sum  of  public  budgets  aggregated  at  national  level  specifically 
devoted to the digitisation of cultural heritage on an annual basis (e.g. France, Spain and 
Finland) or multiannual basis (e.g. Czechia and Belgium ). 

Furthermore,  the  evaluation  study12  points  to  the  limited  adoption  of  policies  on  digital 
access by the institutions (around 31 % of them). Only 42 % of institutions reported that 
they were monitoring the usage of their digital collections through consultations or social 
media  monitoring  (libraries  between  56  %  and  78  %,  museums  around  27  %).  The 
monitoring of use of their digital  collections through  web statistics was higher (~85 %). 

11  European  Commission  (2010).  Second  progress  report  on  the  digitisation  and  online  accessibility  of 
cultural material and on digital preservation in the European Union  
12 Based on ENUMERATE. Digitisation in Cultural Heritage Institutions, Survey Report (2012).  

8 

 
 
 
                                                           
According  to  the  Commission’s  second  progress  report  of  2010,  mentioned  above,  17 
Member States had national coordination mechanisms for digitisation activities. As far as 
the  targets  were  concerned,  although  there  were  quantitative  targets  in  most  of  the 
Member States, they were usually set ‘bottom-up’ by the CHIs themselves and there was 
no comprehensive aggregated planning document at national level.  

With regard to public private partnerships/PPPs (i.e. between CHIs and the private sector) 
although  theoretically  they  could  be  a  useful  instrument  to  pool  new  resources  for 
digitising cultural  material,  the national  reports indicate they were not  a widespread tool 
for financing digitisation.  

Large-scale  PPPs  with  both  public  and  private  partners  (such  as  Google)  had  less 
difficulties 
involving  medium  and  small-scale  cultural 
organisations. 

in  emerging 

than  PPPs 

2.2.2.  Digitisation  and  online  accessibility  of  public  domain  material  and  of  in-copy 
right material 

According  to  the  Commission’s  second  progress  report  of  2010,  only  a  minority  of 
Member  States  (4)  had  mechanisms  making  it  easier  for  cultural  institutions  to  digitise 
orphan works and make them accessible online. Likewise, only a minority of countries (8) 
declared  having  mechanisms  for  out-of-print  (out-of-commerce)  works.  Moreover,  only 
three countries indicated having lists of orphan works and works in the public domain.  

However,  several  of  the  countries  took  part  in  the  EU-funded  ARROW  project 
‘Accessible  Registries  of  Rights  Information  and  Orphan  Works  towards  Europeana’, 
involving national libraries, publishers and collective management organisations. 

2.2.3. Europeana 

The  Commission’s second progress report of 2010 pointed to a steady growth over time 
of access to digitised content through Europeana, reaching and significantly exceeding the 
original target  of 10 million  items  by 2010 (i.e. more than 14 million  items,  including 5 
million text items, were accessible through Europeana). However, despite an improvement 
in the overall balance, many Member States were not contributing a sufficient amount of 
content (i.e. about half of the Member States contributed in total less than 1 % of the total 
number of objects).  

2.2.4. Digital preservation 

The  Commission’s  second  progress  report  of  2010  highlighted  that  the  preservation  of 
digital information remained a significant challenge, despite a growing acknowledgement 
by  Member  States  of  the  costs  linked  to  losing  digital  data.  Almost  half  of  them  had 
adopted  a  strategy  for  digital  preservation13.    However,  the  diversity  of  the  national 
strategies was ‘ranging from a preliminary feasibility study to a much more advanced and 

13 According to the evaluation study 23 % of institutions had a written digital preservation strategy, while 
around one-third were included in a national preservation strategy 

9 

 
                                                           
comprehensive  digital  preservation 
infrastructure  with  processes  already  up  and 
running’14. In addition, according to 'the New Renaissance' report of the Comité des Sages, 
referred to above, while mechanisms and plans had been put in place by several Member 
States,  there  was  still  a  lot  of  work  to  be  done  to  put  all  the  strategies  and  plans  into 
practice. This was the case not only in the Member States that were less advanced in this 
sector (and running the risk of permanent loss of digital content), but also in the Member 
States that were more advanced in this area. 

According  to  the  Commission’s  2010  progress  report,  22  countries  reported  allowing 
multiple  copying  and  migration  of  digital  cultural  material  by  public  institutions  for 
preservation purposes.  

On policies and procedures for the deposit of material originally created in digital format, 
most  Member  States  reported  legal  deposit  policies  in  place  for  born-digital  material, 
although some also reported actual challenges with implementation.  

 2.2.5. General aspects  

From  a  more  general  perspective,  whereas  the  evaluation  study  points  to  considerable 
room  for  improving  the  supply  of  digitised  cultural  material,  on  the  demand  side  it 
acknowledges  that  there  were  few  relevant  and  specific  statistics  on  the  use  of  digitised 
cultural material at that time.  
Overall,  the  baseline  shows  that,  at  the  time,  despite  the  efforts  and  the  progress  of  the 
Member  States  and  CHIs  to  digitise,  preserve  and  provide  online  access  to  digitised 
cultural heritage content, further steps were needed15.  

3.  IMPLEMENTATION / STATE OF PLAY 

3.1. Description of the current situation  

The  Recommendation  represented  a  milestone  in  digital  cultural  policy,  encouraging 
Member States to step up efforts and support CHIs with the digitisation, online access and 
preservation of cultural heritage.  

3.1.1. Digitisation: organisation and funding 

As  mentioned  in  Section  2,  the  Recommendation  encourages  Member  States  to  further 
develop their planning and monitoring of cultural material (targets and overviews), foster 
partnerships  between  cultural  institutions  and  the  private  sector,  make  use  of  the  EU’s 
Structural  Funds  to  co-finance  digitisation  activities,  optimise  the  use  of  digitisation 
capacity and achieve economies of scale. 

According to the information provided by Member States in their implementation reports 
for the reporting period 2015-2017 and consolidated in the Commission progress report of 

14 See Commission’s second progress report (2010) 
15 See also the Council conclusions of 10 May 2012 on the digitisation and online accessibility of cultural 
material and digital preservation (2012/C 169/02), OJ C 169, 15.6.2012, p. 1–4 

10 

 
                                                           
201816, 23 of them have national digitisation plans and/or national funding programmes, as 
summarised in the table below. 

Figure 1: Predominant trends in the organisation of digitisation across the EU 

National 
institutions/ 
Domain specific 
strategies 
√ 
√ 

National 
strategies 

National 
funding 
programmes 

Regional 
strategies 

Regional 
funding 
programmes 

Advisory task 
forces, 
committees & 
work groups 

√ 

√ 

√ 

√ 

√ 

√ 

√ 

√ 

√ 

√ 
√ 

√ 
√ 

√ 
√ 
√ 

√ 
√ 
√ 

√ 
√ 
√ 

√ 
√ 
√ 
√ 
√ 

√ 
√ 
√ 
√ 
√ 
√ 
√ 
√ 
√ 
√ 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 
UK 
Source: Consolidated progress report on the implementation of Commission Recommendation, 
2015-2017 

data not available for the reporting period  

√ 
√ 
√ 
√ 
√ 

√ 
√ 
√ 

√ 
√ 
√ 

√ 
√ 
√ 

√ 
√ 
√ 

√ 
√ 

√ 
√ 

√ 
√ 

√ 

√ 

√ 

√ 

√ 

√ 

The  digitisation  schemes  have  targets  that  vary  considerably  according  to  the  Member 
State and that are usually set by each domain of cultural heritage (e.g. libraries, archives, 
museums, etc.). Digitisation of library and archival cultural  resources is one of the main 
digitisation priorities in more than half of the Member States. In addition, for more than 
one 
(e.g.  monuments,  historical  buildings, 
archaeological  sites)  was  a  priority  in  the  reporting  period  and  these  Member  States 
allocated  funding  programmes  to  it.  At  least  six  Member  States  reported  digitisation 
programmes for intangible culture (e.g. living arts, traditional folklore culture), while 3D 
appears  to  be  gaining  importance  across  the  Union.  However,  as  recalled  also  by 

immovable  cultural  heritage 

third, 

16 
  European  Commission  (2018),  Cultural  heritage:  digitisation,  online  accessibility  and  digital 
preservation. Consolidated progress report on the implementation of Commission Recommendation, 2015-
2017 

11 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                           
contributions  to  the online consultation,  only  22% of the heritage collections  have been, 
on average, digitally reproduced17. 

Discussions in the workshop on digital transformation in the cultural heritage sector, held 
in Lisbon in November 2019, pointed to the fact that the sector struggles to set priorities 
for digitisation and preservation and needs a clearer expression of the expectations and set 
benchmarks.  

Moreover, according to the national reports, public-private partnerships followed a slow, 
upward  trend,  with  mainly  technology  companies  (e.g.  Google)  being  interested  in  such 
schemes,  followed  by  media  publishers  (e.g.  Proquest)  and  banks.  Some  Member  States 
(Czechia,  Germany,  the  Netherlands,  Austria,  Italy)  reported  PPPs  between  national  or 
state libraries and Google: in Czechia around 150 000 books had been digitised and made 
available  since  2011  (when  the  PPP  was  established),  while  in  Germany  the  PPP  with 
Google led to the digitisation of around two million books. Wikimedia, international non-
profit organisations and private individuals also concluded agreements with CHIs. 

Two  thirds  of  the  Member  States  reported  making  use  of  the  EU  Structural  Funds  for 
digitising of cultural material and related services (Bulgaria, Czechia, Germany, Estonia, 
Greece,  Spain,  Croatia,  Italy,  Cyprus,  Latvia,  Lithuania,  Malta,  Poland,  Romania, 
Slovenia, Slovakia, Finland and Sweden). 

There  are  various  ways  to  pool  digitisation  efforts  to  optimise  the  use  of  digitisation 
capacity and achieve economies of scale, ranging from competence centres and national 
or cross-border collaborations to  sharing services and facilities. Eighteen Member States 
(Belgium,  Bulgaria,  Croatia,  Cyprus,  Czechia,  Estonia,  Finland,  France,  Germany, 
Hungary,  Ireland,  Latvia,  Lithuania,  the  Netherlands,  Poland,  Slovakia,  Spain,  Sweden) 
reported competence centres, aiming to  centralise funds and lower costs,  consolidate the 
digitisation  process,  share  best  practices,  etc.  Moreover,  two  thirds  of  Member  States 
reported sharing services and facilities (e.g. repositories, content management systems or 
IT  tools)  to  store  and  manage  digital  objects  and  data  more  efficiently  and  make  better 
quality  content  available  online  (Austria,  Croatia,  Denmark,  Estonia,  Finland,  France, 
Greece,  Italy,  Latvia,  Lithuania,  Luxembourg,  Malta,  the  Netherlands,  Poland,  Slovenia, 
Slovakia, Spain and Sweden). 

A report on digitisation and  IPR in  European museums,18 published in July  2020, shows 
that 43.6% of museum collections are digitised, although with different levels per museum 

17 ENUMERATE 2017  
18  Final  report  on  Digitisation  and  IPR  in  European  Museums,  published  by  the  Network  of  European 
Museum  Organisations  in  July  2020.  The  survey  included  3  different  target  groups;  national  museums 
organisations, national ministries in charge of museums and museums, in Austria, Belgium, Croatia, Estonia, 
Finland, France, Germany, Greece, Italy, the Netherlands, Poland, Slovenia and Spain.  In total 60 museums 
in Europe, 13 Member States, plus Serbia and the UK participated in the survey. 

12 

 
                                                           
category19. According to the participating museums, less than 20% of their collections are 
available online. 

3.1.2 Public domain material and in-copyright material 

The Recommendation encourages digitisation and online access to public domain and in-
copyright material. The national reports pointed to a positive trend among Member States 
to  ensure  that  public  domain  status  is  maintained  after  digitisation;  most  of  them 
reported supporting actions ensuring wider access or use of the digitised cultural heritage 
material in the public domain.  

Based  on  the  information  provided  by  the  Member  States  in  their  national  reports,  the 
measures  laid  down  in  accordance  with  the  Orphan  Works  Directive20  do  not  appear  to 
have contributed to large-scale digitisation of orphan works by CHIs. For the digitisation 
of  out-of-commerce  works,  although  a  significant  number  of  Member  States  reported 
licensing  mechanisms  supporting  it,  their  practical  application  has  been  limited,  due  for 
instance to the fact that only some types of works (e.g. literary works) are concerned. 

3.1.3. Development of Europeana 

The  Recommendation  set  the  indicative  target  of  having  30  million  digital  objects 
available on Europeana by 2015. 

The quantitative targets for Europeana were already reached in 201321 and surpassed, (50 
033 909 in  July 2020),  with  images (27 945 338) and text  (20 819 920) being the main 
categories  of  content22,  compared  with  a  marginal  contribution  of  3D  for  instance  (16 
662). Although the focus of the CHIs appears to be mainly the quantity, quality emerged 
as a need as well. More than a third of the Member States encourage their CHIs to submit 
high quality content and metadata to Europeana and 63.08% of the content is in tier 2 and 
above, as set by Europeana23. 

A  minority  of  Member  States  (Austria,  Czechia,  Germany,  Greece,  Estonia,  Poland, 
Slovakia and Spain) reported making public funding for digitisation projects conditional 
upon making the digitised material available through Europeana. 

Twenty Member States reported contributing to Europeana through national aggregators 
and through cross-border thematic and domain aggregators. Member States highlighted 
various challenges to aggregation, such as the need to strengthen the coordination capacity 
of  national  aggregators  and  to  address  the  digital  capacity  of  CHIs  and  more  generally 

19 Art and design museums have an average of 65% of their collections digitised, natural history museums 
15%, history and archaeology museums 27%. One of the reasons for such differences could be the existence 
of complex 3D objects that need the appropriate technology and financial resources. 
20  Directive  2012/28/EU  of  the  European  Parliament  and  of  the  Council  of  25  October  2012  on  certain 
permitted uses of orphan works, OJ L 299, 27.10.2012, p. 5–12  
21  Europeana  Foundation  (2013).  Europeana  celebrates  5  years  and  30  million  objects.  Available  at: 
https://pro.europeana.eu/post/europeana-celebrates-5-years-and-30-million-objects. 
22 Sound: 805,897 and video: 446,092. 
23 Quality standards and how the content can be used are outlined in the Europeana Publishing Framework 
(https://pro.europeana.eu/post/publishing-framework). The framework has four tiers of criteria on the quality 
of the content. Three tiers of criteria on the quality of metadata were added in October 2019. 

13 

 
                                                           
their digital transformation: for example by tackling difficulties with data submission and 
the lack of expertise in copyright legislation. 

Around  half  of  the  Member  States  reported  a  wide  and  free  availability  of  metadata  of 
digitised  cultural  content.  The  reports  also  pointed  to  the  need  to  further  consider 
emerging technologies that can improve the quality of the digitised material.  

Only a limited number of Member States (Austria, Cyprus, Greece, Luxembourg, Poland, 
Portugal  and  Spain)  reported  raising  public  awareness  of  Europeana  through  national 
communication actions or plans. 

3.1.4. Digital preservation. 

The  national  reports  point  to  a  mixed  coordination  approach  at  different  levels  to  long-
term preservation strategies and action plans, with most of the Member States reporting 
mainly strategies at the national level, specific activities by public institutions or bodies, 
and participation in European projects. 

Almost  half  of  the  Member  States  and  Lichtenstein  reported  explicit  provisions,  in  their 
national  copyright  laws  or  laws  on  archiving,  which  allow  multiple  copying  and 
migration of digital cultural material by public institutions for preservation purposes. This 
reporting includes amendments or new developments at the national level (i.e. in Czechia, 
Estonia, Germany, and Ireland). 

Most  of  the  Member  States  reported  having  in  place  measures  ensuring  that  rights 
holders deliver works to legal deposit libraries and allowing web-harvesting techniques, 
while  half  of  them  have  adopted  legal  provisions  allowing  the  transfer  of  digital  legal 
deposit works among legal deposit libraries. 

4.  METHOD 

4.1. Short description of methodology:  

The findings and conclusions of this report are based on a number of sources, including an 
evaluation study conducted by an external contractor, several consultation activities, desk 
research, various documents and exchanges with Member States and stakeholders.  

The following main methodological steps were followed: 

1.  Inception phase:  

This  process  implied  organising  the  process,  setting  its  objectives  and  the  timeline  and 
selecting the tools and the deliverables. In other words, the first phase focused on starting 
and agreeing on the conditions for the study (i.e. on the objectives and timing), carrying 
out an initial desk review and developing data collection tools. This step mainly involved 
the contractor and the Commission (which included the Interservice Group (ISG). 

2.  Data collection:  

14 

 
This  process  implied  establishing  the  baseline  of  the  analysis,  carrying  out  national 
research, desk research and stakeholder consultations (including targeted and online public 
consultations) and organising workshops. This involved the contractor, various parts of the 
Commission,  Member  States’  competent  authorities,  cultural  heritage  institutions, 
international  and  umbrella  organisations,  the  Europeana  ecosystem  of  organisations  and 
professionals,  technology  stakeholders  with  research,  products  and  services  in  ICT  for 
cultural heritage and the public. 

The  publication  of  a  roadmap  defining  the  scope  of  the  evaluation  gathered  eight 
responses from various respondents.  

The  evaluation  study  was  conducted  by  the  contractor  between  August  2019  and  March 
2020 and updated in November 2020, following the online public consultation. The study 
was one of the main sources of information for this staff working document. 

A  consultation  strategy  was  created  and  agreed  with  the  ISG,  with  the  following 
objectives:  (1)  to  collect  views  on  the  Recommendation’s  implementation  and  (2)  to 
collect views on the impacts of possible future changes to the Recommendation. 

As  part  of  the  consultation  strategy,  the  Commission  held  five  workshops  with 
representatives of Member States, stakeholders,  international  organisations and networks 
in order to collect relevant and solid information on the Recommendation’s impact and on 
the  challenges  and  needs  confronting  the  sector,  that  have  arisen  from  the  digital 
transformation and that have to be addressed in the future.  

Between November 2019 and January 2020, the Commission held a targeted consultation 
of  Member  States’  authorities  with  questions  covering  all  five  of  the  evaluation  criteria. 
The Commission received 20 responses that fed into the evaluation study. 

Between  22  June  and  14  September  2020,  the  Commission  held  a  twelve  week  open 
public consultation, focusing on the importance of digitisation and digital transformation 
in the field and how such processes can be supported, as well as on the Recommendation’s 
impact  on  the  cultural  heritage  sector.  The  consultation  gathered  565  responses.  No 
significant limitations were encountered. More details are provided in the synopsis report 
in Annex 2. 

An  additional  main  source  of  information  for  the  study  and  the  evaluation  were  the 
national  implementation  reports  by  the  Member  States,  which  were  consolidated  in  the 
Commission’s  progress  report  (2018)  on  the  Recommendation’s  implementation  (see 
Section 2 above). 

3.  Data analysis: 

This  implied  assessing  and  compiling  the  results  of  research,  surveys  and  consultations, 
evaluating and assessing findings and discussing them progressively at expert workshops. 
This involved the contractor, as well as the Commission (which included the ISG). 

4.  Synthesis and conclusions:  

15 

 
This  implied drawing conclusions from  the findings  of the evaluation  study, considering 
other inputs from stakeholders, and drawing conclusions on each evaluation criterion. This 
was mainly carried out by the Commission. 

The final steps of this phase mainly consist of: 

  assessing the outcomes  of the evaluation study, of the consultation activities and 

of the implementation reports and then   

  concluding on Recommendation’s impact in the cultural heritage sector across the 
EU  and  more  particularly  on  its  effectiveness,  efficiency,  coherence,  relevance 
and EU added-value. 

A more detailed description is provided in Annex 3. 

4.2. Deviations from the evaluation roadmap  

The  evaluation  roadmap  published  in  July  2019  indicated  that  the  evaluation  would  be 
completed during the third quarter of 2020. However, due to the COVID-19 pandemic and 
related priorities, the Commission opted to extend the evaluation timeframe until the end 
of 2020 to allow sufficient time to process and analyse the results. 

4.3. Limitations and robustness of findings 

Quantifying  the  costs  and  benefits  of  the  intervention  raised  significant  challenges.  This 
was due, on the one hand, to the diffuse and intangible nature of the benefits of cultural 
heritage,  and  on  the  other,  to  the  lack  of  available  or  updated  data  on  costs  in  Member 
States.  The  lack  of  available  or  relevant  data  made  it  difficult  to  provide  a  monetarised 
analysis of the cost-benefits of the Recommendation and its impacts. For example, for cost 
data, the most recent figures on the costs of digitisation, date back to 2017. Moreover, the 
reports do not always provide relevant or consistent data, making it difficult to establish 
causal  links  and  to  draw  proper  conclusions.  However,  where  quantitative  data  are 
missing, alternative proxy data were used and explicitly mentioned in the analysis.  

The evaluation is considered robust and thorough. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

The  main  findings  of  the  evaluation,  based  on  the  five  evaluation  criteria,  i.e. 
effectiveness,  efficiency,  relevance,  coherence,  EU  added  value,  are  summarised  below 
and a list of the evaluation questions is provided in Annex 4. 

5.1   EFFECTIVENESS 

The assessment of effectiveness considers how successful EU action has been in achieving 
or making progress towards achieving its objectives. It looks particularly at: a) the extent 
to  which  Member  States  have  implemented  the  Recommendation’s  provisions,  b)  the 
extent  to  which  the  Recommendation’s  objectives  have  been  achieved  and  the 
achievements  can  be  attributed  to  the  EU  intervention;  what  are  the  factors  preventing 
these objectives from being achieved or that are helping them be achieved. 

16 

 
Key points on the baseline situation 

   The level of digitisation across the Member States was low and CHIs and a vast 

majority of cultural objects worth digitising were not digitised. 

  Despite  an  increasing  trend,  many  Member  States  contributed  rather  little  to  the 

content through Europeana. 

  PPPs were not a widespread tool for financing digitisation. 

Key findings on the current situation: 

  The  Recommendation’s  implementation  by  the  Member  States  varies  to  a  large 

extent. 

  The  Recommendation  led  to  i)  an  overall  improvement  in  digitisation,  online 
access and digital preservation, which was the result  for instance of the adoption 
of  national  strategies  and/or  funding  programmes  for  digitisation  and  domain-
specific  initiatives;  ii)  an  increased  number  of  PPPs;  iii)  more  visibility  and 
exchanges,  leading  to  increased  support  and  funding  for  cultural  heritage;  iv) 
practical measures to optimise the use of digitisation capacity, with positive trends 
since 2011, which can be attributed to the Recommendation. 

  The volume of digitised cultural  heritage resources has increased, with  the target 

levels in Europeana being exceeded ahead of schedule. 

  The  overall  impact  of  Europeana  has  been  positive  and  acknowledged  by  those 
consulted for this evaluation and for the evaluation of the Europeana initiative the 
Commission carried out in September 201824. 

  However, the quantity of digitised cultural heritage should not be dissociated from  
quality;  PPPs were concentrated in  several  countries, with  smaller Member Sates 
experiencing more obstacles; 

  Important  challenges  related  to  copyright  are  expected  to  be  addressed  by  the 
transposition and implementation of the Directive on copyright and related rights 
in the Digital Single Market25 (2019/790/EU) 

  The  Recommendation  needs  to  be  revised  in  order  to  address  also  the  factors 
hindering  the  achievement  of  its  objectives  (e.g.  lack  of  common  standards  and 
approaches,  insufficient  digital  literacy  and  skills,  insufficient  funding)  and  to 
enhance its effectiveness. 

The Recommendation has been effective overall. However, after 10 years, it has reached a 
‘plateau’  in  terms  of  its  effectiveness  and  would  benefit  from  a  revision  to  increase  its 
effectiveness.  This  is  supported  by  the  feedback  from  the  online  public  consultation, 
where  41%  of  the  respondents26  considered  that  the  Recommendation’s  provisions  have 
been implemented effectively and have achieved their objective to improve conditions in 
the areas addressed.  

24 Report on the evaluation of Europeana and the way forward, 6 September 2018, COM(2018) 612 final  
25 Directive (EU) 2019/790 of the European Parliament and of the  Council of 17 April 2019 on copyright 
and related rights in the  Digital Single Market and amending  Directives 96/9/EC and 2001/29/EC ,  OJ L 
130, 17.5.2019, p. 92–125 
26  Moreover,  17%  of  respondents  were  not  in  agreement  with  the  statement,  while  35%  of  respondents 
neither agreed nor disagreed with this statement, 8% did not know. 

17 

 
                                                           
As  shown  in  Section  3  above,  the  extent  to  which  the  various  provisions  of  the 
Recommendation have been implemented27 varies according to the Member State. This is 
the case for instance for the planning and monitoring of digitisation. In this respect, there 
has been a clear overall improvement resulting from i) the adoption of national strategies 
and/or  funding  programmes  for  digitisation,  ii)  domain-specific  initiatives  led  by 
ministries,  the use of regional schemes,  and iv)  quantitative targets  at  national,  regional, 
institutional  or programme level.  For example,  between 2011 and 2013, only  9 Member 
States  reported  using  national  strategies  or  funding  programmes  to  digitise  cultural 
heritage  material28,  whereas  between  2015  and  2017,  23  Member  States  reporting  doing 
so29.  This  increase  can  be  attributed  to  the  Recommendation.  In  particular,  as 
acknowledged  also  during  the  meeting  of  the  DCHE  of  November  201930,  the 
Recommendation  stimulated  discussions  on  digital  for  cultural  heritage.  It  also  raised 
awareness at all levels and across various sectors (e.g. finance), increasing the visibility of 
the domain, its needs and challenges, which led to more concrete support at national level 
(e.g. for digitisation projects). 

During the DCHE meeting of November 2019, participants highlighted the effectiveness 
of the list of indicative targets for minimum content for each Member State’s contribution 
to Europeana, as laid down in Annex II to the Recommendation. The list represented an 
important incentive for the CHIs and Member States in their work to increase the quantity 
of  digitised  assets.  However,  stakeholders31  also  pointed  out  the  fact  that  the 
Recommendation’s current provisions focus rather on increasing the quantity of the digital 
cultural material, and this might have triggered unexpected negative consequences for the 
quality. Achieving a high volume of digitised assets while ensuring high quality remains a 
challenge that needs to be addressed. 

A  key  factor  in  supporting  high-quality  digitised  cultural  material  is  capacity  building, 
with current capacity not responding sufficiently to the needs and challenges of the sector 
(see details below). 

For the private investment through PPPs, the Commission’s consolidated progress report 
(2018) shows that the number of partnerships and institutions involved in PPPs has been 
gradually  increasing  across  the  EU,  with  technology  companies  such  as  Google  and 
Proquest  (for  libraries),  Family  Search  International  (for  archives)  and  Telefónica  (for 
libraries  and  museums)  continuing  to  be  the  main  private  partners  for  large-scale 

27  The  extent  to  which  the  objectives  have  been  achieved  and  the  achievements  attributed  to  the 
Recommendation is assessed across the whole section. 
28  European  Commission  (2014).  Cultural  heritage:  digitisation,  online  accessibility  and  digital 
preservation. Consolidated progress report on the implementation of Commission Recommendation, 2011-
2013. 
29 Commission’s consolidated progress report (2018) 
30  European  Commission  (2019)  Workshop  Report  -  DCHE  workshop  on  the  future  of  the  Commission 
Recommendation  of  27  October  2011  on  the  digitisation  and  online  accessibility  of  cultural  material  and 
digital preservation (2011/711/EU), 5 November 2019 and targeted consultation 
31 Workshop Brussels (January 2020), workshop Lisbon (November 2019) 

18 

 
                                                           
digitisation  projects.  Additional  evidence32  reveals  PPP  agreements  with  local  software 
and technology companies in the field of cultural heritage mostly for digitisation, online 
access  and  digital  preservation  purposes.  The  evaluation  study  further  points  to  specific 
challenges  in  the  role  and  capacity  of  the  PPPs  in  achieving  the  objectives  of  the 
Recommendation,  challenges  that  were  confirmed  during  the  consultation  activities.  For 
instance, while smaller Member States, such as Estonia or Luxembourg, had difficulties in 
defining  such  partnerships,  most  PPPs  tended  to  be  concentrated  in  only  some  bigger 
Member  States,  such  as  France  and  Italy.  Evidence33  on  the  Netherlands,  Hungary  and 
Estonia confirms that they are limited and, as in the other Member States, predictably with 
technology  companies  and  media  publishers  of  educational  materials.  During  the 
workshop  held  by  the  Commission  in  November  2019  in  the  context  of  the  DCHE, 
stakeholders  questioned  the  effectiveness  of  the  provisions/conditions  for  public-private 
partnerships for digitisation, as laid down in Annex I to the Recommendation. Participants 
to the workshop on digital transformation in cultural heritage34 underlined the significant 
role  the  private  sector  has  in  driving  forward  innovation  and  suggested  that  further 
collaboration should be sought to support digitisation efforts. 

Since the Recommendation’s adoption, crowdfunding and match funding have emerged as 
alternative  sources  of  funding  for  digital  cultural  heritage  initiatives35.  In  2016,  for 
example,  Film  Archive  Austria  launched  a  campaign  to  restore  an  historical  silent  film, 
garnering support from over 700 contributors, who donated over EUR 86 000.  

In  case  of  European  Structural  Funds,  18  Member  States  reported  using  European 
Structural  Funds  to  co-finance  digitisation  of  cultural  heritage  during  the  2014-2020 
programming period36. According to the evaluation study incorporating contributions from 
the  Member  States37,  Structural  Funds  played  a  key  role  in  funding  the  digitisation  of 
Europe’s  cultural  heritage  in  those  countries  where  national  sources  were  limited.  The 
targeted consultation  suggests that the Recommendation has been effective in  helping to 
raise  additional  funds  for  digitisation,  for  example  by  giving  the  cultural  heritage  sector 
the opportunity to clearly convey the need for more support from policy makers.  

When consulted, the representatives of the Member States in the DCHE also emphasised 
the  Recommendation’s  effectiveness  in  i)  stimulating  certain  activities  in  their  countries 
(e.g. for the development of a national digital library, digitisation projects), ii) prioritising 
digitisation over other matters and providing financial support, iii) enhancing coordination 
and drawing up digital strategies and iv) building up infrastructure at the national level. 

32  Forbes,  N.  and  Fresa,  A.  (2016).  Public-Private  Partnerships  for  cultural  heritage:  opportunities, 
challenges, future steps, Think Papers Collection 07. Available at:  https://www.digitalmeetsculture.net/wp-
content/uploads/2016/04/rch_thinkpapers_07.pdf 
33 MS58 Study on the costs and funding of digitisation, M24 (Europeana DSI-4)  by Europeana Foundation, 
August 2020 
34 Brussels, January 2020 
35  See  consolidated  report  2018.  Moreover,  the  workshop  held  in  Lisbon  in  2019  emphasised  the 
opportunities offered by crowdsourcing and alternative funding streams. 
36 See consolidated report 2018  
37 Evaluation study incorporating information from the targeted consultation 

19 

 
                                                           
However, stakeholders pointed to  the fact  that  reliable funding sources for CHIs for the 
long  term  remain  a  challenge38.  Findings  from  the  Member  State  consultation  survey 
indicates  that  while  the  Recommendation  has  generally  been  a  useful  instrument  for 
promoting digitisation initiatives at national level, more work needs to be done to unlock 
long-term investments from ministries across Europe39 (see section on Europeana below). 

The Recommendation encouraged Member States to  consider ways to maximise the use 
of  digitisation  capacity  and  achieve  economies  of  scale.  Member  States  reported  that 
practical  measures  to  optimise  the  use  of  digitisation  capacity  since  2011  have  been 
positive,  and  this  can  be  attributed  to  the Recommendation.  Examples  of  such  measures 
include  the  establishment  of  competence  centres,  the  use  of  shared  services  or  facilities, 
and  national  or  cross-border  collaborative  measures  to  pool  digitisation  efforts  and 
resources. One example40 of innovative cross-border initiatives among CHIs is the Digital 
Heritage Network (DEN), previously referred to  as the Netherlands Coalition for Digital 
Heritage. Overall, the Recommendation helped to develop closer cross-border cooperation 
and increase digital capacity in the cultural heritage sector. This has been confirmed by the 
representatives  of 
the 
in 
Recommendation’s effectiveness in facilitating the participation of CHIs in the Europeana 
network and its activities, contributing to a new mind set of openness and sharing. 

the  DCHE41,  who  acknowledged 

the  Member  States 

Participants  in  the  December  2019  workshop  with  international  organisations  and 
networks  of  December  2019  underlined  that  the  Recommendation  would  be  more 
effective 
in 
particularcollaboration at international level (including by providing more support for the 
transfer of technology, the development of skills in non-EU countries). 

the  various  networks  of 

it  further  strengthened 

  CHIs  and 

if 

However,  feedback  from  the  consultation  activities  point  out  that  better  coordination  is 
still needed to ensure interoperability (including semantic) and standardisation42. 

The  Recommendation  encouraged  Member  States  to  improve  access  and  use  of  the 
digitised  cultural  material  that  is  in  the  public  domain,  as  well  as  the  conditions  for 
digitisation  and  online  access  of  in-copyright  material.  The  emergence  of  GLAM  labs 
demonstrates how CHIs have made public domain material more accessible and reusable 
online43. GLAM labs promote innovative ways of working, bringing together institutions, 
technology, people and various communities. Examples include the International GLAM 
Labs  Community,  established  in  2018,  as  a  hub  for  international  collaboration  on 
digitisation and online accessibility. Moreover, at the workshop on digital transformation 
in  cultural  heritage  in  Brussels  in  January  2020,  stakeholders  acknowledged  that  the 
Recommendation had helped to trigger such initiatives. 

38 Workshop Lisbon (November 2019) 
39 See also evaluation study 
40 See the evaluation study 
41 Report workshop DCHE, November 2019 
42 Consultation activities 
43 Evaluation study, international GLAM  Labs Community. Available at:  https://glamlabs.io/books/open-a-
glam-lab/ 

20 

 
                                                           
However,  findings  from  the  workshops  indicated  that  the  Recommendation  does  not 
sufficiently  address  all  the  relevant  copyright  issues  hindering  the  CHIs  in  promoting 
online  access  to  and  the  use/reuse  of  digitised  cultural  heritage44;  CHIs  requested  an 
ecosystem aligning the rules of the Member States (i.e. with EU rules properly supported 
and  enforced45).  Moreover,  there  is  a  general  expectation46  that  the  provisions  of  the 
Directive on copyright and related rights in the Digital Single Market will address some of 
the current obstacles47.  

With regard to the Orphan Works Directive (2012/28/EU), by 2015 all the Member States 
had transposed the Directive. As pointed out in the evaluation study, Member States have 
promoted  the  availability  of  databases  with  rights  information,  connected  at  European 
level,  such  as  ARROW  and  the  Orphan  Works  Database,  established  by  the  European 
Union Intellectual Property Office, in accordance with the Orphan Works Directive. The 
Member  States  also  contributed  to  the  Framework  for  an  EU-Wide  Audiovisual  Orphan 
Works Registry (FORWARD project), which ran from 2013 to 2017.  

The  current  crisis  made  the  need  to  improve  online  access  more  compelling,  as 
acknowledged  by  stakeholders48.  In  particular,  stakeholders  pointed  to  various  obstacles 
when accessing and reusing cultural heritage online, such as a lack of sufficient content, 
insufficient  quality,  the  copyright  and  reuse  status.  They  further  highlighted49    that 
obstacles  to  encouraging  co-creation,  co-curation  and  other  forms  of  interaction  persist. 
Likewise,  collaboration  with  the  archives  is  not  sufficiently  facilitated  to  promote  an 
innovative  work.  Moreover,  for  both  metadata  and  content  data  there  is  insufficient 
multilingual content and more needs to be done to promote this.  

With respect  to  Europeana, the respondents  to  the consultation  generally  acknowledged 
that Europeana had a positive impact on the sector, with spillover effects in education (see 
below) and the creative industries.  

As mentioned in Section 3, the indicative quantitative targets were achieved earlier than 
planned.  However,  while  images  and  text  were  the  main  categories,  there  was  only  a 
marginal  contribution  of  3D  digitised  material  despite  the  generally  acknowledged 
significant  potential  and  value  of  3D  digitisation  technologies.  Various  initiatives 
promoted  the  use  of  3D  technologies  within  the  sector.  For  instance,  the  3D  Content  in 
Europeana  Taskforce50  was  set  up  to  identify  ways  to  enhance  the  accessibility  of  3D 

44 Workshop Brussels (January 2020) 
45 Workshop Lisbon (November 2019) 
46 Workshop Brussels (January 2020)  
47 Article 6 of this Directive requires Member States to introduce an exception to copyright and related rights 
in  their  national  laws  in  order  to  allow  CHIs  to  make  copies  of  works  and  other  subject  matter  for 
preservation  purposes.  Articles  8  to  11  of  the  Directive  provide  a  legal  solution  for  the  digitisation  and 
dissemination of works that are considered to be out of commerce. Article 14 of the Directive ensures that 
reproductions  of  public  domain  works  of  visual  art  cannot  be  protected  by  exclusive  rights  unless  the 
reproduction  material  is  itself  original.  This  Article  prevents  CHIs  from  claiming  copyright  protection  on 
digital copies of works of visual arts that are no longer protected under EU copyright law. 
48 Feedback from the online public consultation 
49 Workshop Brussels, January 2020 
50 https://pro.europeana.eu/project/3d-content-in-europeana 

21 

 
                                                           
collections and Member States collaborated on several joint EU projects51. As mentioned 
above, additional efforts to promote quality content, including multilingual content, need 
to be made. 

In  terms  of  communication  plans  to  raise  awareness  of  Europeana  among  the  general 
public and notably in schools, the Recommendation appears to have been less effective, as 
only seven Member States reported a national communication plan for this action52. Since 
the Recommendation’s adoption, various collaborative projects have taken place, such as 
the ones between the Europeana Foundation and the European Schoolnet, bringing digital 
cultural  heritage  to  classrooms  across  Europe  and  providing  further  opportunities  for 
digital teaching and learning53. However, feedback from consultations highlighted the need 
to  further  enhance  teachers’  awareness  of  Europeana.  Similarly,  Europeana’s  potential 
through CC0 licensing54 should be better emphasised through collaboration with creative 
industries.  

Using  Europeana,  and  more  particularly  the  Europeana  Data  Model,  the  Europeana 
Publishing  Framework  and 
to  develop  a 
standardisation approach in the cultural heritage sector has been important not only in the 
EU,  but  also  at  international  level,  where  our  models  and  best  practices  have  been 
replicated. 

International  Rights  Statements 

the 

Box 1.  

Several of Europeana’s key achievements, i.e. common standards and solutions, such 
as the Europeana Data Model, the International Rights Statements  and the Europeana 
Publishing Framework,  have been taken up widely in the cultural heritage sector in 
Europe and internationally (e.g. in the Digital Public Library of America)55. 

However,  as  mentioned  above,  unlocking  funding  from  ministries  remains  an  important 
challenge overall. Figure 2 below indicates how the member States’ financial contribution 
to  Europeana  increased  in  2011  (when  the  Recommendation  was  adopted),  to  sharply 
decrease afterwards. 

Figure 2. Ministry funding for Europeana 2010-2018 (in EUR) 

51 See the evaluation study 
52 Consolidated report 2018 
53 https://www.europeanschoolnetacademy.eu/courses/course-v1:Europeana+Culture_EN+2020/about 
See also the report “Five years promoting innovative learning with digital cultural heritage”, September 
2020  
54 CC0 is used to waive all the rights in a digital object. By applying this waiver, all possible existing rights 
in the content are waived, and the objects can be used by anyone without any restrictions. Europeana pro 
provides additional details on this. 
55 See Report on the evaluation of Europeana. 

22 

 
 
                                                           
 1.000.000
 900.000
 800.000
 700.000
 600.000
 500.000
 400.000
 300.000
 200.000
 100.000
 -

2010

2011

2012

2013

2014

2015

2016

2017

2018

Ministry Funding

Linear (Ministry Funding)

Source: Evaluation study (Europeana annual reports and accounts, 2010-2018) 

The provisions on digital preservation encourage Member States to have appropriate and 
up-to-date  measures  for  the  digitised  cultural  material,  while  promoting  coordination  at 
EU  level.  As  shown  by  the  evaluation  study,  in  2014  only  26  %  of  CHIs  had  created  a 
written digital preservation strategy that was endorsed by the organisation’s management, 
increasing 
the  Recommendation’s 
contributions to increase digital preservation. Various initiatives have been taken since the 
Recommendation’s  adoption,  including  the  establishment  of  digital  repositories  and 
competence  centres,  the  adoption  of  new  software  and  technologies  for  digital 
preservation (e.g. the use of a repository of e-books on demand in Austria and the Nestor 
catalogue in Germany57).  

to  42%  by  2017.  Stakeholders  confirmed56 

As  shown  in  Section  3  above,  there  has  been  limited  progress  on  multiple  copying  and 
migration, with half of the Member States having explicit legislative provisions to allow 
such activities by public institutions58.  

Although  there  has  been  clear  progress  in  coordinating  approaches  and  preventing 
significant  disparities across the EU  in  legal deposit  arrangements,  there is  considerable 
scope for improvement59. 

Most  of  the  Member  States  currently  have  measures  in  place  to  allow  preservation  by 
using techniques collecting material from the internet, such as web-harvesting techniques, 
although for some this precedes the Recommendation’s adoption (e.g. in Denmark specific 
provisions were laid down through the 2005 Legal Deposit Law and the process is carried 
out by the Royal Danish Library, using the NetArchive web-harvesting platform60). 

56 Targeted consultation DCHE 
57 Evaluation study 
58  Consolidated  report  2018;  European  Commission  (2016).  Cultural  heritage:  digitisation,  online 
accessibility and digital preservation. Consolidated Progress Report on the implementation of Commission 
Recommendation (2011/711/EU) - 2013-2015.  
59 Evaluation study 
60 Evaluation study 

23 

 
 
                                                           
Nonetheless, participants in the November 2019 DCHE workshop highlighted the need to 
strengthen and update the provisions on digital long term preservation, in order to meet the 
needs of the sector, in line with technology developments. 

In order to be fully effective, the Recommendation should also clearly cover all the forms 
of  cultural  heritage  not  addressed  at  all  or  only  minimally  addressed,  e.g.  born  digital, 
intangible  (e.g.  stories,  dances  or  traditions,  performances)  or  immovable.  This  is 
supported  by  the  feedback  from  the  online  public  consultation,  where  the  majority  of 
respondents (65%) were in favour of both broadening the scope of the Recommendation to 
include cultural heritage currently not addressed, and including the digital transformation 
of cultural  heritage institutions.  Moreover, digitisation of intangible cultural  heritage has 
been considered very important by more than half of the overall responses (54%), while 
37%  considered  it  important.  There  is  a  convergence  of  views  across  the  stakeholder 
groups,  from  58%  of  NGOs  considering  it  ‘very  important’  to  47%  of  public  authority 
respondents,  but  also  a  convergence  of  views  across  the  sectors  (e.g.  tourism  full 
agreement). 

Based  on  the  feedback  from  the  consultation  activities,  and  as  illustrated  above,  the 
objectives of the Recommendation can be considered to have been achieved overall.  

The evaluation study pointed to the following  factors that have either contributed to or 
hindered the achievement of the objectives of the Recommendation: 

a) Positive factors:  

The  Member  States’  expert  group,  the  DCHE,  has  been  a  useful  forum  for  the 
Recommendation’s  implementation  and  for  the  exchange  of  best  practices  among  the 
Member  States.  It  has  also  been  acting  as  a  governance  body  for  the  Europeana  Digital 
Service Infrastructure, providing strategic orientations for its future and contributing to the 
sustainability of Europeana. More recently, the DCHE was also instrumental in getting 25 
Member States to sign the 2019 Declaration of Cooperation on advancing the digitisation . 
This has also shown this group’s effectiveness for forward-looking initiatives. 

Tools  have  been  designed  to  support  CHIs  to  become  more  confident  about  using 
digital  to  deliver  on  their  missions,  i.e.  tools  to  design,  organise  and  carry  out  digital 
cultural  heritage  activities61.  The  following  examples  can  be  mentioned:  the  Europeana 
Impact  Playbook62,  a  step-by-step  approach  to  help  CHIs  find  inspiration,  practical 
resources  and  networks  to  more  efficiently  achieve  organisational  goals;  the  Culture24 
project63  promoted  training  and  upskilling  programmes  to  drive  up  digital  capabilities 
across  the  heritage  sector,  or  the  GIFT  BOX64,  a  package  of  open-source  tools  and 
recommendations that help CHIs to make digital experiences richer for their visitors and 

61 Evaluation study 
62 Europeana Impact Playbook: https://pro.europeana.eu/resources/standardization-tools/impac 
63 https://weareculture24.org.uk/ 
64 The GIFT BOX was a result of the Gift Box project, funded by the EU, for more details see: 
https://gifting.digital/ 

24 

 
                                                           
to  enhance  engagement.  The  Europeana  Publishing  Guide65  helps  data  partners  to  share 
collections in Europeana, bringing the existing information to one place and outlining the 
metadata and content requirements for publishing in Europeana. 

Standardisation: Europeana has been particularly successful in the provision of standards 
regarding  the  publication  and  sharing  of  data  that  have  been  taken  up  in  the  cultural 
heritage  industry  across  the  EU  and  internationally,  for  example  by  the  Digital  Public 
Library of America (DPLA). This is, in particular, the case of the Europeana Data Model 
(EDM),  the  Europeana  Publishing  Framework  (EPF)  and  the  RightsStatements.org 
international initiative. The case study done for the evaluation of the Europeana initiative 
in  2018  highlighted  that  the  Europeana  Publishing  Framework  had  a  positive  impact  on 
the overall quality of data, although there was still a lot of scope for further improvement. 

Technological  advancements promoting  open-source software and shared data  have 
improved  the  online  access  of  cultural  heritage  content  and  promoted  collaboration 
between CHIs, with researchers, private or creative sectors, etc. Examples include specific 
factors  like  the  FLOSS  inventory  for  digital  cultural  heritage  and  digital  humanities, 
listing all the Free Libre Open Source Software (FLOSS) relevant to the cultural heritage 
sector, or developments in cloud storage (helping CHIs maintain archives through digital 
preservation)  and  in  linked  data  practices  (promoting  an  open  culture  of  online 
accessibility and reuse in the digital cultural heritage sector).  

Opportunities,  initiatives  and  collaboration  provided  by  EU  funding  for  the  digital 
cultural  heritage  community  (e.g.  under  the  Framework  Programmes  for  Research  and 
Technological  Development,  Connecting  Europe  Facility  or  Horizon  2020)  were  a  key 
factor that helped to achieve the Recommendation’s objectives.  

b) Negative factors:  

Lack of adoption of technical common standards and approaches at regional level  for 
digitised content impeded the achievement of the Recommendation’s intended objectives 
(due,  in  particular,  to  legal  constraints  or  incompatible  file  formats).  The  need  for  more 
interoperability  of  standards  (including  semantic)  appears  to  still  be  a  challenge  for  the 
sector,  as  stated  in  the  workshops.  Similarly,  insufficient  user-friendly  multilingual 
services  must  be  addressed.  Participants  in  the  workshops  held66  by  the  Commission 
highlighted the need to enhance the promotion and application of the FAIR principles, i.e. 
digital data should be findable, accessible, interoperable and reusable. 

Insufficient advanced digital skills and literacy67 in the cultural heritage sector are some 
of the key issues highlighted in the consultations. The digital shift has created new needs 
for the custodians of cultural heritage and new opportunities to be seized to support new 
forms  of  visitor  experience  and  participation.  Research  for  the  evaluation  study 
emphasised  that  insufficient  advanced  digital  skills  and  literacy  have  an  impact  on  the 
availability  of  the  digitised  content,  on  the  efficiency  of  the  process  and  on  the 

65  http://pro.europeana.eu/publication/publishing-framework 
66 Workshop Brussels (January 2020) 
67 A more in-depth analysis will be made in Section 5. 

25 

 
                                                           
possibilities  for  innovation68.  As  underlined  also  in  the  feedback  from  the  online 
consultation, the need for a constant update of knowledge and skills in line with state-of-
the-art  technologies  concerns  all  the  digitisation  related  workflow  and  processes,  from 
digitisation  itself  to  access  and  preservation.  Up-skilling  staff  with  relevant  and 
comprehensive  digital  learning  mechanisms  leads  to  enhanced  digital  skills  and 
knowledge embedded in their work.  

Insufficient  funding  for  CHIs  was  also  singled  out  as  a  major  challenge  by  the 
respondents in the consultation activities69, in particular the difficulty to shift from project 
funding  to  stable  sources  of  funding.  CHIs,  in  particular  the  small  ones70,  struggle  to 
secure  a  sustainable  source  of  funding,  and  this  has  an  impact  on  digitised  content  and 
innovation. The wide ranging impacts of the COVID-19 crisis underlined once more the 
need for the CHIs to get support for their activities, in numerous cases also to ensure their 
long-term survival (see Section below).  

         5.2. EFFICIENCY 

This  section  assesses  the  costs  and  benefits  associated  with  the  Recommendation’s 
implementation,  more  precisely  the  extent  to  which  the  Recommendation  achieved  its 
objectives at a proportionate cost. 

Key point on the baseline: 

  Very often a national budget was not allocated for digitisation activities. 

  For digitisation, CHIs mainly used their own resources (or national and EU funds 

for projects). 

Key findings on the current situation: 

  The absence of available data to quantify the costs of digitisation and preservation 
triggered  by  the  Recommendation’s  implementation  was  a  significant  challenge, 
making  it  difficult  to  draw  conclusions  and  make  comparisons  across  Member 
States. 

  Even if a more detailed analysis of the proportionality costs - benefits is difficult to 

carry out, stakeholders underlined that the benefits outweighed the costs. 

  Most  of  the  costs  were  borne  by  the  Member  States  (with  an  uneven  budget 

allocation across them) and by CHIs; 

  The  Recommendation  provided  an  opportunity  for  more  discussions,  exchanges 
and collaborations for the cultural heritage sector, resulting in more visibility and 
better prioritisation and thus triggering more funding. 

  The  benefits  were  enjoyed  by  the  whole  of  society:  CHIs  have  benefitted  from 
attracting  and  leveraging  funding,  collaborative  networks  and  wider  resources, 
expanding 
in 
cooperation and information sharing; creative sectors, teachers, students, the public 

their  audience;  governments  have  benefitted  from  growth 

68 See evaluation study 
69 Workshop Lisbon (November 2019).  
70 Workshop Brussels (January 2020).  

26 

 
 
                                                           
in  general  have  benefitted  from  a  wider  access  to  digitised  cultural  heritage 
resources.  

  Overall, more than 900 cultural institutions across Europe have benefitted from EU 
support  to  make  their  collections  accessible  through  Europeana,  and  to  carry  out 
targeted  digitisation,  while  50  033  909  cultural  heritage  items  are  digitally 
available through Europeana. 

to 

the  online  public  consultation  gave  positive  feedback  on 

Respondents 
the 
Recommendation’s  efficiency,  with  around  70%  of  them  either  agreeing  or  strongly 
agreeing that the benefits of implementing the Recommendation are significant and justify 
the costs.  

Most  of  the  costs  have  been  borne  by  a)  Member  States  (governments)  and  b)  CHIs, 
while  benefits  have  been  enjoyed  by  everybody/society  as  a  whole  (e.g.  users,  creative 
sectors, teachers, students). 

The main limitation in this section is the absence of available data to quantify the costs of 
digitisation  triggered  by  the  Recommendation’s  implementation.  The  reports  do  not 
always provide relevant or consistent data, making it difficult to establish causal links and 
to  draw  conclusions.  Additional  evidence71  confirms  this  and  highlights  the  fact  that 
national  support  for  digitisation  emerges  from  a  combination  of  programmes  (not 
necessarily  directly linked to  the digitisation of collections) and it is thus  challenging to 
track financial allocations for the digitisation of cultural heritage (along with the scope of 
the heritage field, the cultural policy structure, the coordinating structure). 

Some  of  the  respondents  in  the  targeted  consultation  pointed  to  the  challenges  in 
quantifying the costs or barriers arising from the Recommendation.  

a) Costs for the Member States: 

Most  of  the  evidence  in  this  section  relies  on  the  budget  allocated  to  digitisation  (with 
budgets  only  as  an  indirect  proxy  for  costs72),  together  with  qualitative  analysis,  i.e. 
information  and  views  collected  from  the  consultation  activities  organised  by  the 
Commission  or  in  the  context  of  the  evaluation  study.  Moreover,  as  highlighted  by  the 
evaluation  study,  financial  efforts  for  digitisation  were  already  underway  before  the 
Recommendation’s adoption and they may have continued, but to what extent is uncertain. 
However,  during  the  consultation  activities,  Member  States  and  stakeholders  generally 
acknowledged that the Recommendation provided an opportunity for the cultural heritage 
sector  to  engage  in  more  discussions,  exchanges  and  collaborations,  resulting  in  more 
visibility and better prioritisation and thus triggering more funding. 

The main resources allocated by the Member States to digitisation are presented in Figure 
3. 

71 MS58 Study on the costs and funding of digitisation, M24 (Europeana DSI-4)  by Europeana Foundation, 
August 2020 
72 See evaluation study 

27 

 
 
                                                           
Figure 3. Budgets (in EUR) and sources for digitisation of cultural material 
(2011-2017) 

Country 

2011-2013 
budget 

Sources 

2013-2015 
budget 

Sources73 

2015-2017 
budget 

Sources 

Belgium 

Bulgaria 

Croatia 

Germany 

Greece 

Estonia 

Spain 

France 

Latvia 

Netherlands 

2 800 000 

450 000 

N74 

N75 

370 000 

200 000 

N 

586 778 

384 196 

16 500 000 

N76 

19 300 000 

800 000 

N79 

170 000 000 

EU 

100 000 000 

EU 

5 000 000 

65 000 

400 000 

EU 

N82 

N84 

599 000 

100 000 

2 500 000 

4 500 000 

1 000 000 

110 000 

900 000 

585 000 

173,103 

92 800 

70 000 

2 000 000 

N 

9 631 299 

EU 

N 

N77 

N78 

N80 

N81 

EU 

EU 

N 

N83 

N85 

N86 

N87 

N88 

N 

11 900 000 

EU 

2 100 000 

N 

121,600 000 

5 000 000 

N 

N89 

73 'N' indicates national and 'EU' indicates largely (but not exclusively) European funding. 
74 Flanders. 
75 French community. 
76 DFG (German Research Foundation). 
77 DFG (German Research Foundation). 
78 Baden-Württemberg state. 
79 Berlin municipality. 
80 Brandenburg state. 
81 Saxony state. 
82 SGCB (General Sub-Directorate for Library Coordination). 
83 SGCB (General Sub-Directorate for Library Coordination). 
84 Ministry of Defence. 
85 Ministry of Defence. 
86 Autonomous community – Galicia. 
87 Spanish State Archives. 
88 Spanish Cultural Heritage Institute. 
89 Province of Friesland. 

28 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                           
Country 

2011-2013 
budget 

Sources 

2013-2015 
budget 

Sources73 

2015-2017 
budget 

Sources 

Romania 

Poland 

Slovakia 

Slovenia 

28 000 000 

N 

11 000 000 

28 369 465 

101 000 000 

194 000 000 

EU 

141 000 000 

EU 

141 000 000 

8 000 000 

N 

N 

EU 

EU 

EU 

Source: Evaluation study. 

While  the  figures  presented  above  suggest  that  reporting  of  the  allocated  budgets  is 
uneven,  it  is,  however,  clear  that  some  Member  States  have  been  allocating  substantial 
funds for digitisation activities. Larger Member States have been providing larger amounts 
and smaller Member States (in particular in southern and eastern Europe) have had smaller 
budgets  and  have  been  relying  mainly  on  EU  funds.  For  instance,  in  Germany,  Saxony 
allocated EUR 2.5 million and Baden-Württemberg EUR 0.6 million.  

However, the evaluation study highlights the limited and inconsistent data provided by the 
Member States over the period covering 2011-2017 (i.e. only 14 Member States), without 
a clear separation between national resources and EU funds. This make it difficult to draw 
conclusions and make comparisons across Member States.  

b) Costs for CHIs 

Most of the costs are borne by Member States  (governments) and CHIs, the latter being 
responsible for digitisation. The costs of the CHIs have been used as a proxy for the costs 
stemming from the Recommendation, although it is not possible to establish causal links 
and a direct impact.  

Feedback to from the online public consultation pointed to the fact that the cultural sector 
has been  working with  declining budgets  despite an increase in  technology  related  costs 
and with the pressure for more digitisation (and online access). 
As  shown  by  the  evaluation  study,  based  on  the  ENUMERATE  surveys90,  the  average 
total  costs91  per  institution  doubled  between  2012  and  2017,  from  EUR 176 000  to 
EUR 446 000  (Figure  4).  Total  costs  grew  depending  on  the  type  and  size  of  the  CHI. 
However, it is important to note that increasing the volume of digitised cultural heritage 
material resulted in higher costs. 

90 The ENUMERATE surveys capture costs incurred annually by CHIs for creating, acquiring, maintaining, 
enhancing and preserving the digital collections and are available at: https://pro.europeana.eu/page/past-
surveys  
91 This is an estimate of all costs related to the initial creation, ongoing maintenance, enhancement and 
preservation of the digital collections. 

29 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                           
Figure 4. Average total costs per institution for digital collections (in EUR thousand) 

446 

245 

276 

176 

2012

2013

2015

2017

Source: Evaluation study 
Evidence shows92 that labour costs increased over time, with more full-time equivalents 
(FTE) in  digital  collection activities, from  5.5 FTE in  2012 to  11 FTE in 2017 (05).   In 
terms of paid employment, the average FTE grew from 3.5 FTE to 7 FTE in the period. 
The  increase  in  digitisation  activities  required,  in  addition  to  more  full-time  paid 
employees, volunteers, with an increase from two in 2012 to four in 2017. The growth in 
employment  can  be  considered  a  positive  effect  of 
the  Recommendation’s 
implementation.  

Figure 5. Average FTE in digital collection activities per CHI, 2012-2017 

E
T
F
e
g
a
r
e
v
A

l

a
t
o
T

12

10

8

6

4

2

0

3 

5 

2 

6 

2 

3,5 

4 

7 

2012

2013

2015

2017

Average FTE paid

Average FTE volunteers

Source: Evaluation study  

Moreover,  as  shown  by  the  evaluation  study,  between  2013  and  2017,  the  proportion  of 
costs  remained  the  same,  with  incidental  costs  (e.g.  for  the  selection  of  material,  the 
initial  creation  or  acquisition  of  a  digital  collection)  representing  53 %  and  structural 
costs  (e.g.  ongoing  maintenance,  enhancement  and  preservation  of  a  digital  collection) 
47 % of the total costs.  

92 Evaluation study 

30 

 
 
 
                                                           
 
 
 
In  some  Member  States,  costs  are  also  borne  by  private  sources.  In  the  Netherlands, 
private funding is increasing93, although some sources are more for projects involving the 
digitisation and reuse of archival materials for publications, for example. In Estonia there 
are only a few of these private funds (e.g. the Soros Fund before closure).  

Benefits of the Recommendation’s implementation:  

Despite  the  difficulty  in  quantifying  such  benefits,  during  the  consultation  activities 
stakeholders highlighted the benefits of the Recommendation for the whole of society. 

a) Benefits for the Member States 

As mentioned above, participants in the DCHE meeting in November 2019 noted that the 
Recommendation  was  important  in  promoting  the  sector  and  enhancing  cooperation 
among the Member States, while fostering better organisation and coordination internally. 
This led to better prioritisation, allocation and a more efficient use of financial resources.  

Moreover, raising public awareness of and through digital cultural heritage assets (e.g. see 
Section 2.2.2. on Europeana) has led to spill-over effects in other domains, e.g. tourism.   

b) Benefits for the cultural heritage sector 

the  consultation  activities, 

As  acknowledged  during 
the  Recommendation’s 
implementation  largely  benefitted  the  sector94  by  exchanging  information  and  more 
generally  know-how,  by  promoting  and  sharing  tools,  networks  and  connections  and  by 
facilitating access to resources and digital activities that might not have been carried out 
without the Recommendation. It created additional opportunities to gain new perspectives 
from working with peers and experts in digital cultural heritage. This has been particularly 
relevant for smaller CHIs, whose access to funding and networks is more limited. 

In this regard, the Europeana Network Association95 has been very successful in bringing 
together  heritage  professionals,  tech  providers,  educators,  researchers  and  copyright 
specialists. Its six communities (Europeana Tech, Communicators, Copyright, Education, 
Impact  and  Research)  allow  numerous  CHIs  to  network,  learn  and  exchange  best 
practices. Collaboration between the tech industry and the cultural heritage sector required 
joint forces, creating efficiency gains, stimulating innovation and enhancing the economic 
potential  of  Europe’s  cultural  heritage  sector.  Likewise,  developing  synergies  with  other 
public  institutions  stimulated  progress  on  the  use  of  new  advanced  technologies.  For 
instance, the Tuscany Region in Italy and the University of Florence established the New 
Media  for  Cultural  Heritage  Competence  Centre  (NEMECH)96,  connecting  research 
centres  and  institutions,  promoting  the  transfer  of  research  know-how  and  providing 
digital technologies tools and applications for cultural heritage.  

As mentioned above, increased visibility of the sector led to increased support for cultural 
heritage  and  access  to  more  funding  and  infrastructure.  Overall,  more  than  900  cultural 
institutions across Europe benefitted from EU support to make their collections accessible 
through Europeana, and to carry out targeted digitisation. This raised awareness among the 
cultural heritage sector on the importance of having a digital presence, has advanced the 
digital  experience  of  cultural  heritage  institutions  and  mobilised  cultural  institutions  to 
make  their  material  accessible  online.  More  than  3  700  institutions  have  contributed  to 

93 MS58 Study on the costs and funding of digitisation, M24 (Europeana DSI-4)  by Europeana Foundation, 
August 2020 
94 Report DCHE workshop November 2019 and targeted consultation  
95 https://pro.europeana.eu/network-association/sign-up 
96 Interreg Europe (2018). Digital solutions in the field of cultural heritage  

31 

 
                                                           
Europeana  and  50  033  909  European  cultural  heritage  items  are  digitally  accessible 
through it.  

Also  before  the  Recommendation,  several  countries  did  not  have  digital  strategies,  the 
Recommendation  helped  stimulate  the  adoption  of  such  strategies  and  thus  led  to  more 
funding  becoming  available  for  digital  projects.  Moreover,  the  greater  number  of 
discussions  and  exchanges  of  information  have  enabled  cultural  heritage  custodians  to 
better understand their needs and challenges in  a rapidly evolving technological  context. 
For  instance,  capacity  building  and  particularly  the  upskilling  of  staff  (not  mentioned  in 
the Recommendation) became an important element to consider in order to get prepared to 
efficiently and effectively use the opportunities provided by the digital transformation. 

As pointed out by the evaluation study, although there is no direct or quantifiable evidence 
of the Recommendation’s impact on other sectors (e.g. tourism, education and research), 
the  responses  during  the  consultation  activities  highlighted  the  Recommendation’s 
contribution to the rest of the economy within and across Member States and the synergies 
with other projects and initiatives.   

Box 2. 
 The  EU-funded  APEx97  project  (Archives  Portal  Europe  network  of  eXcellence)  ran 
from March 2012 to September 2015, funded under ICT Policy Support  programme, as 
part  of  the  Competitiveness  and  Innovation  framework  programme  (CIP).  The  APEx 
project involved 33 partners from archival and research organisations from 30 countries, 
and hundreds of participating and content providing archives. APEx's overall goal was to 
provide easy access via the Archives Portal Europe (APE) to as much archival content of 
European institutions as possible and to channel all digitised and digital archival material 
to  Europeana.  APEx  was  by  far  one  of  the  biggest  archival  cooperation  projects  in 
Europe.  The  main  result  from  the  project  was  the  Archives  Portal  Europe,  including 
substantially  improved  stability  and  capacity  and  usability,  and  significantly  enriched 
content; this gave users online access to a great  amount of European archival data (and 
metadata).   

c) Benefits for the general public and society at large 

As  explained  in  the  sections  above,  some  of  the  benefits  of  the  Recommendation,  as 
mentioned during the consultation activities, include wider access to previously restricted 
content  and  democratisation  of  culture  and  knowledge,  grater  efforts  to  innovate  and 
develop  new  technologies  and  services  with  spillover  effects  ino  other  sectors  (e.g. 
tourism, research, education).  

Moreover,  the  Recommendation  stimulated  policies,  strategies  and  projects  at  EU, 
national and local levels for the benefit of the whole of society; it encouraged co-creation 
and co-curation, while keeping people and communities connected. Historiana is  a  good 
example  of  an  initiative  promoting  virtual  learning  for  educational  purposes;  its 
collaboration  with  Europeana  aimed  to  create  new  ways  for  history  educators  to  create, 
share and use e-learning activities with content from the Europeana collections website98. 

97 www.apex-project.eu 
98 https://pro.europeana.eu/post/presenting-the-opening-up-historiana-project 

32 

 
 
 
 
                                                           
Responses to  the  consultations indicated that improving digital  skills  and  digital  literacy 
leads to  further efficiency  gains,  as mentioned above. The Recommendation  should take 
into account how digital content is produced and used by cultural heritage professionals, 
creative  sectors,  research,  education,  users,  etc.  Up-skilling  staff  with  relevant  and 
comprehensive  digital  learning  abilities  leads  to  enhanced  digital  skills  and  knowledge 
embedded in  their work. Doing sowould lower costs and maximise resources. Similarly, 
increased collaboration and common initiatives, sharing of know-how and networks would 
lead to a more efficient use of resources and the fulfilment of organisational missions. 

Figure 6. Comparison of benefits and costs 

Stakeholder 

General public, creative 
sector, teachers, students 
and researchers  

CHIs and professionals 

Member States  

Benefits 

Costs 

Access to content 

Learning 

Co-creation and co-curation 

Funding and collaboration 
networks 

Digital skills 

Audience expansion and public 
mission 

Co-operation and information 
sharing 

+++ 

++ 

+ 

++ 

+ 

+ 

Incidental and 
structural costs 

+++ 

Labour costs 

++ 

++  Digitisation 

+ 

funding 

Legend: the '+' indicates the Recommendation’s impact, ranging from small (+), medium (++), and 
large (+++). 

Source: Evaluation study 

During the consultation activities respondents highlighted that digital transformation is not 
an  option  anymore,  it  is  a  reality,  it  is  a  requirement  of  the  current  technological 
development  of  society  and  CHIs  must  embrace  and  support  it.  This  may  translate 
(initially) into additional costs (e.g. in infrastructure, in digital skills and literacy), but also 
benefits  (e.g.  more  agile  processes,  improved  knowledge  and  visibility,  reaching  wider 
audiences). Results can be achieved with fewer resources only once the required changes 
have  been  implemented  and  the  processes  are  optimised  and  working.  Increased 
collaboration  between  private  and  public,  as  well  as  between  cultural  heritage 
professionals, will lead to efficiency gains, while greater synergies with other sectors will 
benefit the whole of society.  

Overall, it was acknowledged that implementing the right digital tools and acquiring the 
necessary  skillsets  can  help  cultural  heritage  professionals  to  optimise  processes  and 
resources, while building confident CHIs and engaged users. 

However, in a context where the COVID-19 crisis strongly hit the sector, threatening the 
long-term financial sustainability of many CHIs, funding becomes even more important in 
a context where governments will most likely make additional budgetary cuts99. 

99 See paper on the challenges and opportunities presented by the COVID-19 pandemic for the heritage 
world by Europa Nostra 

33 

 
 
 
 
 
 
                                                           
5. 3. RELEVANCE 

The  main  question  addressed  in  this  section  is  whether  and  to  what  extent  the 
Recommendation  has  been  and  is  still  relevant  to  the  needs  of  the  stakeholders.  This 
section  looks  at  how  well  the  Recommendation  is  adapted  to  future  technological  and 
social advances. 

Key points on the baseline: 

  Despite  the  efforts  and  progress  of  the  Member  States  and  CHIs  to  digitise, 
preserve  and  provide  online  access  to  digitised  cultural  heritage  content,  further 
steps were needed.  

  The need for more digitisation, online access and preservation, along with the need 
for  more  harmonised  efforts  across  the  EU  to  unlock  Europe’s  economic  and 
cultural potential made the case for the Recommendation.  

Key findings on the current situation: 

  Stakeholders  generally  acknowledged 

the  Recommendation’s  relevance 

in  

responding to the needs identified at the time of its adoption; 

  Evidence  pointed  not  only  to  the  Recommendation’s  relevance,  but  also  to  the 

need to strengthen it to adapt it to the current needs and challenges. 

  Socio-economic changes, technological advancements, legislative changes create a 
different context for CHIs that must be taken into account. The Recommendation 
needs to better reflect the huge potential of advanced digital technologies (e.g.3D 
digitisation) and emerging technologies (including immersive technologies such as 
VR  or  AR  and  AI),  bringing  unprecedented  opportunities  for  digitisation,  online 
access and preservation. 

  The  need  for  advanced  digital  literacy  and  skills  is  one  of  the  key  issues 

highlighted in all the consultation activities. 

   It  is  important  to  make  full  use  of  cultural  heritage  data  for  the  benefit  of  the 

sector and to enhance interoperability and capacity building. 

  Intangible and born-digital  cultural  heritage need to  be taken into account  in  any 

future revision of the Recommendation. 

feedback  gathered 

The 
revealed  a  general 
acknowledgement  of  the  relevance  of  the  Recommendation  in  responding  to  the  needs 
identified at the time of its adoption.  

the  consultation  activities100 

from 

As  mentioned  above,  the  Recommendation  sought  to  address  the  need  to  a)  unlock 
Europe’s  economic  and  cultural  potential,  b)  better  harmonise  national  approaches 
across Member States to digitisation and the preservation of cultural heritage, as well as 
c)  increase  digitisation,  online  access  and  preservation  to  avoid  losing  our  cultural 
heritage assets and their memory (due to floods, fires, mass tourism, etc.). 

At  the  time  of  the  Recommendation’s  adoption,  the  context  for  digitisation  efforts  was 
changing: more people were consuming culture online, CHIs needed to change tools and 
organisational  culture  and  the  creative  sectors,  which  are  a  driver  for  innovation  and 

100Targeted consultation and DCHE workshop of November 2019. 

34 

 
 
                                                           
economic  growth,  were  flourishing.  Managing  the  ‘digital  shift’  required  a  strategic 
approach defining how creativity could be better promoted and how access to and the use 
of  digitised  cultural  resources  could  be  facilitated,  while  preserving  them  for  the  next 
generations101.  

Stakeholders  and  Member  States  representatives  acknowledged  the  Recommendation’s 
contribution  in  encouraging  public  authorities  to  prioritise  and  increase  support  for 
digitisation  and  provide  more  funding,  while  increasing  online  access  and  preservation. 
Using digital  for creation and culture led to  unlocking the potential for  broader societal, 
cohesive  and  economic  benefits  from  sectors  such  as  sustainable  tourism,  education  and 
creative sectors. The impact was thus on the whole of society: on CHIs (e.g. able to reach 
out to broader audiences and engage with new users), creative sectors (e.g. stimulation of 
creativity  and  content),  and  teachers,  students,  the  public  (e.g.  wider  access  to  cultural 
material), etc.  

As  mentioned  in  Section  5.1,  participants  in  the  DCHE  workshop  of  November  2019 
highlighted the importance of the Recommendation in helping CHIs to participate in the 
Europeana network and its activities, thus encouraging the sharing of knowledge and best 
practices. Through Europeana, the Recommendation has also played a significant role in 
the  digital  development  of  the  cultural  heritage  sector  in  Europe  by  strengthening 
cooperation and standardising cross-border activities.  

Moreover,  the  Recommendation’s  relevance  through  Europeana  and  more  particularly 
through  the  synergies  created  within  the  sector  of  education,  i.e.  to  provide  further 
opportunities for digital teaching and learning (see Section 5.1.) has been confirmed by the 
responses  to  the  2017  Eurobarometer  on  cultural  heritage102,  where  a  large  majority  of 
respondents (88%) agree that Europe's cultural heritage should be taught in schools.  

Box 3.  

In  March  2017  the  Europeana4Education  initiative  was  launched  to  help  inspire 
learners  and  enrich  educational  resources  with  Europeana  content.  Europeana 
created  a  dedicated  education  area  on  Europeana  Pro,  allowing  visitors  to  explore 
partnerships,  browse  case  studies  of  educational  applications,  resources  and 
platforms featuring Europeana content, and get practical help from the collection of 
resources. Today, Europeana Classroom103 provides a wealth of innovative learning 
for  educators  and  learners  with  a  selection  of  educational  resources  using  digital 
culture. Partnerships with EuroClio, the European Association of History Educators, 
for example, have produced useful case studies that provide insights into the use of 
online collections for and by educators104. 

101 See footnote 10. 
102 European Commission (2017). Special Eurobarometer 466 on cultural heritage. 
103 https://www.europeana.eu/en/europeana-classroom 
104 https://pro.europeana.eu/post/making-your-collections-fit-for-education-a-case-study-from-euroclio 

35 

 
 
                                                           
As indicated in Section 2 above, evidence points to the high number of objects that were 
not  digitised105  before  the  Recommendation  and  to  the  key  role  it  had  in  promoting  and 
supporting digitisation and online access to digitised content. This further helped to avoid 
the  complete  loss  of  the  memory  of  cultural  heritage  assets  due  to  natural  or  human 
derived factors. 

the  consultations  also 

While the outcome of the consultation  activities revealed a  general  acknowledgement of 
Recommendation’s  the  relevance  in  responding  to  the  relevant  needs  and  achieving  its 
the 
objectives, 
Recommendation, to make it better reflect the current technological, economic and societal 
state of play. More particularly, 95% of the respondents to the online public consultation 
agreed that the Recommendation should be updated to reflect today’s needs in the area and 
increase the potential of Europe’s cultural heritage. 

revealed  a  strong  need 

to  strengthen 

Since  the  Recommendation’s  adoption,  there  have  been  significant  and  interlinked 
changes  in  the  context  in  which  digitisation,  online  access  and  digital  preservation  take 
place:  socio-economic  changes  leading  to  different  and  increased  user  needs, 
technological  changes  (e.g.  AI,  machine  learning),  legislative  changes  (e.g.  on 
copyright). These changes resulted in new opportunities and challenges for the CHIs and 
guided  their  choices  and  actions.  The  majority  of  the  responses  from  the  DCHE 
representatives in the targeted consultation pointed to the fast paced changes in the digital 
environment  that  the  sector  needs  to  cope  with  and  urged  the  Commission  to  revise  the 
Recommendation to keep it relevant for the future.  

As emphasised by the evaluation study, better access to and a greater use of the internet, 
together  with  increased  consumption  of  online  goods  and  services,  higher  cultural 
employment  and  government  expenditure  on  recreation,  religion  and  culture  are  all 
examples of what has helped to create a new environment for both providers and users of 
Europe’s digitised cultural heritage. The development of these factors is shown in Figure 
6.  

Figure 6. User characteristics in relation to internet use and the culture 
sector, EU-28 average106, 2011 - 2017/19 

Internet use 

2011 

Most recent data 
available 

Change 

Households – level of internet 
access107 
Individuals – internet use108 
Internet use: purchasing cultural goods and services109 

73 % 

73 % 

90 % (2019) 

88 % (2019) 

Films and music 

17 % 

17 % (2018) 

+ 

+ 

+ 

105 See also NUMERIC study  
106106 The statistics in this table were obtained from the Eurostat database. The indicator codes below provide 
a reference to the specific dataset where each indicator can be found.  
107 Eurostat indicator code: isoc_ci_in_h. 
108 Eurostat indicator code: isoc_ci_ifp_iu. 
109 Eurostat indicator code: isoc_ec_ibuy. 

36 

 
 
 
 
                                                           
2011 

21 % 

21 % 

8 % 

Most recent data 
available 

22 % (2018) 

Change 

27 % (2018) 

14 % (2017) 

7.9 million  

8.7 million (2018) 

3.6 % 

3.8 % 

1.8 million 

2 million (2018) 

Books, magazines and 
newspapers 

Tickets for events 

Films and music delivered or 
upgraded online 

Cultural engagement 

Cultural employment (absolute 
value)110 

Cultural employment (% total 
employment) 

Persons working as creative and 
performing artists, authors, 
journalists and linguists111 
Government expenditure112 

Recreation, religion and culture 

2.3 % 

Cultural services 

1.0 % 

2.3 % (2017) 

1.0 % (2017) 

Source: Evaluation study 

+ 

+ 

+ 

+ 

+ 

+ 

= 

= 

The 2017 Eurobarometer113 reveals that over half of the respondents (55%) have used the 
internet in the last 12 months for at least one of a range of cultural heritage purposes, and a 
large part of them (31%) for the  accessibility, facilities and main features of a museum, 
historical  monument  or  traditional  event  in  preparation  for  a  visit  or  their  holidays.  In 
addition,  almost  one  quarter  (23%)  used  the  internet  for  buying  or  booking  services  for 
events or activities (e.g. tickets, guided tours, etc.) while 21% used it for viewing cultural 
heritage-related  content, such as  the description of a work of art or historical  monument 
during  a  visit,  historical  information  about  a  traditional  event  they  attend;  11%  of  the 
respondents have created or shared cultural heritage-related content, such as a picture or a 
video  of  a  work  of  art  or  historical  monument,  etc.  This  shows  a  high  interest  of 
Europeans  in  getting  a  wider  virtual  access  to  cultural  heritage  and  confirms  the 
Recommendation’s objectives and relevance.  

The responses to the online public consultation highlighted the sector’s efforts to increase 
online access during confinement, through more digital learning resources (e.g. recording 
lectures  and  support  material  to  be  streamed);  more  digital  activities  (such  as  extending 
online access to curated audio-visual assets, virtual tours of exhibitions), increased social 
media  presence  and  online  collaboration.  The  COVID-19  pandemic  has  acted  like  a 
catalyst  for  increased  online  activities  and  services,  with  the  majority  of  respondents 
stating that they envision additional changes by their organisations in the future (e.g. better 
procedures,  better  quality  of  the  digital  content,  more  frequent  use  of  collaboration 
platforms especially for teaching and research activities). However, if on the one hand the 
crisis  accelerated    the  digital  transformation  of  the  heritage  sector,  on  the  other  it  also 

110 Eurostat indicator code: cult_emp_age. 
111 Eurostat indicator code: cult_emp_art. 
112 Eurostat indicator code: gov_10a_exp. 
113 European Commission (2017). Special Eurobarometer 466 on cultural heritage.  

37 

 
 
 
 
                                                           
deepened the current inequalities between large and small CHIs (i.e. the latter not having 
the same resources to share their collections and materials online, missing out on digital 
opportunities)114. 

The  development  of  digital  technologies  such  as  AI,  computer  vision,  deep  learning, 
machine  learning,  cloud  computing,  Big  Data,  VR  and  AR  has  brought  unprecedented 
opportunities  for  digitisation,  online  access  and  preservation.  Digital  technologies  can 
empower  and  encourage  people  to  participate  in  culture  in  a  more  active  and  creative 
way115. In particular, AI allows manifold uses of diversified and rich data across Member 
States and CHIs. For instance, it has a great potential to extract knowledge from different 
data sets of cultural artefacts held by CHIs across Member States. AI can allow users to 
navigate  through  extremely  vast  amounts  of  cultural  and  creative  content116,  while 
reducing  the  efforts  and  resources  required  for  such  effort  by  CHIs.  AI  can  also  boost 
automated annotation of digitised cultural heritage, or can help to reconstruct lost cultural 
heritage information or to extract new knowledge from cultural heritage data117. 

As  underlined  in  the  feedback  from  the  public  consultation118,  VR/AR  technologies  can 
enhance  the  user  experience,  while  providing  great  potential  for  use  in  education  and 
research. 

3D  technologies  are  now  mature  enough  to  help  in  preservation  and  restoration  or  in 
providing virtual access to cultural heritage where it is difficult or impossible to get (e.g. 
under water). These technologies provide numerous opportunities for CHIs to reach wider 
audiences, with more immersive experiences, for both onsite and online visitors. The need 
to  focus  more  on  higher  quality  and  more  particularly  on  3D  emerged  from  both  the 
consultation activities, and from the reports of the Member States119. For instance, a large 
majority of overall respondents to the public online consultation consider it very valuable 
or valuable to create digital twins of cultural heritage buildings, monuments and sites such 
as Notre Dame, ahead of any damage by fire or other disasters and more than half of the 
respondents (53%) agreed that 3D technologies are very valuable for creating high-quality 
3D models of museum objects (in addition to 38% agreeing it is valuable). There is a clear 
need  to  strike  the  right  balance  between  quantity  and  quality,  moving  away  from  the 
current  approach  focused  mostly  on  quantity.  In  this  respect,  stakeholders  argued120  that 
since the Recommendation’s adoption, the emphasis has been on increasing the volume of 
digitised cultural  assets  and ensuring a high quality for all of them (including metadata) 
and this remains a challenge.  

Therefore,  in  order  to  remain  relevant,  the  Recommendation  should  address  the  role 
advanced  technologies  can  play  in  the  field  (e.g.  strengthening  the  ways  CHIs  could 

114 See paper on the challenges and opportunities presented by the COVID-19 pandemic for the heritage 
world by Europa Nostra 
115 See responses to the online public consultation 
116 See Motion for a European Parliament Resolution on artificial intelligence in education, culture and the 
audiovisual sector (2020/2017(INI)) 
117 See responses to the online consultation. Additional uses indicated were automated transcription, 
automated translation and visual recognition. 
118 Feedback from the online consultation. 
119 Consolidated Report 2018. 
120 Workshop Brussels (January 2020). 

38 

 
                                                           
deliver  higher  quality  content121,  supporting  new  forms  of  visitor  experience  and 
engagement), while fostering inclusivity, equality, diversity. It is important to note that a 
large majority of the  respondents  to  the public consultation  (and also  across  the sectors) 
agreed that, based on the lessons learned from the COVID-19 crisis, digital technologies 
would  be  very  important  in  making  their  organisation  more  resilient  in  the  future  while 
helping to increase the number of people accessing cultural heritage through the internet.  

Participants  in  the  workshop  on  digital  transformation  held  in  Brussels  (2020)  indicated 
the need to properly address the issue of making full use of cultural heritage data for the 
benefit of the sector. Cultural  heritage data  are currently not  sufficiently  collected, used 
and  reused  within  this  sector  or  across  other  sectors  (e.g.  education,  research)  and  the 
Recommendation would need to properly reflect this. Social media and the existing web 
data could, for example, be an important source of data and more efforts should be placed 
on this. In terms of repositories or other elements of data infrastructure, maximising their 
impact through a wide scale initiative is important122. The respondents to the online public 
consultation (96%), considered common data spaces for sharing cultural heritage data and 
building  high-value  data  sets  to  be  valuable  instruments  in  the  field.  This  view  was 
particularly  shared  by  stakeholders  such  as  academic  or  research  institutions,  public 
institutions,  the  public  or  sectors  such  as  the  technology  /ICT,  cultural,  education  and 
research  sectors.  Likewise,  respondents  to  the  online  consultation  considered  repository 
infrastructure  for  digitised  cultural  heritage  content  (95%),  cloud  infrastructure  (90%), 
supercomputing  capabilities  for  large-scale  cultural  heritage  simulations  (76%)  key  to 
cultural heritage. 

While  the  Recommendation  has  contributed  to  more  interoperability  in  the  sector  by 
supporting the development of a framework of interoperability and increasing awareness 
of  common  frameworks,  there  is  still  work  to  be  done  for  instance  on  semantic 
interoperability or interoperability of standards123.  

Ensuring  appropriate  means  for  preservation  implies  not  only  sustaining  single  digital 
objects,  but  also  creating  the  right  context  for  them  to  be  understood  in  the  future. 
Overcoming technology  obsolescence is  a key element in  avoiding a complete or partial 
loss of digital objects and is still a complex task124.   

Users  have  become  more  demanding  with  regard  to  the  products  and  services  they 
consume/use, wanting to participate, enjoy and learn more. They challenge the CHIs more 
and  require  a  stronger  link  with  them,  i.e.  to  maximise  and  diversify  the  possibilities  to 
digitally interact with cultural content.  
According to stakeholders125, in order to embrace digital transformation, a key factor that 
professionals of the sector need to address is capacity building. In this respect, providing 
staff with advanced digital skills and literacy remains a big challenge for many of them. 
In particular, half of the respondents to the public online consultation confirmed that their 
organisation had struggled with the advanced digital skills needed to manage their online 
presence during the pandemic. Respondents working in the education and research sectors 
were  likely  to  be  the  most  affected,  followed  by  respondents  from  the  culture  sector. 
Additional  evidence126  collected  during  the  pandemic  confirms  that  many  CHIs  in  rural 

121 See also responses to the online public consultation. 
122 Workshop international organisations, December 2019 
123 Workshop DCHE, November 2019 Luxembourg , responses to the online public consultation 
124 Digital Cultural Heritage Roadmap for Preservation: https://www.dchrp.eu/ 
125 Particularly in the workshop on digital transformation, Brussels (January 2020) 
126 See paper on the challenges and opportunities presented by the COVID-19 pandemic for the heritage 
world by Europa Nostra 

39 

 
                                                           
areas  did  not  have  the  necessary  digital  skills  to  carry  out  their  activities  during  the 
lockdown, leading to a significant void in culture on offer.  

In  addition  to  the  large  variety  of  CHIs  and  the  many  different  tasks  that  staff  need  to 
carry  out,  knowledge  and  skills  should  be  constantly  updated  and  kept  in  line  with 
technological  developments.  Feedback  from  stakeholders,  particularly  from  the  January 
2020  workshop127  pointed  to  the  importance  of  having  a  common  standard  and 
classification  of  skillsets  for  cultural  heritage  professionals  (e.g.  for  senior  and  middle 
management, as well as expert levels), depending on the specific needs and domains (e.g. 
legal,  technical,  life-cycle management,  communication). Other stakeholders pointed out 
for  example  that  researchers  and  practitioners  do  not  have  the  knowledge  and  ability  to 
master  digital  tools  and  devices  to  deal  with  the  accelerated  development  of  ICT128. 
Ensuring the necessary (specific) skills and capacities of the professionals of the sector is a 
priority for the creation, management and development of digitised cultural heritage, with 
spillover effects ino other sectors. Feedback from the online consultation highlights what 
stakeholders consider to be the core set of digital skills that professionals from the cultural 
heritage  sector  should  acquire:  data  and  metadata  management  and  analytics,  and 
copyright and licensing, followed by digitisation (2D and/or 3D)129. Furthermore, findings 
from  the  DCHE  workshop  in  November  2019  highlighted  the  importance  of  a  higher 
competence level for staff that would drive up capabilities across the heritage sector.  

At  a  more  general  level,  in  the  consultation  on  the  strategic  objectives  of  the  Digital 
Europe programme, held between 25July and 25October 2019130, 74% of the respondents 
indicated that the EU should support the Member States in improving the advanced digital 
skills necessary to operate the up-to-date digital infrastructures. Likewise, a large number 
of responses considered EU support for culture very important. 

New  forms  of  cultural  heritage  have  emerged,  and  their  collection  and  preservation 
remain a challenge, as indicated by stakeholders during the consultation activities (e.g. for 
born-digital  heritage,  meaning  it  has  been  created  digitally,  without  an  analogue 
equivalent or cultural content created outside CHIs, such as on social media or the gaming 
industry131). Due to a rapidly evolving technology, with hardware and software becoming 
quickly  obsolete,  born-digital  cultural  heritage  needs  to  be  properly  collected,  managed 
and preserved to be accessible and usable in the long run.  

While in the short term, most of the CHIs have been focusing on the immediate challenges 
raised by the COVID-19 pandemic, new models and ways of working will emerge in the 
longer term132. CHIs will learn from the pandemic, by creating innovative approaches for 
digital  literacy  and  skills,  processes,  content  and  communication,  through  a  new 
organisational culture and mindset. 

127 Workshop Brussels ( January 2020) 
128 https://www.vi-mm.eu/wp-content/uploads/2016/12/ViMM-Manifesto-Revised-Final-Revised-19-
November.pdf 
129 Presentation of digital content, communication and outreach (of audience analysis and user-centric 
methodologies) were additionally referenced by stakeholders. 
130 https://ec.europa.eu/digital-single-market/en/news/digital-europe-programme-summary-report-targeted-
consultation-future-investment-europes-digital 
131  Games  are  an  example  of  a  creative  participation  in  culture  using  digital  technologies,  with 
commercial/economic and social impact. 
132 See footnote 4. 

40 

 
                                                           
5. 4. COHERENCE 

This  section  assesses  how  the  various  actions  have  worked  together  to  achieve  the 
objectives. It looks particularly at the extent to which the Recommendation is aligned and 
consistent  with  other  EU  policies,  with  Europeana-  related  policy/initiatives  and  with 
cultural heritage policies in the Member States. 

Key points on the baseline: 

  The Recommendation has created synergies with the European Agenda for Culture 

(2007), helping to shape the policy framework for the cultural sector. 

  It has built on the work done through Europeana (2008). 

  At the time of its adoption, the Recommendation was an important action for one 
of the key areas tackled by the Digital Agenda for Europe, a flagship initiative of 
the Europe 2020 strategy (2010).  

Key findings on the current situation: 

  Overall, the Recommendation is aligned with several other policy interventions in 
different  fields, such as culture or copyright.  In particular, evidence suggests that 
the Recommendation is consistent with the New European Agenda for Culture, the 
Work  Plan  for  Culture  covering  the  period  2019-2022,  the  European  Framework 
for Action on Cultural Heritage, the Digital Agenda for Europe, the Digital Single 
Market strategy, the Directive on copyright and related rights in the Digital Single 
Market and the Directive on open data and the reuse of public sector information. 

  However,  a  future  revision  of  the  Recommendation  should  aim  to  ensure  better 
alignment  with  these  more  recent  initiatives  such  as  the  new  digital  strategy 
‘Shaping Europe’s Digital Future’ to create a Europe fit for the digital age and with 
’A European strategy for data’. 

A majority of the respondents to the online public consultation (around 59%) considered 
that the Recommendation had been coherent with other EU policies and initiatives.  

The Recommendation is consistent with several other EU policy interventions in various 
fields,  such  as  culture  and  copyright,  or  is  aiming,  more  generally,  to  unlock  Europe's 
innovative  capabilities,  and  to  exploit  the  economic  and  societal  benefits  of  a  digital 
society. 

In  the  field  of  culture,  the  Recommendation  has  been  sharing  the  same  objectives  and 
efforts  with  more  specific  actions  in  the  field,  to  promote  culture  and  its  key  role  in 
Europe’s society and economy. For instance, the Recommendation has created synergies 
with the European Agenda for Culture, adopted in 2007133, helping to shape the policy 
framework for the cultural sector. By encouraging the Member States to take actions for 
greater  digitisation,  online  accessibility  and  preservation,  the  Recommendation  has 
contributed to the objectives of the European Agenda for Culture, namely:  

  promotion of cultural diversity and intercultural dialogue;  
  promotion  of  culture  as  a  catalyst  for  creativity  in  the  framework  of  the  Lisbon 

Strategy for growth and jobs;  

  promotion of culture as a vital element in the Union's international relations. 

133 COM(2007) 242 final 

41 

 
                                                           
The  Recommendation  was  an  important  action  for  the  Europe  2020  strategy  of  March 
2010, and in particular for its flagship initiative ’A Digital Agenda for Europe’, aiming to 
deliver  sustainable  economic  and  social  benefits  through  a  Digital  Single  Market. 
Providing active support for the digitisation of Europe's rich cultural heritage was one of 
its commitments, and it translated, among other  things, into the support  provided by the 
Recommendation. It built on the work of one of the stepping stones of digital for cultural 
heritage, i.e. Europeana, launched in 2008. 

As highlighted in the Commission’s Communication ‘Towards an integrated approach to 
cultural  heritage  for  Europe’134,  digital  tools  can  contribute  to  better  public  access  to 
different forms of cultural and linguistic content, while facilitating their preservation.  

However,  to  be  more  consistent  with  the  EU  policy  on  culture,  the  Recommendation 
should  further  address  key  aspects  for  cultural  heritage,  as  underlined  by  the 
Communication mentioned above, for instance the need to:  

  encourage  the  modernisation  of  the  heritage  sector,  raising  awareness  and  engaging 

new audiences ; 

  seize  the  opportunities  offered  by  digitisation;  to  reach  out  to  new  audiences  and 

engage young people in particular;                                                    
identify skills needs and improve the training of heritage professionals and 

 
  continue developing more participative interpretation and governance models that are 
better  suited  to  contemporary  Europe,  through  greater  involvement  of  the  private 
sector and civil society.   

The  Recommendation  is  consistent  with  the  New  European  Agenda  for  Culture135 
adopted in 2018, sharing the same objectives of the Agenda’s three dimensions, namely: 

  Social  dimension  -  harnessing  the  power  of  culture  and  cultural  diversity  for  social 
cohesion  and  well-being.  For  instance,  enhanced  digitisation  and  its  wider  access 
contributed to  the protection and promotion  of Europe's cultural  heritage  as a shared 
resource.  Particularly  through  Europeana  and  its  resources  and  initiatives,  it 
contributes  to  raising  awareness  of  our  common  history  and  values  and  reinforce  a 
sense of common European identity136. 

  Economic  dimension  -  supporting  culture-based  creativity 

in  education  and 
innovation,  and  for  jobs  and  growth.  For  instance,  through  Europeana  (see  Section 
5.1.on  effectiveness)  it  has  provided  further  opportunities  for  digital  teaching  and 
learning,  promoting  arts,  culture  and  creative  thinking  in  education  and  lifelong 
learning. 

  External  dimension  -  strengthening  international  cultural  relations  particularly 
through  Europeana  and  the  collaboration  with  other  countries.  Europeana  is,  for 
instance,  still  the  only  initiative  of  its  kind  with  the  largest  digital  collections  in  the 
world offering material in almost 40 languages.  

134 Communication from the Commission to the European Parliament, the Council, the European Economic 
and  Social  Committee  and  the  Committee  of  the  Regions,  Towards  an  integrated  approach  to  cultural 
heritage for Europe, COM (2014) 477 final, 22 July 2014.  
135 A New European Agenda for Culture, COM (2018) 267.  
136 See Section 4 of the New Agenda for Culture 

42 

 
                                                           
However, in order to be more aligned with the New Agenda, the Recommendation should, 
for example, further promote digital skills needed by the sector, strengthen the innovation 
capacity of creative sectors through a wider access to digitised resources. 

Moreover, the Recommendation has synergies with the Council Work Plan for Culture 
for 2019-2022137. In line with the strategic framework of the new Agenda for Culture, this 
Council  work  plan  acknowledges  major  developments  such  as  the  digital  shift  and  the 
changing working environments, as well as the need to deal with these opportunities and 
challenges through joint efforts in cultural policy. 

The  Recommendation  is  closely  aligned  with  European  Framework  for  Action  on 
Cultural Heritage, proposed for the first time in 2019, to capture and scale-up the success 
of the European Year of Cultural Heritage 2018138. For instance, the Recommendation 
shares  specific  objectives  with  the  Framework,  e.g.  engaging  the  wider  public  and 
removing barriers to access to cultural content. However, while the Framework considers 
the  tangible,  intangible  and  digital  dimensions  of  cultural  heritage  as  inseparable  and 
interconnected,  the  Recommendation  would  need  to  be  further  strengthened  to  address 
also intangible and digital-born cultural heritage. 

The Recommendation is aligned with the Digital Agenda for Europe139 and its successor 
the  Digital  Single  Market  strategy140,  which  highlight  the  opportunities  brought  by 
technological advancement to address societal challenges, i.e. digitising Europe's cultural 
heritage and making it available to society at large. The Recommendation’s purpose is to 
enhance  digitisation  and  to  make  digital  content  widely  available,  while  supporting 
preservation. Both  strategies  also  pointed to  the lack of digital  literacy  and skills,  which 
remains  a  challenge  for  the  cultural  sector  that  was  widely  acknowledged  during  the 
consultation activities and that the Recommendation currently does not address. However, 
the  Recommendation  would  need  to  be  aligned  with  the  priorities  of  the  new  digital 
strategy  ‘Shaping  Europe’s  Digital  Future’141  to  create  a  Europe  fit  for  the  digital  age 
and  with  the‘European  strategy  for  data142’  adopted  in  February  2020,  which  calls  for 
appropriate policies and investments to pool European data in key sectors, with EU-wide 
common and interoperable data spaces. 

The  Recommendation  is  closely  aligned  with  the  Declaration  of  cooperation  on 
advancing  digitisation  of  cultural  heritage143  signed  by  27  European  countries  and 
aimed  to  encourage  a  better  use  of  state-of-the-art  digital  technologies  for  cultural 
heritage,  enhancing  its  use  and  visibility,  while  improving  public  engagement.  The 

137 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52018XG1221(01) 
138 https://op.europa.eu/en/publication-detail/-/publication/5a9c3144-80f1-11e9-9f05-01aa75ed71a1 
139 Communication from the Commission to the European Parliament, the Council, the European Economic 
and Social Committee and the Committee of the Regions, ‘A Digital Agenda for Europe’, COM(2010)245 
final, 19 May 2010 
140 Communication from the Commission to the European Parliament, the Council, the European Economic 
and Social Committee and the Committee of the Regions, ‘A Digital Single Market Strategy for Europe’, 
COM(2015) 192 final, 6 May 2015 
141  Communication from the Commission to the European Parliament, the Council, the European Economic 
and Social Committee and the Committee of the Regions, ‘Shaping Europe's digital future’,  COM(2020) 67 
final, 19 February 2020 
142 Communication from the Commission to the European parliament, the Council, the European Economic 
and  Social  Committee  and  the  Committee  of  the  Regions    ‘A  European  strategy  for  data’,  COM/2020/66 
final, 19 February 2020  
143https://ec.europa.eu/digital-single-market/en/news/eu-member-states-sign-cooperate-digitising-cultural-
heritage 

43 

 
                                                           
synergies  between  the  Recommendation  and  the  Declaration  concern  for  instance  the 
reuse  of  digitised  cultural  resources  to  promote  citizen  engagement,  innovative  use  and 
spillover  to  other  sectors,  with  closer  cross-border  cooperation.  However,  while  the 
Declaration  recognises  capacity  building  as  one  of  the  key  challenges  that  has  to  be 
addressed, the Recommendation would need further alignment in this respect. 

The  Directive  on  copyright  and  related  rights  in  the  Digital  Single  Market144 
modernises the copyright framework and lays down provisions addressing the way CHIs 
operate in  the digital  environment. The Recommendation is  coherent  with  the Directive, 
sharing  some  of  its  main  objectives,  such  as  increased  cross-border  access  to  content 
online  and  wider  opportunities  to  use  copyrighted  material  in  education,  research  and 
cultural  heritage.  The  new  provisions  of  the  Directive  should  facilitate  CHIs  digitisation 
activities, use and reuse of digitised resources through a legal mechanism to facilitate the 
conclusion of collective licensing agreements for the digitisation and dissemination of out-
of-commerce works (e.g. books that publishers have stopped printing and selling) held by 
CHIs in their permanent collections. The origin of the out-of-commerce works provisions 
of  the  new  Copyright  Directive  goes  back  to  and  expands  on  the  non-binding  2011 
Memorandum  of  Understanding  on  Key  Principles  on  the  Digitisation  and  Making 
Available of Out-of-Commerce Works145, which the Recommendation directly refers to. 

Participants  in  the  Workshop  on  Digital  Transformation  in  Cultural  Heritage146,  along 
with  feedback  on  the  roadmap,  indicated  that  the  Recommendation  does  not  sufficiently 
address certain copyright issues that may prevent CHIs from promoting online access, use 
and  reuse  of  digitised  content.  However,  it  is  expected  that  the  Directive’s  transposition 
and implementation will address important copyright challenges.  

Likewise, the Recommendation is  aligned with  the  Orphan’s Works  Directive, sharing 
common  objectives  to  improve  digitisation  and  online  accessibility  of  in-copyright 
material and encourage its transposition and implementation.  

The Recommendation shares synergies with Directive (EU) 2019/1024 on open data and 
the reuse of public sector information147 whose transposition and implementation should 
remove remaining barriers and cater for a more efficient use and reuse of digital cultural 
resources.  

The Europeana strategy for 2015-2020 focused on improving data quality, opening the 
data and making it available without restriction for reuse, thus creating value for partners. 
The  provisions  of  the  Recommendation  encourage  the  achievement  of  similar/close 
objectives  as  well,  bringing  the  Recommendation  in  line  with  the  strategy.  The  main 
priority of the new  Europeana strategy for 2020-2025 is  to  enhance CHI’s  capacity  to 
seize  the  opportunities  and  how  best  to  respond  to  the  numerous  challenges  opened  by 
digital  transformation.  As  suggested  in  most  of  the  consultation  activities,  the 
Recommendation should be revised in  order to  better  support CHIs to  make the most  of 
digital technologies to record, preserve, document and share cultural heritage online. This 

144 Directive (EU) 2019/790 of the European Parliament and of the Council of April 2019 on copyright and 
related rights  in the Digital Single Market and amending  Directives 96/9/EC and 2001/29/EC,  OJ L 130, 
17.5.2019, p. 92–125 
145 http://www.eblida.org/Experts%20Groups%20papers/EGIL-papers/MoU-OOC.pdf 

146 Workshop Brussels (January 2020) 
147Directive (EU) 2019/1024 of the European Parliament and of the Council of 20 June 2019 on open data 
and the re-use of public sector information, OJ L 172, 26.6.2019, p. 56–83 

44 

 
                                                           
 
implies addressing in a more effective and efficient manner emerging needs, such as the 
need  to  build  capacity.  Indeed,  advancing  digital  literacy  and  skills  has  emerged  as  a 
recurrent topic raised by the professionals of the sector, while underlining the importance 
of  a  network  of  supporting  competence  centres.  Moreover,  a  stronger  emphasis  on  the 
quality of data and metadata, by using new/advanced technologies like machine-learning, 
would increase coherence with the Europeana strategy for 2020-2025.  

Feedback  from  the  representatives  of  the  Member  States  to  the  DCHE  underlines  the 
Recommendation’s coherence with and complementarity to cultural heritage policies in 
the Member States. The various actions carried out by the Member States as a follow-up 
to  the  Recommendation  confirm  the  coherence  between  the  Recommendation  and  their 
national  policies.  The  digitisation  policies  put  in  place  by  the  Member  States  vary  to  a 
large  extent  according  to  their  priorities,  resources  and  coordination  mechanisms. 
However,  although  there  is  no  size  fits  all,  for  the  2015-2017  reporting  period,  most 
Member  States  had  national  strategies/plans  and  national  funding  programmes  in  place, 
which  corresponds  to  the  Recommendation’s  priorities.  Likewise,  the  digitisation  of 
library  and  archival  documents  was  one  of  the  main  digitisation  priorities,  including 
manuscripts,  books,  journals  and  historical  newspapers,  underlining  the  coherence 
between the Recommendation and the national policies in the field. 

As  highlighted  in  the  Europeana  evaluation148,  the  initiative  is  in  close  alignment  with 
Member  States’  national  policies  on  digitisation  and  the  online  accessibility  of  cultural 
heritage  material.  90%  of  the  respondents  indicated  some  degree  of  complementarity 
between  Europeana  and  national  initiatives  (such  as  Gallica,  Hispana  or  the  German 
Digital  Library), highlighting in particular Europeana’s role as the only organisation that 
provides  access  to  cultural  material  across  borders  in  Europe.  Moreover,  61  %  also 
indicated some degree of overlap with these. As shown by the national  reports, Member 
States  have  stepped  up  their  support  to  the  Europeana  ecosystem,  through  national  and 
regional  aggregators;  national  and  regional  aggregation  initiatives  have  increased  in 
importance  at  national  level  and  for  Europeana,  while  participation  in  European 
aggregation projects has expanded149. 

5.5. EU ADDED VALUE 

This  section  assesses  the  benefits  stemming  from  the  intervention,  more  particularly  the 
extent  to  which  it  a)  generated  an  increase  in  the  digitisation  of  and  online  access  to 
cultural  material,  as  well  as  in  the  actions  taken  for  digital  preservation  (beyond  what 
Member States would have achieved alone); b) led Member States to broaden the types of 
measures  implemented  to  promote  the  digitisation  of  and  online  access  to  cultural 
material;  c)  promoted  innovation  and  the  transfer  of  ideas  across  Member  States  in  this 
field. 

Key points on the baseline: 

  Efforts to digitise cultural heritage content were in their initial stages. 

148 Evaluation of Europeana and orientations for its future development, following adoption of Council 
Conclusions by EYCS Council on 31/05/2016 
149 See Consolidated Report 2018 

45 

 
 
                                                           
  It  was  mainly  due  to  the  CHIs  ’own  resources  for  financing  digitisation,  as  very 

often governments did not allocate a national budget/resources. 

 Key findings on the current situation: 

  The  Recommendation  had  a  clear  added  value  in  increasing  digitisation  (e.g. 
quantitative  targets  were  exceeded)  and  online  access  to  cultural  heritage 
resources, as acknowledged in the context of the consultation activities. 

  It  significantly  contributed  to  raise  awareness  within  and  beyond  the  sector,  in 
promoting  national/regional  and  cross-border  collaboration,  exchanges  of 
experience, knowledge, best practice and useful tools among the Member States. 

  The Recommendation provided a forum  for discussions and cooperation between 
Member  States  and  between  CHIs,  leading  to  increased  visibility  and  funding 
opportunities for the latter. 

A  large  majority  of  the  respondents  (80%)  to  the  online  public  consultation  provided  a 
positive feedback about  the added-value of the Recommendation compared to  initiatives 
that Member States would have taken in the absence of it.  

As  shown  in  the  previous  sections,  there  was  a  clear  increase,  supported  by  the 
Recommendation,  in  the  digitisation  of  and  access  to  digitised  cultural  resources.  The 
quantitative  targets  encouraged  by  the  Recommendation  were  surpassed  earlier  than 
planned.  The  Council  Conclusions  of  31  May  2016  highlighted  not  only  the  progress 
towards the digitisation, online accessibility  and  digital  preservation of  cultural  heritage, 
but also the joint efforts of CHIs, Member States and the Commission150. This would have 
been less likely without the Recommendation151. 

The  Recommendation  had  a  significant  contribution  in  raising  awareness  within  and 
beyond the sector, in strengthening exchanges of experience, knowledge, best practice and 
tools among the Member States. It urged them to take action and track progress in order to 
increase  the  digitisation  and  online  accessibility  of  European  cultural  heritage  and  to 
improve  their  national  policies/strategies  or  strategic  planning.  For  instance,  it  was  an 
important  source  of  inspiration  for  the  development  of  the  National  Digital  Library  in 
Finland.  It  stimulated  greater  harmonisation  and  deployment  of  standards,  better 
interoperability and preservation requirements. 

Furthermore,  involving  the  private  sector  in  digitising  cultural  material  led  to  successful 
and  innovative  partnerships.  For  example,  in  Austria,  the  digitisation  of  ‘Cinema  in  the 
Austro-Hungarian Monarchy’ was supported in 2016 by the private cultural institution the 
Klimt-Foundation152.  

The provisions of the Recommendations on optimising the use of digitisation capacity and 
achieving  economies  of  scale  appeared  to  be  important  for  all  Member  States  and  to  be 
applied  to  a  large  extent.  Joint  efforts  for  national  or  cross-border  collaborations 
competence centres or sharing services and facilities such as repositories or IT tools were 

150 Council Conclusions of 31 May 2016 on the role of Europeana for the digital access, visibility and use of 
European cultural heritage 
151 See also feedback on the roadmap 
152 See national report (2015-2017) provided by Austria  

46 

 
                                                           
deployed in  order to  make digitisation related activities more effective and efficient. An 
example in this respect was the Estonian experience of cross-border collaborations sharing 
best  practices  and  establishing  uniform  standards  for  mass  digitisation  (see  box  4). 
Without  the Recommendation, it is  less likely that  this cross-border collaboration would 
have taken place to this extent. 

Box 4.  

  Swedish-Baltic seminar on text-mining services and tools that libraries should 

develop to serve digital humanities with their digital collections (2017). 

 

 Lithuanian, Latvian, Estonian summer seminar of Baltic National Libraries (2016) 
on how to ensure collections stay relevant to researchers. 

  Lithuanian, Latvian, Estonian joint museology course (every year) dealt, for 

instance, with issues concerning the collection, organisation and use of museum 
related information. 

 

International project "Collaborative Digitisation of Natural and Cultural Heritage: 
CD-ETA” (Interreg), aiming to improve adoption of the digitisation policy for 
natural and cultural heritage and contributing to the establishment of uniform 
standards in the mass digitisation 

As highlighted in the evaluation study, the promotion of coordinated approaches on legal 
deposit  arrangements  is  a  part  of  the  scope  effect  of  the  Recommendation.  This  led  to 
increased efforts enabling CHIs to access a wide variety of materials.  

With  respect  to  Europeana,  the  Europeana  evaluation  report  highlights  an  overall 
appreciation of the European added-value of the Europeana initiative by cultural heritage 
institutions  across  the  EU,  bringing  them  together  and  enabling  them  to  collaborate  and 
share  their  material.  This  has  not  only  led  to  the  sharing  of  best  practices  on  common 
standards,  but  also  helped  nurture  a  European  network  of  data  partners,  aggregators  and 
professionals in various fields which has stimulated capacity building and the exchange of 
expertise.  

In addition, the findings of the workshop of January 2020 point to the  need to maximise 
the impact of Europeana as an enabler of digital transformation in the cultural heritage 
sector.  This  could  be  the  case  for  instance  by  using  the  multiplier  effect,  developing 
national strategies or making public funding conditional on the adoption of standards. This 
would give the Recommendation more added value. 

Also, the findings of the DCHE workshop of November 2019, together with the targeted 
consultation of the representatives of the Member States in the DCHE, confirmed that the 
contribution  of  the  Recommendation  in  strengthening  the  role  of  Europeana  and  more 
generally in the increase of digitisation, online access and preservation of digitised cultural 
material.  It  is  important  to  note  that  87%  of  the  respondents  to  the  online  public 
consultation considered the EU’s support to Europeana as (very) important. 

47 

 
 
The reporting system laid down in the Recommendation has been an important means to 
create a structured process for its implementation (e.g. to incentivise the focus on priorities 
and to collect information), in addition to giving CHIs and authorities the opportunity to 
provide feedback. 

Finally,  according  to  the  majority  of  the  respondents153  (98%)  to  the  online  public 
consultation,  the  EU  and  Member  States  should  do  more  to  help  cultural  heritage 
institutions  address  the  challenges  and  seize  the  opportunities  of  the  digital  era.  A  large 
majority  (around  49%  strongly  agree  and  32%  agree)  considered  that  the  digital 
transformation  of  European  cultural  heritage  institutions  should  be  addressed  and 
coordinated at the EU level.  

6.  CONCLUSIONS 

This section provides the conclusions of the evaluation, highlighting what is working and 
not working with the EU intervention and why, and what would be the most appropriate 
future course of action concerning the Recommendation, i.e. whether the identified issues 
need  to  be  addressed  by  action  at  EU  level  or  can  be  resolved  by  action  carried  out  by 
Member States/CHIs over time. 

While its implementation by Member States varies to a large extent, the Recommendation 
led to an overall improvement in the field, as acknowledged by the various reporting and 
consultation activities. This stems, for instance, from i) the adoption of national strategies 
and/or funding programmes for the digitisation of cultural heritage assets; ii) the increased 
number  of  PPPs;  iii)  greater  visibility  and  closer  cooperation  among  CHIs;  iv)  practical 
measures to optimise the use of digitisation capacity; v) increased volume of the digitised 
cultural  heritage  resources  and 
impact  of  Europeana.  The 
Recommendation’s  objectives  have  been  reached  and  remain  relevant  to  a  large  extent. 
Several factors contributed to this, such as tools designed to support CHIs or technological 
advancements  that  promote  open-source  software  and  shared  data,  while  other  factors 
hindered  the  achievement  of  the  objectives  (e.g.  lack  of  adoption  of  common  standards 
and approaches, insufficient advanced digital skills and literacy). 

the  positive  overall 

Despite  insufficient  data  being  available  to  quantify  the  costs  triggered  by  the 
Recommendation’s implementation and to make a comparison across the Member States, 
stakeholders were positive about the intervention’s efficiency, considering that overall the 
benefits outweigh the costs. 

In terms of relevance, there is a general acknowledgement that the Recommendation was 
useful in responding to the sector’s needs, as identified at the time of its adoption. 

Likewise, the Recommendation has proved to be coherent not only with other initiatives at 
the EU level, such as the initiatives on copyright or more generally on culture and cultural 
heritage, but also with the actions and objectives that Member States promoted at national 
level. 

153   A majority was reached also across stakeholder groups and sectors. 

48 

 
                                                           
The  Recommendation’s  added  value  (e.g.  of  increasing  digitisation  and  online  access  to 
digital  resources,  of  encouraging  closer  cooperation  and  promoting  innovation  and  the 
transfer of ideas across the EU) was also highlighted during the consultation activities. 

Nonetheless,  in  order  to  increase  the  intervention’s  effectiveness,  efficiency,  relevance, 
coherence and added value, the evaluation process has pointed to a number of important 
aspects for the future course of action.  

The  cultural  heritage  landscape  has  changed  significantly  over  the  past  years,  bringing 
new  needs,  but  also  new  opportunities.  In  particular,  new  political,  technological  and 
socio-economic  parameters  have  brought  about  new  challenges.  Moreover,  in  a  general 
context  where the  entire world  is  struggling to  overcome an unprecedented crisis  and to 
define robust policies to recover and rebuild our economies, our society turns more than 
ever to the cultural heritage sector for its rich resources for a variety of purposes. A clear 
message that emerged from the consultation activities was that while the sector needs to 
turn the crisis  into an opportunity for more  growth  and social  cohesion, it  must also  be 
equipped with the right tools to do so. More particularly, CHIs need to be powered by 
digital technologies, embrace the digital transformation and thus lead the change.  

The COVID-19 crisis has underlined once more the need for and importance of improving 
online access to digitised cultural material and improving its use/reuse; this is true for 
the CHIs, for the creative sectors and for society at large. The crisis has been a catalyst for 
more online activities and services, but has also led to some inequalities within the sector 
and  thus  more  missed  digital  opportunities.  Funding,  a  persistent  challenge  for  the 
cultural heritage sector before the crisis, has become more stringent in the current context 
for many CHIs and even more vital to their survival. 

Advanced  or  emerging  digital  technologies  can  empower  the  sector  to  unlock  the 
potential  of  the  cultural  heritage  sector  through  a  more  widespread  adoption  of  3D, 
AR/VR,  AI,  data,  etc.,  while  providing  new  ways  for  cultural  objects  to  be  shared  and 
used. They allow visitors to visit different and new places regardless of their location and 
cost,  they  offer  a  more  appealing  and  engaging  experience.  They  support  our  cultural 
heritage institutions in narrating European stories154.  

Higher quality of the digitised cultural heritage resources would enhance innovation and 
creation  through  the  use  and  reuse  of  digitised  cultural  material  across  the  sectors.  The 
current  focus  mostly on increasing the quantity of the digitised assets has to  go together 
with higher quality.  

Further  adoption  of  common  standards  and  approaches  for  digitised  content  would 
remove  some  of  the  remaining  interoperability  related  obstacles,  while  facilitating 
cooperation  and  exchanges  that  strengthen  the  capacity  for  innovation.  Europeana  has 
been  key  in  strengthening  cooperation  and  standardisation  activities  across  borders  with 
the overarching goal of increasing the use of standards in digitising and sharing digitised 
cultural  material  throughout  Europe.  The  initiative’s  involvement  has  also  been  vital  for 

154 As underlined in the feedback to the public consultation 

49 

 
                                                           
promoting  open  cultural  data  and  it  should  continue  to  push  forward  standardisation 
activities to facilitate the work of the cultural heritage institutions.  

At the same time, Europeana has to play a more prominent role in enabling CHIs to seize 
the  opportunities  provided  by  the  digital  transformation  in  the  cultural  heritage  sector. 
Europeana’s  three  strategic  priorities  are  to  strengthen  its  infrastructure,  improve  the 
quality  of  data  and  content  and  support  the  cultural  heritage  sector  to  engage  in  digital 
transformation.  The  results  of  the  latest  consultation  activities  have  confirmed  these  and 
particularly the role of the initiative in supporting cultural heritage institutions to embrace 
digital.  

Broadening  the  scope  of  the  Recommendation  to  cover  key  cultural  heritage  marginally 
addressed or not addressed at all yet, like immovable, born digital and intangible, would 
create a more complete framework of action for the CHIs and would be in line with the 
feedback from the online public consultation. 

Advanced  digital  literacy  and  skills  have  become  paramount  for  CHIs,  in  a  context 
where  the  COVID-19  pandemic  speeded  up  the  importance  of  finding  new  ways  of 
working and connecting to audiences and sharing creative content online. It has also been 
a recurrent issue raised in the consultation activities. 

Making  full  use  of  cultural  heritage  data  for  the  benefit  of  the  sector  by  for  instance 
promoting national strategies and cross-sectoral collaboration that encourage the creation 
of  high  value  datasets  would  be  beneficial  not  only  to  the  cultural  sector,  but  to  other 
sectors  as  well  (e.g.  research,  sustainable  tourism,  education)  and  would  contribute  to 
building the data space for cultural heritage.  

European  efforts  to  further  digitise  and  broaden  online  access  of  cultural  heritage  have 
been recognised at international level, be it through the acknowledgement of the Member 
States Declaration of 2019 by UNESCO or through the work of the Europeana initiative 
with international organisations, such as the World Digital Library or the Digital Library 
of  America.  Further  partnerships  in  the  field  would  help  in  finding  shared  responses  to 
common challenges and exchanging best practices as well as showcasing and promoting 
European culture, values and success stories. Europeana is, as mentioned above, a unique 
initiative offering material in many languages. It has a lot to offer in terms of experience 
and knowledge.  

Some of the aspects above may be further addressed by the Member States alone, although 
it is not certain to what extent, especially in the light of the wide-ranging impacts of the 
COVID-19  crisis  and thus  of the declining budgets  assigned to   cultural  heritage-related 
activities.  However,  the  evaluation  analysis  (including  the  outcome  of  the  consultation 
activities)  points  to  a  strong  impact  of  an  EU  intervention  supporting  the  efforts  of  the 
Member States. 

50 

 
 
 
 
Annex 1: Procedural information 

1.  LEAD DG, DECIDE PLANNING/CWP REFERENCES 

Directorate General Communication Networks, Content and Technology (CNECT) is the 
lead DG for this evaluation, and in particular Unit G2 – Interactive Technologies, Digital 
for Culture and Education.  

The DECIDE reference is: PLAN/2019/5584. 

2.  ORGANISATION AND TIMING 

In  accordance  with  the  Better  Regulation  Guidelines,  an  ISG  was  set  up  with 
representatives from various Directorates General and services of the Commission155. 

The  meetings  of  the  steering  group  were  chaired  by  DG  Communications  Network, 
Content and Technology. The first meeting took place on 17 September 2019 and laid the 
ground for the work related to the evaluation. The ISG was regularly consulted over the 
course  of  the  evaluation,  typically  in  conjunction  with  the  submission  of  specific  draft 
reports by the contractor responsible for carrying out the external study. The ISG was also 
consulted during the drafting of this staff working document, which includes the various 
remarks made. 

The evaluation was extended, given the fact that the public consultation was launched later 
than  initially  planned,  due  mostly  to  the  COVID-19  crisis  (the  public  consultation  was 
online from 22.06.2020 to 14.09.2020). 

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

No exceptions from the usual procedure laid down in the Better Regulation Guidelines 
requirements were requested/made. 

4.  CONSULTATION OF THE RSB:  

Not applicable. 

5.  EVIDENCE, SOURCES AND QUALITY 

Most of the evidence was collected with the support of an external contractor (see “Study 
to  support  the  evaluation  and  possible  revision  of  the  Commission  Recommendation 
(2011/711/EU) on the digitisation and online accessibility of cultural material and digital 
preservation, conducted by ICF, Final Report, March 2020).  

155 SG, EAC, DIGIT, RTD, Ref. Ares(2019)4357448 - 08/07/2019 

51 

 
 
 
                                                           
The  evidence  is  based  on  external  expertise  and  knowledge  from  Member  States’ 
international  and  umbrella 
competent  authorities,  cultural  heritage 
organisations,  the  Europeana  ecosystem  of  organisations  and  professionals,  technology 
stakeholders  with  research,  products  and  services  in  ICT  for  cultural  heritage,  citizens, 
through the numerous consultation activities held by the Commission (see Annex 2).  

institutions, 

However, the quantification of the costs and benefits of the intervention was difficult, due 
to the diffuse and intangible nature of the benefits of cultural heritage, and to the lack of 
available or updated data on costs in Member States (see Section 4.3). Alternative proxy 
data  were  used  and  explicitly  mentioned  in  the  analysis  where  quantitative  data  are 
missing. 

The overall evaluation is considered robust and thorough. 

52 

 
 
Annex 2: Synopsis report of the consultation activities 

1.  INTRODUCTION 

The Commission Recommendation on the digitisation and online accessibility of cultural 
material  and  digital  preservation  (2011/711/EU)156  (“the  Recommendation”)  is  the 
Commission’s  main  policy  tool  in  the  area  of  digital  for  cultural  heritage,  aiming  to 
optimise  the  use  of  information  and  communication  technologies  to  unlock  the  full 
economic and cultural potential of Europe’s cultural heritage.  

The  Recommendation  dates  back  to  October  2011.  Some  of  the  challenges  facing  the 
cultural heritage sector at that time (e.g. an urgent need to protect and preserve European 
cultural  heritage  in  danger),  are  still  present  today,  but  there  are  also  others  that  have 
arisen  since  (e.g.  major  advances  in  digital  technologies).  Moreover,  the  COVID-19 
pandemic confirmed once more the importance of enabling the cultural heritage sector to 
seize the opportunities provided by digital technologies.  

It was therefore necessary to assess the Recommendation and decide whether it still met 
the needs and expectations of the cultural heritage sector, of the users, creative sectors, 
the 
citizens  and  society 
Recommendation took place in order to support the Commission’s decisions on the next 
steps,  shaping  an  appropriate  policy  framework  to  preserve  Europe’s  valuable  cultural 
assets.  

in  general.  A  clear,  evidence-based  assessment  of 

To this end, a wide stakeholders’ consultation on digitisation and digital transformation 
in the cultural heritage sector was necessary.  

The following consultation activities were organised:  

1.  Feedback on the evaluation roadmap  
2.  Several workshops 
3.  Targeted  consultation  of  the  Expert  Group  on  Digital  Cultural  Heritage  and 

Europeana (DCHE) representatives 

4.  Delphi panel  
5.  Open Public Consultation  

The feedback received from the consultation activities, in particular on the effectiveness, 
efficiency,  relevance,  coherence,  EU-added  value  of  the  Recommendation,  has  been 
integrated  in  the  evaluation,    while    various  key  issues  raised  by  stakeholders  will  be 
considered for the future policy framework (i.e. for the revision of the Recommendation).   

2.  CONSULTATION PROCESS AND ANALYSIS OF RESPONSES 

2.1. Feedback on the evaluation roadmap 

An  evaluation  roadmap  was  published  for  a  four-week  feedback  period  (from  29  July 
2019  to  26  August  2019).  It  aimed  to  inform  citizens  and  stakeholders  about  the 
Commission’s work, encouraging them to provide feedback and to participate effectively 

156 OJ L 283, 29.10.2011, p. 39–45 

53 

 
 
                                                           
in  consultation  activities.  The  evaluation  roadmap  and  the  feedback  from  stakeholders 
can be consulted on the “Have your Say” page of the initiative. 

The  evaluation  roadmap  aimed  to  inform  citizens  and  stakeholders  about  the 
Commission’s work, encouraging them to provide feedback and to participate effectively 
in consultation activities.  

A  total  of  eight  responses  were  received  from  a  variety  of  stakeholders  (e.g.  business 
organisations, non-governmental organisations, public authorities, citizens). 

Respondents acknowledged the positive impact of the Recommendation so far, including 
the  significant  changes  and  benefits  for  the  cultural  heritage  sector.  Stakeholders 
welcomed  the  evaluation  of  the  Recommendation  in  the  current  context,  defined  by  a 
rapid progress of advanced digital technologies. 

Several stakeholders also mentioned bottlenecks and issues to be addressed in the future, 
such  as  quality  of  metadata,  infrastructure  to  disseminate  digitised  material,  copyright 
and licensing issues. 

2.2. Workshops 

a)  Two  workshops  were  organised  in  the  context  of  the  6th  Meeting  of  the  Expert 
Group  on  Digital  Cultural  Heritage  and  Europeana  (5-6  November  2019, 
Luxembourg).  The  first  one  specifically  focused  on  both  backward  (impact/functioning 
of the Recommendation) and forward-looking aspects (i.e. areas for improvement). 

According to the participants, the Recommendation has contributed to raising awareness 
about  the  cultural  heritage  sector,  to  facilitating  more  specific  policies,  strategies  and 
projects  at  national  and  local  levels.  The  Recommendation  has  contributed  to  more 
cooperation,  openness  and  sharing,  to  more  interoperability,  while  strengthening  the 
development of Europeana and its activities. 

The respondents also underlined the need to address specific areas, such as better quality 
of  data  and  metadata,  enhanced  interoperability,  more  effective  digital  preservation 
policies, and enhanced capacity building. Moreover, the Recommendation should have a 
more holistic approach and scope and cover other categories of cultural heritage, such as 
born-digital cultural heritage. The importance of Europeana has been highlighted, as well 
as the need to maximise its role as enabler of digital transformation.  

The second workshop, more focused on the Declaration of Cooperation on advancing the 
digitisation of cultural heritage157, called for views on 3D documentation, 3D repository, 
technology  take-up  and  capacity  building  and  the  instruments  contributing  to  those. 
Respondents considered building on existing standards and/or processes important, while 
the  elaboration  of  common  standards  should  incorporate  the  fundamental  principles  of 
usability  and  openness.  The  need  for  advanced  digital  skills  and  literacy  has  also  been 
highlighted, along with the need for increased knowledge sharing. Participants pointed to 
the need for European and national  funding to encourage and stimulate the adoption of 
advanced  digital  technologies.  Cooperation  at  international  level  has  been  considered 
important, both from a content and a technology point of view. 

157 https://ec.europa.eu/digital-single-market/en/news/eu-member-states-sign-cooperate-digitising-cultural-
heritage 

54 

 
                                                           
 
b)  Workshop  in  the  context  of  the  Europeana  Conference  (28  November  2019, 
Lisbon) 

The workshop took place during the Europeana Conference in  Lisbon on 28 November 
2019 and explored similar topics to the first two workshops, but delving more in-depth 
based on the responses and gaps identified in the first workshops.  

Participants pointed to the fact that the low levels of digital literacy and skills remain a 
challenge,  particularly  for  galleries,  libraries,  archives,  and  museums  (GLAMs),  while 
retaining  expert  staff  is  still  difficult  for  cultural  heritage  institutions.  The  sector  needs 
more capacity building and Europeana should have an enhanced role in this, by sharing 
good  practices  and  expertise.  Cultural  heritage  institutions  have  been  struggling  to 
digitise and preserve particularly intangible cultural heritage. Participants mentioned the 
challenge  of  ensuring  better  interoperability  in  the  sector  (for  instance  between  various 
formats used and Europeana Data Model). Cultural heritage institutions need more robust 
and sustainable funding, the smaller ones in particular. Quality of digitisation should also 
become a priority, contrarily to the current approach which focused mainly on quantity. It 
was  emphasised  that  digital  transformation  requires  coordinated  efforts,  developing 
holistic  approaches  at  all  levels  and  a  stronger  collaboration  between  institutions. 
Europeana  should  have  an  enhanced  role  in  the  context  of  digital  transformation, 
including  by  promoting  further  standardisation,  a  multilingual  access,  exploring 
crowdsourcing  opportunities  and  securing  alternate  funding  streams,  while  building 
bridges in the sector. 

c) Workshop with the international organisations on digital for cultural heritage (10 
December 2019, Brussels) 

Twelve representatives of various international organisations and networks took part in a 
workshop  on the orientations and priorities for EU funding,  activities and policy in  the 
area of digital for cultural heritage (including on the Recommendation).  

Participants raised the need for enhanced capacity building, to ensure better training and 
knowledge  of  staff  (including  to  enable  them  to  run  digitisation  projects  and  evaluate 
what can be reused), for increased cross-border digitisation efforts. Further development 
of common standards (relating to 3D, metadata enrichment, multilingual access) was also 
mentioned. The use of advanced technologies would maximise the impact needed in the 
sector. The quality of content should be an important part of any policy framework. 

The  workshop  emphasised  that  the  Recommendation  would  be  more  effective  if  it 
strengthened  the  networks  of  the  cultural  heritage  institutions  more,  enhanced  the 
knowledge  exchange  and  best  practice,  including  by  an  increased  collaboration  at 
international  level  (e.g.  by  providing  more  support  for  transfer  of  technology, 
development of skills in third countries). 

d)  Workshop  with  stakeholders  on  digital  transformation  in  the  cultural  heritage 
sector (10 January 2020, Brussels) 

Various  stakeholders  (cultural  heritage  institutions,  international  organisations  in  the 
field of culture, research organisations, etc.) took part in this workshop, building on the 
information collected during previous workshops.   

According to the feedback provided by stakeholders, more needs to be done to advance 
digital  literacy and skills  in  the cultural heritage  sector. The Recommendation does not 

55 

 
sufficiently  address  the  need  to  strike  a  balance  between  quality  and  quantity  of  the 
digitised  material,  while  the  sector  needs  to  move  forward  from  an  approach  focused 
mostly on quantity. 

Stakeholders highlighted that the current Recommendation does not sufficiently address 
the need to make full use of cultural heritage data for the benefit of the cultural heritage 
sector and other areas. There is room for improvement to increase the collection, access, 
use  and  reuse  of  cultural  heritage  data.  Data  sharing  among  cultural  heritage 
organisations should be further enhanced. 

With  regard  to  Europeana,  stakeholders  underlined  its  positive  impact  on  the  cultural 
heritage  institutions.  Further  partnerships  and  cooperation  at  international  level  should 
help to find shared responses to common challenges and exchange best practices, while 
promoting European culture, values and success stories. 

2.3.  Targeted  consultation  of  the  Expert  Group  on  Digital  Cultural  Heritage  and 
Europeana representatives  

A  targeted  consultation  of  the  Expert  Group  representatives  was  organised  between 
November 2019 and January 2020, to complement previous consultations and to build a 
coherent and cohesive view of both the Recommendation’s implementation and possible 
options for the future. 

A total of 20 respondents, from 18 Member States, completed the targeted consultation. 

The targeted consultation suggested that the Recommendation had been overall effective, 
for instance in helping to raise additional funds for digitisation. Respondents agreed that 
the  Recommendation  had  been  relevant  for  the  needs  identified  originally,  while  also 
widely acknowledging the changes in the context in which cultural heritage institutions 
carry out their activities now. This is mainly due to technological advancements, which 
need to be better reflected in a future revision of the Recommendation.  

The Expert Group representatives emphasised the importance of increasing the quality of 
digitised cultural  material,  of providing common (comparable, interoperable) standards, 
strengthening  the  cooperation  among  cultural  heritage  institutions  and  with  private 
sector,  having  wider  online  access  to  digitised  cultural  material,  for  use  and  reuse, 
reinforcing  national  strategies  for  long-term  preservation  of  digital  material,  updating 
action plans implementing the strategies.  

2.4. Delphi panel 

An  online  Delphi  panel  composed  of  experts  from  the  Europeana  ecosystem,  cultural 
heritage  institutions,  research/academia,  Member  States  was  launched  to  support  and 
validate the identification of challenges/problems in the sector, and the most appropriate 
ways to respond to those.  

The panel considered several scenarios for the next steps, such as the situation where no 
policy  change  would 
the 
Recommendation;  updating  specific  provisions  and  expanding  the  scope  of  cultural 
material covered by the Recommendation; updating provisions, expanding the scope and 
addressing the digital transformation of cultural heritage institutions. The outcome of this 
panel  was  that  a  more  complex  policy  initiative  that  would  update  the  relevant 

the  specific  provisions 

take  place;  updating 

in 

56 

 
provisions,  expand  the  scope  covered  so  far  and  address  digital  transformation  would 
have higher (positive) impact, including in terms of efficiency and effectiveness. 

2.5. Open Public Consultation  

The public consultation opened 22 June 2020 and closed on 14 September 2020. The aim 
of  the  consultation  was  to  gather  stakeholders’  opinions  on  digitisation  and  digital 
transformation  in  the  cultural  heritage  sector,  as  well  as  on  the  Commission’s 
Recommendation  on  the  digitisation  and  online  accessibility  of  cultural  material  and 
digital  preservation  (2011/711/EU).  The  consultation  collected  views  from  citizens, 
public  administrations,  cultural  heritage  institutions,  international  organisations  and 
networks, the Europeana ecosystem, companies, research organisations, and academia.  

The questions covered in the consultation focused on:  

a)  the  importance  of  digitisation  and  digital  transformation  in  the  cultural  heritage 

sector and the ways of supporting such processes. 

b) the relevance and future of the Recommendation. 

Who participated to the consultation? 

There were 565 respondents, including 186 citizens, 145 academic/research institutions, 
68  public  authorities,  53  companies/business  organisations,  37  NGOs,  and  75  other 
interested parties.   

Error! Reference source not found. Respondents by stakeholder group.  

Citizens
Academic/research institution
Other interested parties
Public authority
Company/business organisation
NGOs

75 

68 

53 

37 

186 

145 

0

20

40

60

80

100 120 140 160 180 200

Number of respondents 

Source: Open public consultation on digital access to European cultural heritage. N=564 

About  half  of  the  respondents  indicated  that  they  are  active  in  the  culture  sector  (261 
responses),  followed  by  education  or  research  (169),  Technology/ICT  (60),  Creative 
arts/media  (29),  Tourism  (3).  42  respondents  indicated  that  they  are  active  in  another 
sector. 

57 

 
 
 
 
Figure 2. Q2. In which sector are you or your organisation active? 

Culture

Education or research

Technology/ICT

60 

Other (please specify)

42 

Creative arts/media

29 

Tourism

3 

261 

169 

0

50

100

150

200

250

300

Number of respondents 

Source: Open public consultation on digital access to European cultural heritage. N=564 

Cultural heritage sector in the context of the COVID-19 crisis 

The crisis affected organisations mostly through loss of income. While the majority did 
not have to dismiss staff, many of them had to suspend contracts with freelance workers.  

The tourism sector appears to have been most severely hit by the crisis, with almost two 
thirds of respondents referring to dismissal of staff. On the other hand, results by types of 
stakeholders show that the public sector was the least affected by the crisis in relation to 
the staff dismissed. 

Almost  half  of  the  respondents  noted  that  they  struggled  with  advanced  digital  skills 
needed to manage the online presence, particularly in the education/research and cultural 
sectors.  The  pandemic  had  a  significant  impact  on  the  digital  activities  of  the 
respondents.  Many  of  them  changed  their  online  offer  and  made  more  use  of  digital 
technologies.  This  resulted  in  an  extended  online  access  to  curated  audio-visual  assets, 
virtual  tours  of  exhibitions,  more  academic  research  material  (e.g.  journals,  books), 
increased  social  media  activities,  online  collaboration  tools  (e.g.  for  workshops, 
conferences)  and  e-learning  materials.  According  to  the  respondents,  the  majority  of 
these online offers/services will remain in place after the pandemic and will most likely 
be expanded. 

Based  on  lessons  learned  from  the  COVID-19  crisis,  most  of  respondents  (98%) 
considered  digital  technologies  as  (very)  important  for  making  their  organisation  more 
resilient in the future (as shown in figure 3).  

Figure 3. Question 7.  Based on the lessons learned from the COVID-19 crisis, how important do 

you think digital technologies will be in order to make your organisation more 
resilient in the future? 

100,00%

80,00%

60,00%

40,00%

20,00%

0,00%

80% 

Very important

Important

Not very important

I don't know/no opinion

18% 

2% 

1% 

Total

58 

 
 
 
 
Source: Open public consultation on digital access to European cultural heritage N=557 

Moreover, the majority  are  already making use  of digital  technologies to overcome the 
difficulties posed by the COVID-19 related measures. However, the respondents pointed 
out  that  the  use  of  these  technologies  mostly  depends,  in  many  cases,  on  the  funding 
available.  

Digitising cultural heritage 

There  was  a  strong  agreement  on  the  importance  of  digital  technologies  for  cultural 
heritage. Overall, a majority of respondents strongly agreed (67%) or agreed (27%) that 
digital  technologies  can  help  in  the  reconstruction  of  damaged  cultural  heritage,  97% 
agreed  that  digital  technologies  can  help  to  increase  the  number  of  people  accessing 
cultural  heritage  through  the  Internet,  while  for  98%  digitisation,  online  access  and 
digital  preservation  of  cultural  heritage  are  important  to  society.  There  were  no  major 
differences identified by types of stakeholders. 

A large majority of respondents considered 3D technologies valuable for creating digital 
twins  of  cultural  heritage  of  buildings,  monuments  and  sites  (92%),  for  creating  high-
quality 3D models of museum objects (91%), for creating 3D models of cultural heritage 
buildings,  monuments  and  sites  or  museum  object  for  online  visualisation  (93%)  or 
immersive experiences (88%). 

According  to  certain  respondents,  additional  areas  where  the  application  of  3D 
technologies would be valuable are education sector (to facilitate collaborative research 
e.g.  virtual  archaeology)  and  creative  industry  (e.g.  for  films,  video  games  and  other 
virtual reality projects). 

Using and reusing digitised cultural heritage assets 

Respondents were asked to express their agreement to a set of statements related to using 
and  reusing  digitised  cultural  heritage  assets,  based  in  particular  on  the  lessons  learned 
from the COVID-19 crisis. 

Improving access to digital cultural heritage was perceived as very important overall and 
across all sectors and stakeholder types (see figure 4). Findings show that the availability 
of digitised cultural heritage content for reuse is mostly perceived very important in the 
research and education sectors.  

Figure 4. Question 14. In your opinion, in the light of the COVID-19 crisis, 
how important is it to improve online access to digital cultural heritage? 

90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

84% 

Very important

Important

Not very important

Not important at all

I don’t know / no opinion 

14% 

1% 

Total

0% 

0% 

Source: Open public consultation on digital access to European cultural heritage N=559 
59 

 
 
In  terms  of  ways  to  present  cultural  heritage,  the  most  appealing  to  respondents  were 
curated  content  (e.g.  online  exhibitions),  tutorials/online  classes  on  specific  topics  and 
virtual visits of sites (e.g. 3D or VR).  

Respondents highlighted the main difficulties and shortcomings in relation to accessing 
and  reusing  European  heritage  content  online,  including  insufficient  quality  (e.g.  low-
resolution images, or poor or no explanatory text), unclear copyright and re-use status of 
digital objects, insufficient content available.  

The vast majority of respondents (90%) considers that digital technologies empower and 
encourage  people  into  more  active  participation  such  as  developing  new  content  or 
uploading their cultural content online, or blogging about it. 

Digital transformation 

Digital  transformation  has  been  the  issue  that  most  of  the  respondents  considered 
important in the context of a future revision of the Recommendation.  

In the light  of the COVID-19 crisis,  providing online access  to  digitised content (81%) 
and  creating  a  more  agile  digital  environment  (71%)  were  considered  to  be  the  most 
important aspects of digital transformation in the cultural heritage sector. 

With  regard  to  digital  skills  in  the  aftermath  of  COVID-19,  data  and  metadata 
management and analytics, copyright and licensing, followed by digitisation are the core 
skills set most of respondents consider important to acquire. 

On the applications of data and AI technologies in the area of cultural heritage, the vast 
majority  of  respondents  (91%)  found  the  automated  annotation  of  digitised  cultural 
heritage to be (very) valuable in the area of cultural heritage (in particular by respondents 
from the tourism and creative arts/media sectors). Around 90% of the respondents found 
the  automated  recognition  of  objects  and  fragments  of  objects  to  be  (very)  valuable, 
followed by the extraction of new knowledge from cultural heritage data (89%) and the 
reconstruction of lost cultural heritage information (85%). 

On the digital infrastructure elements, 96% of respondents consider common data spaces 
for sharing cultural heritage data and building high-value data sets to be (very) valuable. 
97%  of  respondents  considered  repository  infrastructure  for  digitised  cultural  heritage 
content to be (very) valuable, followed by cloud  infrastructure for research on digitised 
cultural heritage material (90%) and supercomputing capabilities for large-scale cultural 
heritage simulations (76%). 

The  majority  of  respondents  (98%)  consider  that  the  EU  and  Member  States  should 
intensify their actions in order to help cultural heritage institutions address the challenges 
and  seize  the  opportunities  of  the  digital  era  and  (81%)  agreed  that  the  digital 
transformation  of  European  cultural  heritage  institutions  should  be  addressed  and 
coordinated at the EU level  

The EU’s support to Europeana, Europe’s only digital platform for cultural heritage was 
perceived important by 87 % of the respondents.  

Specific questions about Recommendation 2011/711/EU 

60 

 
  Effectiveness: 41% of respondents either agreed (33%) or strongly agreed (8%) that 
the provisions of the Recommendation have been implemented effectively and have 
achieved their objective to improve conditions in the areas addressed.  

  Efficiency: 70% of respondents either agreed (44%) or strongly agreed (26%) that the 
benefits  of  implementing  the  Recommendation  are  significant  and  justify  the  costs, 
which are proportionate. with this statement 

  Coherence:  59%  of  respondents  either  agreed  (45%)  or  strongly  agreed  (14%)  that 
the  Recommendation  has  been  coherent  with  other  EU  policies  and  initiatives. 
Approximately with this statement. 

  Relevance: 95% of respondents either strongly agreed (70%) or agreed (25%) that the 
Recommendation should be updated to reflect today’s needs in the area of digitised 
cultural heritage and increase the potential of Europe’s cultural heritage.  

  EU added-value: 80% of respondents either agreed (45%) or strongly agreed (35%) 
that the Recommendation provides added value compared to initiatives that Member 
States would have taken in the absence of it.  

61 

 
 
 
 
 
 
 
Annex 3: Methods and analytical models 

This  annex  provides  additional  details  on  the  main  methodological  steps  of  the 
evaluation, i.e. inception phase, data collection, data analysis, synthesis and conclusions. 

1) Inception phase: 

This  phase  laid  down  the  foundations  for  the  subsequent  stages  of  the  evaluation.  It 
included  the  preparation  of  the  intervention  logic  and  of  the  evaluation  questions  on 
which the evaluation was built. The evaluation questions are presented at the end of this 
annex. 

This  phase  included  also  the  meeting  with  the  contractor  and  the  ISG  members  to  lay 
down the basis for the process and deliverables. 

2) Data collection and 3) Data analysis: 

An evaluation roadmap defining the scope of the evaluation was published between 29 
July  2019  and  26  August  2019  for  feedback.  The  Commission  received  eight 
contributions  from  the  Swedish  National  Heritage  Board  (Sweden),  Europeana 
Foundation (Netherlands), Land Salzburg koordiniert vom Landes-Europabüro (Austria), 
Federation  of  European  Publishers  (Belgium),  Wikimedia  (Bulgaria),  Jukka  S. 
RANNILA (Finland), Andre REBENTISCH (Germany) and a citizen. The Commission 
took these comments into account to the extent possible and informed the evaluation. 

The evaluation study,  i.e.  Study to support the  evaluation  and possible revision of  the 
Commission Recommendation (2011/711/EU) on the digitisation and online accessibility 
of  cultural  material  and  digital  preservation  was  conducted  by  ICF  between  August 
2019  and  March  2020  and  updated  at  a  later  stage,  in  November  2020,  following  the 
online  public  consultation.  The  general  objective  was  to  carry  out  an  evidence-based 
assessment  of  the  results  and  impacts  of  Recommendation  across  the  EU  and  to  assess 
the potential impacts of a possible revision of the Recommendation. Data weregathered 
through desk research and stakeholder consultation activities. More information on data 
collection in Annex 2 of the evaluation study. 

A  consultation  strategy  has  been  created  and  agreed  with  the  ISG.  The  consultation 
activities  aim  at  collecting  the  views  of  Member  States’  competent  authorities,  cultural 
heritage institutions, international and umbrella organisations, the Europeana ecosystem 
of organisations  and professionals, technology stakeholders with research, products and 
services in ICT for cultural heritage, and citizens. All these different stakeholder groups 
are  expected  to  have  important  information  and  insights  on  actions  taken  for  the 
Recommendation’s implementation and its results, as well as an interest in and opinions 
on shaping the debate about the possible options for the future. 

The  consultation  strategy  has  two  main  objectives:  (1)  collect  views  on  the 
Recommendation’s 
the  retrospective 
evaluation  of  the  Recommendation)  and  (2)  collect  views  on  the  impacts  of  possible 
future changes to the Recommendation (to support the forward-looking assessment). 

implementation(to  support 

the  analysis  on 

62 

 
 
 
The  Commission  held  a  series  of  workshops  with  representatives  of  Member  States, 
stakeholders, international organisations and experts in order to collect relevant and solid 
information  on  the  impact  of  the  Recommendation,  as  well  as  on  the  challenges  and 
needs of the sector in the context of the digital transformation, be addressed in the future. 
More particularly, the Commission held: 

a)  An  workshop  on  the  future  of  the  Commission Recommendation  of  27 October 
2011  on  the  digitisation  and  online  accessibility  of  cultural  material  and  digital 
preservation  (2011/711/EU),  in  the  context  of  the  DCHE  group  meeting, 
November 2019, Luxembourg; 

b)  An  workshop  on  the  implementation  of  the  Declaration  of  Cooperation  on 
advancing  the  digitisation  of  cultural  heritage,  in  the  context  of  DCHE  group 
meeting (November 2019, Luxembourg); 

c)  An  workshop  on  the  digital  transformation  in  the  cultural  heritage  sector  in  the 

context of the Europeana conference ( November 2019, Lisbon)158;  

d)  An workshop on the digital cultural heritage, with the internationals organisations 

and networks relevant for the sector (December 2019, Brussels);  

e)  An  workshop  on  Digital  transformation  in  cultural  heritage159,  (January  2020, 

Brussels)160. 

Between  November  2019  and  January  2020,  the  Commission  held  a  targeted 
consultation of the DCHE representatives, aiming to collect data from stakeholders with 
the  most  expertise  on  the  Recommendation  (and  where  applicable  national  cultural 
heritage institutions, depending on each Member State), and including questions covering 
all  five  evaluation  criteria.  The  Commission  received  20  responses,  from  18  Member 
States, that informed the evaluation study. 

Between  22  June  and  14  September  2020,  the  Commission  held  an  open  public 
consultation,  focusing  on  important  issues  for  the  sector,  such  as  the  importance  of 
digitisation  and  digital  transformation  in  the  field  and  the  ways  of  supporting  such 
processes, as well as on the impact of the Recommendation. The objective was to collect 
diverse  opinions  and  experiences  from  all  stakeholder  groups.  The  consultation  was 
available  in  all  the  EU  official  languages161    and  gathered  565  responses  overall.  The 
results of the online consultation are analysed in a separate Annex (see Annex X) to this 
report)  as  well  as  integrated  into  the  evidence  base  for  the  evaluation.  No  significant 
limitations have been encountered. 

An  additional  main  source  of  information  for  the  study  and  the  evaluation  were  the 
national  implementation  reports  provided  by  the  Member  States  and  assessed  and 
consolidated  by 
the 
Recommendation’s implementation (see Section 2). 

in  a  (consolidated)  progress  report  on 

the  Commission 

158 Organised with the assistance of the contractor within the framework of the Study supporting the 
evaluation of the Recommendation 
159 Organised with the assistance of the contractor within the framework of the Study supporting the 
evaluation of the Recommendation 
160 Building on the information collected from previous workshops 
161 Except for Gaelic 

63 

 
 
                                                           
4) 

Synthesis and conclusions 

The final steps of this phase mainly consist of: 

  an  assessment  of  the  outcomes  of  the  evaluation  study,  of  the  consultation 

activities and of the implementation reports, followed by   

  conclusions on the impact of the Recommendation in the cultural heritage sector 
across  the  EU  and  more  particularly  on  its  effectiveness,  efficiency,  coherence, 
relevance and EU added-value. 

64 

 
Annex 4: Evaluation criteria and questions 

Effectiveness criterion 

Evaluation question  

Judgment criteria  

Indicators  

Sources 

1. To what extent have the 
provisions of the Recommendation 
been implemented by the Member 
States?  

2a. To what extent have the 
objectives of the Recommendation 
been achieved?  

2b. To what extent can the 
achievements be attributed to the 
Recommendation?  

■ Actions have been taken to 
implement the provisions set out in 
the Recommendation 

■ Activities related to the five 
provisions by the Recommendation 
by Member State (e.g. Presence of a 
digitisation strategy/action plan) 

■ The Recommendation contributed 
to the realisation of Europe’s cultural 
heritage potential 

■ Investment (MS and private 
sector) in digitisation, online access 
and digital preservation 

■ The Recommendation contributed 
towards Europe’s position as a 
leading international player in the 
field of culture and creative content  

■ The Recommendation helped to 
unlock investment by Member States 
and the private sector 

■ Desk research (e.g. Member State 
reports, evaluation of Europeana)  

■ Member States consultation 
questionnaire 
■ Desk research 
■ Public consultation; interviews 
■ Workshops 

■ Member States consultation 
questionnaire 

■ Workshops 

■ Member States consultation 
questionnaire 

■ Online profile of Europe’s 
cultural heritage 

■ Utilisation of Europe’s cultural 
heritage in the creative industries, 
sustainable tourism and education  

■ Stakeholders view on the 
synergies and economies of scale 
created by the Recommendation 
within the cultural heritage sector 
(e.g. cross-border collaboration, or 
cross-border programmes)   
■ Challenges/facilitators in the 
Recommendation’s implementation  
■ Digital trends and technical 
developments in Europe – inter-
relationships with the 

65 

3. What factors (internal or external) 
facilitated or hindered the 
achievement of the objectives of the 
Recommendation?  

■ External factors - factors that are 
not related to the Recommendation’s 
implementation and would be 
present even in its absence (e.g. 
internet coverage) - 

 
 
 
 
Evaluation question  

Judgment criteria  
promoted/hindered the achievement 
of the objectives of the 
Recommendation 
■ Internal factors – factors related to 
the design and implementation of the 
recommendation (e.g. shifting needs 
and expectations from cultural 
heritage institutions - 
promoted/hindered the achievement 
of the objectives of the 
Recommendation 

Efficiency criterion 

Evaluation question  
4a. What have been the barriers/costs 
associated with the 
Recommendation’s implementation?  

4b. How do the barriers/costs vary 
by different types of stakeholders 
(e.g. small-medium institutions and 
companies)?  

Judgment criteria  
■ The Recommendation’s 
implementation has generated costs 
for the Member States and other 
stakeholders 
■ The recommendation’s 
implementation is associated with 
barriers such as access  

Indicators  
Recommendation 
■ Needs of cultural heritage 
institutions and users 

Sources 

■ Desk research (e.g. Member State 
reports, Eurostat) 

Indicators  
■ Financial investments and 
recurrent costs  
■ Human resources dedicated to 
carrying out activities related to the 
Recommendation 
■ Stakeholder opinions on costs 
incurred due to the 
Recommendation’s implementation  
■ Stakeholders experience related to 
costs incurred due to 
implementation  

Sources 
■ Desk research (e.g. Member State 
reports, Evaluation of Europeana, 
“Enumerate” survey results) 
■ Workshops 

■ Member States consultation 
questionnaire 

5. What have been the benefits 
associated with the 
Recommendation’s implementation?  

■ The Recommendation has led to 
an increased digitisation of all 
cultural material and increased cross-
border sharing of expertise and 
resources (output level) 

■ Quantity and quality of 
digitisation of all types of cultural 
material 
■ Private sector engagement 
■ Online engagement of users 

■ Desk research (e.g. Member State 
reports and "Enumerate" survey) 
■ Workshops 

■ Member States consultation 

66 

 
 
  
 
 
 
 
 
 
 
 
Judgment criteria  
■ The Recommendation has led to 
an increased online accessibility and 
reuse of all types of cultural material 
in national and EU portals and has 
attracted private sector investment 
(outcome level) 
. 
■ The costs to implement the 
Recommendation are outweighed by 
the benefits generated. 
■ The activities supporting the 
Recommendation’s implementation 
are efficient in generating social and 
economic impacts. benefits.   

Evaluation question  

6a. Are the costs to implement the 
Recommendation proportional with 
the benefits (e.g. social and positive 
economic impacts) generated?  

6b. Could the benefits (e.g. social 
and economic impacts) generated by 
the Recommendation been achieved 
with fewer EU and/or national 
resources? Is there potential for 
burden reduction and simplification?  

Relevance criterion 

Indicators  
■ Cross-border sharing  

Sources 
questionnaire 

■ Human and financial resources 
related to the implementation of the 
Recommendation 
■ Benefits generated by the 
Recommendation (see EQ 5) 
■ Stakeholders views on the balance 
between benefits and costs arising 
from the Recommendation’s 
implementation  

■ Desk research (e.g. Member State 
reports, evaluation of Europeana, 
“Enumerate” survey results) 
■ Workshops 

■ Member States consultation 
questionnaire 

Evaluation question  

Judgment criteria  

Indicators  

Sources 

■ Desk research  

7. To what extent has the 
Recommendation been relevant in 
view of the identified needs at the 
time it was introduced? 

■ The Recommendation addresses 
the lack of a harmonised approach 
across Member States in the 
digitisation of cultural heritage 

■ Stakeholders views on the 
relevance of provisions as they were 
set out and scope for change, gaps, 
and components of cultural heritage 
not covered by the Recommendation  

■ The Recommendations addresses 
the untapped potential of Europe's 
cultural heritage 

■ The Recommendations addresses 

67 

 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation question  

Judgment criteria  
the wide range of threats to Europe's 
cultural heritage  

Indicators  

Sources 

■ Significant technical developments 
took place since 2011 

■ Relevant technology trends in the 
cultural heritage sector 

8a. To what extent has the context 
changed (e.g. 3D, augmented reality) 
since the introduction of the 
Recommendation?  

8b. What are the implications of 
contextual changes for the needs of 
different stakeholder groups?   

■ The lifecycle of digitisation aligns 
with contextual changes due to 
technical progress since 2011 (e.g. 
emergence of AI and Big Data, 3D 
technologies, linked open data, etc.)  

■ New/latest directions in 
digitisation, online access and digital 
preservation of cultural heritage at 
national levels in Member States 

■ Relevant audience related trends 
relevant to digital cultural heritage 

■ Stakeholders opinions on what 
could be amended to improve 
relevance considering the most recent 
contextual trends (e.g.technological 
progress)  

■ Desk research  

■ Online public consultation; 
interviews 

■ Member States consultation 
questionnaire 
■ Workshops 

■ Desk research  

■ Online public consultation; 
interviews 

■ Member States consultation 
questionnaire 
■ Workshops 

9a. To what extent is the 
Recommendation relevant in view of 
current needs?  

■ Some provisions have a limited 
relevance to current needs 

9b. Are there certain provisions that 
are dated and not in line with 
technological developments?  

Coherence criterion 

Evaluation question  

Judgment criteria  

Indicators  

Sources 

10. To what extent is the 
Recommendation aligned and 
consistent with other EU legislation 
related to open data, copyright, 

■ The different provisions of the 
Recommendation are coherent with 
the following EU policies:  

■  Links and synergies between the 
digital lifecycle approach and the 
current Digital Single Market, and 
culture policies   

68 

■ Desk research (e.g. review of EU 
policies) 

 
 
 
 
 
 
 
 
 
 
Evaluation question  
culture and digitisation that was in 
place in 2011 and that was adopted 
since that time?  

Judgment criteria  

Indicators  

Sources 

– Directive (EU) 2019/1024 on open 
data and the re-use of public sector 
information 

– Orphan Works Directive 
(2012/28/EU) 

– New copyright package  

– European Year of Cultural 
Heritage  

– European Framework  for Action 
on Cultural Heritage  

- Agenda for Culture 

■ The Recommendation supports the 
overall objectives of the Digital 
Single Market: 

-  Digital Single Market Strategy   
-  Shaping Europe’s Digital Future 
-  A European  strategy for data 

11. To what extent is the 
Recommendation aligned and 
consistent with the Europeana 2015-
2020 strategy, and the new 
Europeana strategy 2020-2025? 

■ The different provisions of the 
Recommendation are reinforced by 
the strategies related to Europeana.  

■ Links and synergies between the 
provisions of the Recommendation 
and key elements of the Europeana 
strategies.  

■ Desk research (e.g. Evaluation of 
Europeana, review of Europeana 
policies) 
■ Interview with the Europeana 
ecosystem 
■ Workshops 

69 

 
 
 
 
 
Evaluation question  

Judgment criteria  

Indicators  

Sources 

12. To what extent is the 
Recommendation coherent with and 
complementary to cultural heritage 
policies in the Member States? 

■ The different provisions of the 
Recommendation have synergies 
with cultural heritage policies in the 
Member States  

■ Cross-references and gaps between 
national policies and the 
Recommendation  

■ Desk research (e.g. Member State 
reports)  

■ Member States consultation 
questionnaire 

EU-added value criterion  

Evaluation question  

Judgment criteria  

Indicators  

Sources 

13. To what extent did the 
Recommendation generate an 
increase in the digitisation and 
online accessibility of cultural 
material, as well as actions taken for 
their digital preservation above and 
beyond what Member States would 
have achieved alone? (Volume effect)  

■ The Recommendation generated 
an increased level of digitisation and 
online accessibility of cultural 
material that is markedly greater 
than what could be achieved by the 
Member States alone 

■ Stakeholders view on the synergies 
and economies of scale created by the 
Recommendation within the cultural 
heritage sector (e.g. cross-border 
collaboration, or cross-border 
programmes)   

■ Public consultation; interviews 

■ Member States consultation 
questionnaire 

■ Workshops  

14. To what extent did the 
Recommendation lead Member 
States to broaden the types of 
measures/actions implemented to 
promote the digitisation and online 
accessibility of cultural material? 

■ Evidence suggesting that the scope 
of the intervention would have 
changed if MS had implemented a 
similar intervention alone  

■ Stakeholders opinion on how 
difficult would have been for 
Member States in isolation to 
deliver/achieve a similar performance 
(scope) without the Recommendation  

■ Desk research (e.g. Member State 
reports) 

■ Member States consultation 
questionnaire 

■ Evidence suggesting 

70 

 
 
 
 
 
 
 
 
 
 
 
Evaluation question  
(Scope effect)  

Judgment criteria  
heterogeneous capacity across 
Member States to implement the 
provisions in the Recommendation  

■ Evidence of economies of scope in 
place  

Indicators  

Sources 

■ Stakeholders opinions on 
economies of scale  

15. To what extent did the 
Recommendation – direct or 
indirectly through Europeana - 
promote innovation and the transfer 
of ideas across Member States with 
respect to the digitisation, online 
accessibility and digital preservation 
of cultural material (Role effect)  

■ Evidence of share of good 
practices  

■ Trends in time of good practice 
sharing examples (e.g. cross-border 
aggregators) 

■ Stakeholders feedback on examples 
and trends of knowledge sharing 
experiences, and the coordinated 
effort as enabling factor  

■ Desk research (e.g. Member State 
reports)  

■ Public consultation; interviews 

■ Workshops 

71