REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS Final evaluationof the implementation of the multiannual Community Programme to make digital content in Europe more accessible, usable and exploitable
Date: 2011-09-09
Institutions: European Commission
Subjects: EU programme, access to information, digital archiving, digital technology, dissemination of information, education, virtual library

EUROPEAN COMMISSION 

Brussels, 9.9.2011 
COM(2011) 548 final 

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE 
COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE 
COMMITTEE OF THE REGIONS 

Final evaluation 
of the implementation of the multiannual Community Programme to make digital 
content in Europe more accessible, usable and exploitable 

EN 

   EN 

 
 
 
 
  
 
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE 
COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE 
COMMITTEE OF THE REGIONS 

Final evaluation 
of the implementation of the multiannual Community Programme to make digital 
content in Europe more accessible, usable and exploitable 

1. 

INTRODUCTION 

This  Report  concerns  the  final  evaluation  of  the  eContentplus  Programme  (2005-2008) 
referred to in this document as ‘the Programme’1. 

The Programme ran for a period of 4 years from 1 January 2005 to 31 December 2008 with a 
budget of EUR 149 million, of which 100 % has been fully committed to projects, studies and 
administration  (cost  related  to  the  organisation  of  evaluations  of  proposals  and  project 
reviews). 

The  objective  of  the  Programme  was  to  make  digital  content  in  Europe  more  accessible, 
usable and exploitable, facilitating the creation and diffusion of information, in areas of public 
interest, at Community level. 

The  Programme  was  part  of  the  Commission’s  policy  announced  in  ‘i2010  —  A  European 
Information Society for growth and employment’. In aiming to facilitate content access, use 
and exploitation and to stimulate content enrichment, the eContentplus Programme addressed 
two  of  the  four  main  challenges  posed  by  digital  convergence  for  the  creation  of  a  single 
European  Information  Space,  as  identified  by  i2010,  namely  ‘interoperability’  and  ‘rich 
content’. More specifically, it set out to help achieve Objective 1 of i2010: A Single European 
Information Space offering affordable and secure high bandwidth communications, rich and 
diverse content and digital services.  

In  order  to  achieve  the  overall  aim  of  the  Programme,  the  following  lines  of  action  were 
addressed: 

(a) 

(b) 

facilitating  at  Community  level  access  to  digital  content,  its  use  and 
exploitation; 

improving  quality  and  enhancing  best  practice  related  to  digital  content 
between content providers and users, and across sectors; 

(c) 

reinforcing cooperation between digital content stakeholders and awareness. 

Via those lines of action, the Programme targeted areas which had a public interest and which 
would not have developed, or would have developed at a slower pace, if left to the market:  

1 

Decision No 456/2005/EC of the European Parliament and of the Council, OJ L 79 of 24.3.2005, p.1. 

EN 

2 

   EN 

 
                                                 
–  Geographic information: The objective of eContentplus is to contribute to the wider use 
of  geographic  information  (i.e.  any  data  that  directly  refer  to  a  specific  location  or 
geographic area) by public sector bodies, private companies, including small and medium-
sized enterprises, and citizens in line with the PSI Directive2 on the re-use of public sector 
information  as  well  as  the  INSPIRE  Directive3  which  aims  to  make  available  relevant, 
harmonised  and  quality  geographic  information  for  the  purpose  of  formulating, 
implementing,  monitoring  and  evaluating  Community  policy-making.  In  this  context  the 
programme  has  funded  projects  applying  the  following  INSPIRE  principles  which  are 
particularly  relevant  for  increasing  reuse,  whereby  it  should  be  easy  to  discover  which 
spatial  data  are  available,  to  evaluate  their  fitness  for  purpose  and  to  know  which 
conditions apply for their use and it should be possible to combine seamlessly spatial data 
from different sources and share it between many users and applications. 

–  Educational content: Educational digital content is content that can be used for learning 
in different contexts: in formal education and training programmes, in non-formal general 
education, in vocational training courses, as well as for self-learning. eContentplus aims to 
foster  the  efficient  use  of  pedagogically  sound  digital  educational  content  across  borders 
and  to  complement  other  Community  initiatives  like  Decision  No.  1720/2006/EC  of  the 
European  Parliament  and  of  the  Council  of  15  November  2006  establishing  an  action 
programme in the field of lifelong learning which includes among its objectives support to 
the  development  of  innovative  ICT-based  content,  services,  pedagogies  and  practice  for 
lifelong learning. 

–  Cultural and scientific/scholarly content digital libraries: Digital libraries are organised 
collections  of  digital  content  made  available  to  the  public  by  cultural  and  scientific 
institutions (libraries, archives and museums) and private content holders (e.g. publishers) 
in the EU Member States or other countries participating in the programme. eContentplus 
aims  at  leveraging  the  multilingual  availability  of  significant  assets  of  digital  cultural, 
scientific and scholarly content by supporting the development of interoperable collections 
and  objects  -  on  which  multilingual  and  cross-border  services  can  be  built  -  and  by 
supporting solutions to facilitate exposure, discovery and retrieval of these resources4. On 
30 September 2005, the Commission adopted a Communication outlining the vision of the 
«i2010: Digital Libraries» initiative, which aims to make European cultural heritage easier 
to  access  and  use  in  an  online  environment.  Within  this  initiative,  the  Commission  has 
supported the creation of the European Digital Library (europeana.eu) which is expected to 
make  available  online  at  least  ten  million  books,  documents  and  other  cultural  works  by 
2010. 

The eContentplus Programme was implemented by means of annual calls for proposals. Their 
specific  objectives  and  priorities  were  set  out  in  annual  work  programmes,  which  also 
provided  the  available  budget  as  well  as  an  indicative  timetable  for  the  evaluation  and 
selection of project proposals submitted in response to the calls. Funding was allocated to 67 
projects  for  a  total  of  EUR  146.9  million  with  more  than  800  European  organisations 
participating directly in the projects. 

2 
3 
4 

Directive 2003/98/EC of the European Parliament and of the Council, OJ L345 of 31.12.2003, p. 90. 
Directive 2007/2/EC of the European Parliament and of the Council, OJ L 108, 25.4.2007, p.1. 
Some  of  these  activities  were  complemented  by  the  MEDIA  2007  programme  which  comprises  a 
scheme supporting video on demand platforms and digitisation of cinemas. Decision No 1718/2006/EC 
of the European Parliament and of the Council of 15 November 2006 

EN 

3 

   EN 

 
                                                 
The Commission also offered contracts for studies to improve understanding of the economic 
importance  of  the  activities  covered  by  the  Programme  and  provide  input  for  defining  the 
priorities  of  future  research  and  non-research  activities  developed  at  EU  level.  These 
included:  a  study  on  socio-economic  drivers  and  the  impact  of  longer-term  preservation  of 
digital  resources,  the  first  systematic  attempt  to  map  or  measure  the  social  and  economic 
impact of the public domain in the information society and, following the 2008 review of the 
Directive  on  the  re-use  of  Public  Sector  Information  (PSI),  a  study  to  establish  a  European 
PSI central platform to further stimulate action and monitor progress towards a stronger and 
more transparent environment for the growth of national and European markets in PSI re-use. 

2. 

EVALUATION OBJECTIVES 

Pursuant  to  Article  5(3)  of  the  Programme  Decision,  the  Commission  was  to  submit  a  final 
evaluation report at the end of the Programme. This evaluation has been carried out with the 
assistance of a panel of external experts who evaluated the effectiveness of the Programme, its 
achievements, its impact, its sustainability and its complementary with other initiatives within 
and beyond the European Union, as well as with national initiatives. 

The findings of this final evaluation are based on available documentation, Commission data, 
a survey questionnaire, and follow-up interviews with selected project participants. 

3. 

EVALUATION FINDINGS 

The evaluators found that the Programme has been successful in achieving the objectives set 
out  in  the  Programme  Decision  and  in  the  annual  work  programmes  adopted  for  its 
implementation.  It  has  contributed  to  making  digital  content  in  Europe  more  accessible, 
usable and exploitable, facilitating the creation and diffusion of information, in areas of public 
interest. More specifically, the evaluation gave rise to the following conclusions: 

The enabling role of the Programme and Programme achievements 

The  Programme  achievements  are  in  line  with  the  objectives  of  the  Programme.  The 
Programme  succeeded  in  creating  better  conditions  for  accessing,  using,  re-using  and 
exploiting digital material, thus making it possible to build added-value products and services 
across Europe in the target areas selected. It helped content stakeholders (providers and users) 
realise  the  full  potential  of  digital  content.  Content  providers,  i.e.  public  and  private 
organisations and institutions that create, collect or own digital content, were able to increase 
the  visibility  of  their  digital  material,  not  only  in  their  own  country,  but  also  in  other  EU 
Member  States  and  countries  participating  in  the  programme.  Content  users,  including 
European citizens, students and researchers as well as organisations and enterprises, in turn, 
are now able to find and use such content, irrespective of location or language. 

By bringing together Europe-wide consortia and contributing to the creation of cross-border 
datasets  and  services,  the  eContentplus  programme  has  contributed  to  making  content  more 
accessible,  usable  and  exploitable  in  the  above-mentioned  target  areas.  As  the  survey  and 
interviews  confirmed,  most  projects  would  not  have  been  realised  without  financial  support 
from the eContentplus Programme, underlining the usefulness of such financial mechanisms 
at European level. 

EN 

4 

   EN 

 
At project level, results such as the establishment of the European Digital Library Europeana, 
the  common  multilingual  access  point  to  Europe’s  digital  cultural  heritage,  would  not  have 
been achieved. The creation of this flagship project, around which some 20 ‘feeder’ projects 
cluster, providing access to around 17 million cultural items from more than 1500 European 
cultural institutions (directly or indirectly through aggregators), is a significant achievement. 
Participation  in  this  initiative  has  heightened  the  visibility  of  the  project  partners  and  given 
them a much greater incentive to maintain the service once the funding cycle is complete. As 
regards  educational  content,  there  have  been  major  achievements  thanks  to  more  intensive 
networking  and  collaboration.  The  MELT  project,  for  instance,  provides  the  Learning 
Resource Exchange portal for schools (lreforschools.eun.org) where you can find more than 
130,000 open educational resources and assets from many different countries and providers, 
including 17 Ministries of Education. In the area of geographic information, the Programme 
has  brought  together  data  providers,  technology  providers,  users  and  other  stakeholders, 
creating networks that will carry on providing a platform for sharing information, state-of-the-
art technology and lowering interoperability barriers in the EU. Thus, the project EURADIN 
will implement, test and validate a pilot solution for a European infrastructure to be used as a 
reference  for  all  Member  States  to  fulfil  the  INSPIRE  recommendations  with  respect  to 
addresses. 

Programme efficiency 

The  Programme  was  managed  efficiently.  Both  applicants  and  project  participants  were 
pleased  with  the  quality  of  information  provided  by  the  Commission,  although  some 
newcomers  and  organisations  not  familiar  with  EU-funded  projects  initially  raised  some 
minor issues relating to the clarity of programme objectives and funding schemes. 

The  publication  of  clear  programme  objectives  and  award  criteria  on  the  one  hand  and  the 
two-stage  evaluation  process  on  the  other  had  very  positive  effects  on  the  submission  and 
selection of high-quality proposals. 

Simplifications  introduced  in  the  eContentplus  grant  agreement  have  accelerated  the 
procedures for claiming project costs and receiving payments. 

Best  Practice  Networks  were  introduced  as  a  new  project  type  to  promote  the  adoption  of 
standards and specifications for making digital content in Europe more accessible and usable. 
This was achieved by combining the consensus-building and awareness-raising functions of a 
network with the large-scale implementation, in a real-life context, of one  or more concrete 
solutions by its members. 

Programme impacts 

in  eContentplus  projects  has 

The  impact  on  organisations  participating  in  the  Programme  can  only  be  estimated,  since 
many projects are still running or have just ended. While most organisations expect increased 
visibility and exploitation of their digital content in the near future, they already recognise that 
participation 
international  and  cross-border 
collaboration  and  made  them  more  aware  of  the  economic  and  cultural  value  of  the  digital 
content they own. eContentplus projects, has supported for example the conversion to digital 
text  of  more  than  20  million  pages  of  scanned  books  and  newspapers  held  by  national 
libraries. As a result of this effort, in the near future citizens will be able to full-text search 
such  content  through  European  and  the  various  national  libraries'  portals.  European  TV 
broadcasting archives managed to create a portal through which part of their content can be 

increased 

EN 

5 

   EN 

 
accessed,  allowing  citizens  to  explore  the  development  of  television  in  Europe,  but  also  the 
televising of cultural and historical events within and across nations. 

In  the  area  of  digital  libraries,  the  eContentplus  Programme  is  the  first  programme  whose 
main impact has been to make material directly available to individual citizens on a very large 
scale.  Indeed,  rather  than  influencing  stakeholders  or  making  technical  adjustments  and 
standards  which  encourage  or  permit  such  availability,  europeana.eu  has  become  the  access 
point  for  European  cultural  content.  The  Programme  has  also  highlighted  a  number  of 
challenges that have to be addressed. Increased public funding is needed to finance large-scale 
digitisation,  alongside  initiatives  with  private  partners  provided  that  they  allow  a  general 
accessibility of  Europe's common cultural heritage online. Fragmentation and complexity in 
the  current  licensing  system  also  hinders  the  digitisation  of  a  large  part  of  Europe's  recent 
cultural  heritage.  Rights  clearance  must  be  improved,  and  the  role  and  responsibilities  of 
private  and  public  organisations  for  digitising  orphan  works  as  well  as  material  that  is  in-
copyright but no longer commercially available should be clarified. 

In the area of educational content, a significant increase in the use of the underlying content is 
expected  as  a  result  of  programme  support.  For  example  the  portal  lreforschools.eun.org 
enables  teachers  and  pupils  to  easily  find  and  use  educational  content  from  many  different 
countries through one single access point. In addition, the involvement of national Ministries 
of  Education  in  many  of  the  projects  is  also  likely  to  have  an  impact  on  the  inclusion  of 
eLearning in curricula. 

With  regard  to  geographic  information,  programme  support  is  expected  to  increase 
interoperability  within a number of thematic areas under the  INSPIRE  Directive  with  direct 
and measurable impacts on the implementation rules of INSPIRE. 

Sustainability 

Long-term sustainability is difficult to assess at this stage. However, projects report a strong 
link  to  national  policy  makers  and  most  are  optimistic  that  projects  will  carry  on  based  on 
funding from public and/or private institutions involved in the project. 

4. 

MAIN RECOMMENDATIONS 

There  is  an  increasing  demand  for  quality  digital  content  in  Europe  from  citizens,  students, 
researchers,  SMEs  and  other  business  users  or  re-users  wishing  to  exploit  digital  content 
resources to create services. This calls for continued support at European level to increase the 
availability of European digital content, in particular in areas of public interest where progress 
is likely to be slow and where access to content is limited by language and cultural barriers in 
particular. 

A number of general points have emerged during this evaluation that could be indicative for 
future actions:  

(1) 

Funding  should  be  linked  to  clear  and  defined  policy  objectives.  Even  if  funding 
actions in other areas have positive outcomes and effects, focusing on areas where the 
EU  has  clear  and  defined  policy  objectives  is  likely  to  produce  more  sustainable 
outputs  with  a  higher  impact.  The  link  between  the  target  area  of  cultural 

EN 

6 

   EN 

 
(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

content/digital  libraries  and  the  i2010  digital  library  initiative  is  a  particularly  good 
example of this. 

The Digital Libraries Initiative should continue to receive economic support from the 
European Commission to ensure the continued development of Europeana and similar 
services at European level.  

Although Public Sector Information (PSI) has enormous economic value, it is still not 
extensively exploited. The implementation of the PSI Directive should continue to be 
supported by activities that maximise the re-use of PSI in Europe and unlock its full 
economic potential. This is particularly evident in the area of geographic information 
where  the  fragmentation  of  datasets  hampers  cross-border  use  and  prevents  their 
exploitation for scientific and commercial purposes. 

In order to continue boosting the sharing of spatial information between public sector 
organisations  and  citizens,  eContentplus  follow-on  activities  should  support  projects 
helping  to  create  a  more  standardised  electronic  environment  for  storing  and 
harmonising spatial datasets covering one or more of the topics mentioned in Annex I-
III  of  the  INSPIRE  Directive.  Projects  such  as  EURADIN,  the  European  Addresses 
Infrastructure, and EGN (EuroGeoNames) are showcases in this respect. 

Both  the  multilingual  and  the  multicultural  dimensions  of  the  Programme  have  been 
highly appreciated and the European Commission should continue to emphasise such 
aspects. 

Simplifications  in  the  eContentplus  grant  agreement  have  been  perceived  to  be  very 
efficient, in particular the introduction of a flat rate for overheads and the payment of 
the Community contribution as pre-financing in instalments. The publication of a clear 
mapping  of  programme  objectives  and  award  criteria  as  well  as  the  two-stage 
evaluation process have had very positive effects on the submission and selection of 
high-quality proposals. They should be considered for grant agreements implementing 
future funding programmes. 

Best  Practice  Networks  have  proved  to  have  had  a  significant  impact  on  networking 
and  practical  testing  by  providing  the  most  appropriate  solutions  and  involving  all 
relevant  stakeholders.  This  funding  instrument  should  be  considered  for  funding 
activities that seek to achieve competitive and innovative solutions. 

In  the  educational  content  area,  continuous  action  is  needed  to  support  the 
implementation  of  efficient  context-aware  brokerage  systems  and 
learning 
technologies  standardisation.  Actions  carried  out  should  focus  on  the  use  of 
educational  content  by  specific  groups  (e.g.  SMEs)  in  specific  educational  contexts 
while still contributing to global standardisation of learning technologies. 

5. 

CONCLUSIONS 

In recognition of the need for further support at European level to increase the availability of 
European  digital  content,  particularly  in  areas  of  public  interest,  measures  to  make  digital 
content in Europe more accessible, usable and exploitable have been continued under the ICT 
PSP since 2009. 

EN 

7 

   EN 

 
The  themes  included  in  the  ICT  PSP  2009  and  2010  work  programmes  as  eContentplus 
follow-up  activities  are  in  line  with  the  evaluators’  recommendation  that  funding  actions 
should be linked to clearly defined policy objectives. The objectives for these themes seek to 
foster  projects  in  support  of  Commission  policy  in  the  following  fields:  digital  libraries; 
digitisation  and  online  accessibility  of  cultural  content,  and  digital  preservation;  access, 
dissemination and preservation of scientific information in the digital age and re-use of Public 
Sector Information. 

These areas have also been recognised among the priorities for action of the Digital Agenda 
for  Europe5.  The  ICT  PSP  will  continue  to  be  used  to  support  the  policy  objective  of 
strengthening  Europeana  and  supporting  initiatives  allowing  improved  accessibility  of 
Europe's cultural heritage online.  

In  line  with  the  recommendations  of  the  reflection  group  on  bringing  Europe's  cultural 
heritage online6, ICT PSP will be used to encourage the digitisation and online availability of 
cultural material - including books, maps, newspapers, sound, archival material, photographs, 
museum  objects,  films7  and  audiovisual  material  –  in  order  to  make  it  more  accessible  for 
Europe's citizens and easier to use for leisure, work and study. 

Funding will be used to encourage public bodies to realise the objective of stimulating content 
markets  by  making  their  information  available  on  transparent,  effective,  non-discriminatory 
terms.  

The Digital Agenda for Europe also sets out the policy objective of mainstreaming eLearning 
in the national policies in order to modernise education and training. eContentplus results in 
the  area  of  educational  content  could  become  a  major  building  block  on  the  way  towards 
achieving  this  objective.  Within  the  Digital  Agenda  for  Europe  pillar  of  "Enhancing  digital 
literacy,  skills  and  inclusion"  eLearning  will  foster  the  acquisition  of  the  digital  skills 
necessary for innovation and growth in a knowledge-based society. 

The Commission invites the European Parliament, the Council, the European Economic and 
Social Committee and the Committee of the Regions to: 

(1) 

(2) 

5 

6 

7 

take note that the Programme has been successfully implemented; 

continue  to  assist  the  Commission  in  its  work  of  increasing  the  visibility  of  the 
initiatives  aimed  at  making  digital  content  in  Europe  more  accessible,  usable  and 
exploitable and stimulate a continued dialogue on digital content issues. 

COM(2010) 245, A Digital Agenda for Europe, Communication from the Commission to the European 
Parliament,  the  Council,  the  European  Economic  and  Social  Committee  and  the  Committee  of  the 
Regions 
"Comité des Sages" on Bringing Europe's Cultural Heritage On-line 
http://ec.europa.eu/information_society/activities/digital_libraries/comite_des_sages/index_en.htm 
Commission  Communication  on  opportunities  and  challenges  for  European  cinema  in  the  digital  era, 
COM(2010) 487 final 

EN 

8 

   EN