Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions - Final evaluation of the eTEN Programme
Date: 2008-06-03
Institutions: European Commission
Subjects: EU programme, report, telecommunications, trans-European network

EN 

EN 

   EN 

 
 
COMMISSION OF THE EUROPEAN COMMUNITIES 

Brussels, 22.01.2009 
COM(2008)334 final/2 

CORRIGENDUM : 
Ce document annule et remplace le COM(2008)334 final du 3.6.2008. 
Concerne la version EN. 

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN 
PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL 
COMMITTEE AND THE COMMITTEE OF THE REGIONS 

Final evaluation of the eTEN Programme 

(presented by the Commission) 

EN 

   EN 

 
 
 
  
 
 
COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN 
PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL 
COMMITTEE AND THE COMMITTEE OF THE REGIONS 

Final evaluation of the eTEN Programme 

(Text with EEA relevance) 

1. 

INTRODUCTION 

This  Communication  provides  a  final  evaluation  of  the  eTEN  Programme  covering 
the  period  from  re-orientation  in  2002  until  programme  end  in  December  2006  in 
response  to  the  requirement  expressed  in  Article  19  of  the  Financial  Regulation 
covering the programme.  

2. 

BACKGROUND 

eTEN  evolved  from  the  TENs  initiative1  whose  objectives  were  to  provide  Trans-
European  networks  in  the  areas  of  transport,  telecommunications  and  energy. 
Developing  European  transport,  telecommunications  and  energy  infrastructures  are 
ambitious objectives2 and are referred to in the Treaty and the Guidelines for growth 
and jobs3. 

The  objective  of  eTEN  was  to  contribute  to  the  development  of  the  Information 
Society in terms of growth, employment, social cohesion and participation for all in 
the  knowledge-based  economy.  The  main  themes  covered  were  eGovernment  and 
eAdministration, eHealth, eInclusion and eLearning. A sixth theme paying particular 
attention to SMEs was added at a later stage.  

Until 2005, all TENs shared a common financial legal base4 (hereinafter referred to 
as "The Financial Regulation"). under which a maximum grant of 10% of the Total 
Investment Cost5 could be made. Within this limit eTEN funded projects to a level of 
50%  for  Feasibility  and  Evaluation  Studies6,  and  to  a  level  of  10%  for  projects 
involving the deployment of services. The financial legal base for eTEN was revised 

1 

2 

3 
4 

5 
6 

The  Trans-European  Network  (TEN)  initiative  is  based  on  articles  154,  155  and  156  of  the  Treaty 
establishing  the  European  Community,  which  set  the  objective  of  establishing  Trans-European 
networks in the areas of transport, telecommunications and energy. 
COM  (2007)  0135  Communication  from  the  Commission  –  Trans-European  networks:  Towards  an 
integrated approach {SEC(2007) 374}. 
Guidelines for growth and jobs (2005-2008) No 9, 10, 11 and 16. 
Regulation  (EC)  No1655/1999  of  the  European  Parliament  and  of  the  Council  of  19  July  1999, 
amending  Regulation  (EC)  No  2236/95  laying  down  general  rules  for  the  granting  of  Community 
financial aid in the field of trans-European networks (OJ L 282, 24.11.1995, p. 16). 
Article 5 of Regulation No 1655/1999. 
Article 4 para 1(a) of Regulation No 1655/1999. 

EN 

2 

   EN 

 
 
                                                 
in  20057  to  allow  increased  grants  of  up  to  30%  of  the  Total  Investment  Cost  for 
deployment projects.  

In  conformance  with  requirements  of  the  Treaty,  a  set  of  Guidelines  was  produced 
covering  the  objectives,  priorities  and  broad  lines  of  measures  envisaged  and 
identifying  projects  of  common 
to  as  "The 
Guidelines"). The Guidelines were modified in 2002 to bring the programme more in 
line with Community policies and especially with i2010. At the same time the name 
of the programme was changed from TEN Telecom to eTEN.  

interest8  (hereinafter  referred 

Article 19 of the TEN's Financial Regulation requires the Commission to submit to 
the European Parliament and the Council a comprehensive report on the experience 
gained with the mechanisms under the Financial Regulation for granting Community 
aid,  in  particular  the  mechanisms  and  provisions  laid  down  in  Article  4.  This 
document responds to this request. 

eTEN was implemented through cost-shared projects selected on the basis of public 
calls  for  proposals.  A  total  of  149  cost  shared  projects  were  established  following 
public  calls  for  proposals  with  a  total  funding  of  just  over  160  million  €.  Projects 
from the 2003 and 2004 programmes have been completed but those from 2005 and 
2006 are still ongoing. All projects are expected to complete by the end of 2009. 

A midterm evaluation was completed in 20059 and this covered the results of projects 
from programmes up to and including 2002 as well as initial results from the eTEN 
programme which started in 2002.  

3. 

CONDUCT OF THE EVALUATION 

The  evaluation  assessed  the  following  issues:  relevance  of  the  programme's 
objectives,  priorities  and  means  of  implementation,  the  effectiveness  and  impact  of 
the programme, its efficiency and cost effectiveness, its utility and sustainability, and 
causal  links  from  resources  used  through  to  activities  and  presumed  impacts  (the 
intervention logic). 

This evaluation was carried out in 2007 by a team of professional evaluators from a 
company10, which was contracted for that purpose. They provided a clear assessment 
through  a  thorough  analysis  of  the  available  evidence  consisting  of  monitoring  and 
evaluation  material,  and  programme  documentation  and  outputs.  The  evaluators 
further enhanced the quality of the evidence base by undertaking interviews with key 
stakeholder representatives. The Commission hereby communicates the results of the 
evaluation in accordance with the Regulation concerning the programme. 

7 
8 

9 
10 

Regulation (EC) No 1159/2005 of the European Parliament and of the Council of 6 July 2005 
Decision No 1376/2002/EC of the European Parliament and of the Council of 12 July 2002 amending 
Decision  No  1336/97/EC  on  a  series  of  Guidelines  for  trans-European  telecommunications 
networks.(OJ 2002/ L200/1 of 30.07.2002) 
COM (2005)354 final, 1 Aug 2005. 
RAND Europe, contracted on the basis of a restricted call for tenders issued by DG Information Society 
in the autumn of 2006. 

EN 

3 

   EN 

 
                                                 
4. 

EVALUATION RESULTS 

Annex  1  of  this  Communication  contains  the  complete  set  of  Findings  provided  in 
the evaluator's report.  

The Commission welcomes the evaluation report and its findings confirming both the 
effectiveness of the programme management and the utility of the programme. 

The  Commission  believes,  contrary  to  the  findings  of  the  evaluation  report,  that 
certain  synergies  between  the  eTEN  programme  and  the  European  Cohesion  Fund 
have  already  been  created.  Entities  from  New  Member  States  have  been  very 
successful in eTEN based on the financial contribution from New Member States of 
about  5%  and  their  return  in  terms  of  project  funding  which  has  been  about  11%. 
Actual  experience  with  project  results  is  a  sound  basis  for  further  deployment  of 
services  and  it  is  known  that  some  entities  have  already  approached  the  European 
Cohesion Fund to support the further deployment of services. When the majority of 
projects  have  been  completed  further  synergies  can  be  expected.  The  Commission 
will  report  on  this  matter  in  the  midterm  report  of  the  ICT  Policy  Support 
Programme of the CIP in 2009. 

It also recognises some weaknesses identified by the professional evaluators, notably 
on  the  time  from  closure  of  call  to  contract  award  and  long  project  time-scales.  It 
notes  that  these  weaknesses  have  however  had  a  minor  effect  on  the  overall 
programme  implementation:  The  Commission  has  already  initiated  new  services 
which will reduce such delays in the future. 

It  welcome  the  positive  assessment  of  the  programme  design;  the  efficiency  of  the 
programme  management;  and  the  programme's  potential  to  deliver  sustainable 
results,  generate  positive  impacts  and  add  real  value  by  mobilising  all  of  the 
functions necessary to successful ICT deployment and catalysing structural changes. 
The  results  provide  also  evidence  that  the  Policy  Support  Programme  of  the 
Competitiveness  and  Innovation  Programme  (CIP)  can  be  expected  to  build  on  the 
results of eTEN. 

It considers that implementing such a unique set of funding mechanisms has been a 
particular  challenge  for  all  actors  involved,  and  welcomes  that  the  evaluators  have 
recognised the efforts which resulted in major improvements in the late stages of the 
programme. 

5. 

CONCLUSIONS OF THE COMMISSION 

The  Commission  considers  that  this  is  a  very  positive  report  confirming  both  the 
effectiveness of the programme management and the utility of the programme. Some 
weaknesses  have  been  identified  but  these  have  had  a  minor  effect  on  the  overall 
programme implementation.  

Immediate  impacts  are  the  involvement  of  stakeholders  from  New  Member  States, 
SMEs  and  public  bodies  favouring  the  further  deployment  and  uptake  of  project 
outputs at a pan-European level and the competitive health of markets for these and 
related services. Based on the results so far the programme may thus reasonably be 
expected to produce strong overall impacts by stimulating new areas of activity and 

EN 

4 

   EN 

 
demonstrating the ways in which ICTs may contribute to effective policy relating to 
innovation  and  the  Lisbon  Agenda  and  i2010  objectives.  The  results  provide 
evidence that the follow on programme, the Policy Support Programme of the CIP, 
can be expected to build effectively on the results of eTEN.  

Work  will  continue  into  2009  by  both  the  evaluators  and  the  Commission  to 
determine  the  impact  of  projects  from  later  years  of  the  eTEN  Programme.  The 
Commission will report on this work in the midterm evaluation of the Policy Support 
Programme of the CIP. 

EN 

5 

   EN 

 
Annex 1: 

Key Findings from the Evaluator's Report 

RAND Europe has  undertaken a final  evaluation of the eTEN programme. This programme 
nominally  finished  at  the  end  of  2006  although  some  project  activities  contracted  late  in  its 
life continue. Key findings of the evaluation are as follows:- 

Overall success of the Programme 

eTEN  was  ultimately  a  well  run  and,  in  its  later  stages,  successful  programme,  which 
contributed to the attainment of its over-arching objectives. This finding is, however, qualified 
in two ways. Success was not achieved without difficulties, particularly in the early years of 
the programme. Additionally the extent of its success is not quantifiable – in part because its 
impacts are not yet fully visible and in part because its overarching objectives were expressed 
in qualitative terms without specific measurable success criteria. 

Programme Implementation Mechanisms  

Funding  mechanisms  other  than  grant  aid  were  not  utilised:  this  is  unsurprising  given  the 
modest  size  of  the  projects,  and  we  do  not  consider  that  this  significantly  affected  the 
programme’s  performance.  Linkage  to  the  European  Regional  Development  Funds  and  the 
European Social Fund was weak, despite the apparent scope for benefits.  

The Market Validation (MV) project mechanism was initially over-utilised (in part because of 
its  more  favourable  financial  arrangements  for  participants);  this  was  remedied  in  the  later 
stages of the programme. Many MV projects commissioned before the reorientation did not 
result in any visible subsequent deployment; this situation also improved in the later stages of 
the programme. 

The  Initial  Deployment  (ID)  project  mechanism  came  on  stream  late,  but  worked  well, 
particularly following the much-delayed implementation of the 30% cap on overall support as 
a  proportion  of  total  cost.  Some  promising  deployments  are  now  in  progress  and  prospect, 
including some originating from earlier MV projects.  

The funding contracts allowed for insufficient flexibility. Project reviews were consequently 
limited in their ability to either direct or sanction necessary major changes of direction during 
project  implementation.  In  interviews,  some  project  managers  reported  an  impression  that 
they  had  no  realistic  means  of  modifying  this  situation  even  when  the  problem  had  been 
recognised. 

Efficiency of Programme Management 

From  2003  no  major  management  issues  arose  and the standards of management processes; 
fairness,  openness  and  transparency,  and  efficiency  of  day-to-day  management  were 
excellent. 

The  operational  procedures  for  preparing  Work  Programmes  and  Calls,  their  accompanying 
documentation and the receipt and evaluation of proposals improved continuously throughout 
the  programme.  Significant  initial  problems  (particularly  with  proposal  evaluation)  were 
overcome  and  these  mechanisms  were  subsequently  satisfactory  and,  by  the  end  of  the 
programme, became excellent.  

EN 

6 

   EN 

 
Objective  formulation  and  its  reflection  in  work  programme  creation  were  sound  and 
appropriate: we found no reason to criticise the evolution of objectives and indeed consider 
this responsiveness to change in the programme’s external market (and policy) environment a 
very positive feature. 

The  process  leading  to  contracting  projects  (a  sequence  consisting  of  selection,  credential 
checking, and negotiation of the technical annexes to contracts) took too long. This had been 
noted  as  a  problem  by  the  Intermediate  Evaluation,  and  continued  thereafter.  In  addition, 
project time-scales were long relative to private-sector counterparts and the dynamics of fast-
moving sectors. This, together with the delays in contracting projects, frequently resulted in a 
very  long  ‘time  to  market’,  which  is  likely  to  have  worked  against  successful  deployments 
and reduced their potential impacts. 

Effectiveness, Achievements and Impacts 

The  programme  made  considerable  progress  in  involving  stakeholders  from  New  Member 
States, SMEs and public bodies. Their participation strongly favours the further deployment 
and uptake of project outputs at a pan-European level and the competitive health of markets 
for these and related services. 

Activities in the latter stage of the programme promise good impacts through the formation of 
sector-based value chain communities (especially in the public sector). The emphasis placed 
in  the  latter  stages  of  activity  on  inclusion  in  projects  of  the  full  value  chain  including 
‘sustaining  partners’  and  ‘sustaining  revenue  streams’  was  fully  validated  and  represents  an 
important lesson for future programmes. 

Fulfilment of Programme Objectives 

The later stages of the programme we find to have added real value through actions seeding 
and/or  exemplifying  structural  change  and  also  through  placing  actual  deployment  at  the 
centre  of  policy  development.  The  programme  may  thus  reasonably  be  expected  to  produce 
strong  overall  impacts  by  stimulating  new  areas  of  activity  and  demonstrating  the  ways  in 
which ICTs may contribute to effective policy relating to innovation and the Lisbon Agenda 
and i2010 objectives.  

Sustainability of Activities 

Long-term  sustainability  of  activities  is  difficult  to  assess  at  this  stage.  However,  the 
involvement of the whole value chain (adopted in the later and more productive stages of the 
programme), and the specific example provided by some of the projects that have proceeded 
into wider deployment via these value chains, strongly suggests a positive prognosis. 

EN 

7 

   EN