Report from the Commission to the Council - Final Report on the second phase of the Tempus programme (1994-2000)
Date: 2004-02-16
Institutions: European Commission
Subjects: CIS countries, Central and Eastern Europe, EU programme, higher education, report

COMMISSION OF THE EUROPEAN COMMUNITIES 

Brussels, 16.02.2004 
COM(2004) 109 final 

REPORT FROM THE COMMISSION TO THE COUNCIL 

Final report on the second phase of the Tempus programme (1994-2000) 

EN 

   EN 

 
 
 
  
1. 

INTRODUCTION 

This  report  is  being  presented  pursuant  to  the  provisions  of  Council  Decision  96/663/EC  of 
21/11/1996 (OJ L 306, 28/11/1996), amending Decision 93/246/EEC and extending the second phase 
of the Tempus programme until 2000. The Commission is thereby required to present a final report on 
the second phase of the programme by 30 June 2004 at the latest. This report examines the impact of 
the Tempus programme in the countries eligible to take part and its role in stimulating the reforms that 
have taken place in those countries. It follows the Tempus II Programme Interim Report1 presented by 
the  Commission  in  March  1998  and  it  is  based  on  the  findings  and  conclusions  of  the  independent 
evaluator’s report. 

The report has the following structure: 

1. 

2. 

2.1. 

2.2. 

3. 

3.1. 

3.2. 

4. 

Introduction...............................................................................................................................2 

Tempus: A programme evolving over time ..............................................................................3 

The launching of the programme ..............................................................................................3 

From Tempus I to Tempus II ....................................................................................................4 

Overview of the external evaluation exercise ...........................................................................5 

The Terms of Reference............................................................................................................5 

The evaluation approach ...........................................................................................................7 

The Main Conclusions of the external evaluation.....................................................................8 

4.1. 

The programme outcomes.........................................................................................................8 

4.1.1. 

Impact on legislation.................................................................................................................8 

4.1.2. 

Contribution to HE reform........................................................................................................9 

4.1.3. 

Contribution to human resource development ........................................................................11 

4.1.4. 

Towards a sustainable partnership? ........................................................................................11 

4.2. 

Programme management ........................................................................................................12 

5. 

Summary of conclusions.........................................................................................................13 

List of documents ...................................................................................................................................14 

Annexes ...............................................................................................................................................15 

1 

COM(1998) 379 

2 

 
 
  
 
                                                 
2. 

TEMPUS: A PROGRAMME EVOLVING OVER TIME 

2.1. 

The launching of the programme 

Over  more  than  a  decade,  the  TEMPUS  Programme  has  provided  assistance  to  partner  countries 
reforming or restructuring their higher education systems by supporting co-operation with the Member 
States. The programme is based on the assumption that higher education institutions are of particular 
importance for the social and economic transition process as well as for cultural development; they are 
also  pools  of  expertise  and  of  human  resources  and  provide  for  the  training  of  new  generations  of 
political leaders. 

It is perhaps  worth emphasizing that, while the Tempus programme is fundamentally geared toward 
assisting  partner  countries,  the  programme  also  gives  EC  member  states  better  access  to  areas  of 
knowledge  where  some  of  these  countries  have  reached  at  high  level  of  development  and  creates 
enduring partnerships, thus ensuring mutual benefit. 

The Tempus programme was first proposed at the meeting of the Council of Ministers of Education in 
December 1989, the very year of the fall of the Berlin wall, an event which gave rise to the process 
that would eventually completely re-draw the map of Europe. 

The European Community seized this historic opportunity and launched two programmes (Phare2 and 
later  Tacis3)  to  assist  the  Central  and  Eastern  European  countries  (CEEC)4  and  the  countries  of  the 
former Soviet Union (plus Mongolia) in their transition towards democracy and market economies and 
to ensure political stability in the region. 

Education  was  perceived  as  one  of  the  pillars  of  the  transition  process.  The  reform  and  overhaul  of 
partner  countries’  education  structures  and  systems  was  necessary  to  prepare  citizens  for  the 
challenges of a competitive economy and for a multiparty system where civil society was to play an 
ever greater role. 

As  part  of  Phare,  the  European  Community  adopted  the  first  phase  of  the  Tempus  programme 
(Tempus I) in May 1990. 

The  programme  operated  initially  in  the  CEEC  and  between  1992  and  1993  it  was  extended  to  the 
Russian  Federation,  Ukraine,  Belarus  and  to  Albania.  For  a  short  period  (1991)  it  operated  in 
Yugoslavia as well. The budgets allocated to the programme were decided on a national basis within 
the overall budgets of the Phare and Tacis programmes. 

The  programme  was  designed  to  promote  multilateral  co-operation  between  higher  education 
institutions  in  the  Member  States  and  in  the  partner  countries  thereby  encouraging  the  transfer  of 
know-how with a view to stimulating: 

– 

– 

2 
3 
4 

Academic  development  and  improvement,  including  the  development  of  new  curricula, 
pedagogic methods, didactic material, etc. 

The opening-up of institutions to international co-operation (including greater inter-regional 
co-operation)  in  general  and,  in  particular,  to  co-operation  within  the  emerging  European 
education area. 

Council Regulation EEC No 3096/89 of 18/12/1989, OJ L 375, 23/12/1989, p. 11-12 
Council Regulation (Euratom, EC) No 1279/96 of 25/06/1996, OJ L 165, 04/07/1996, p. 1-11 
i.e. Bulgaria, the Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Slovakia and 
Slovenia 

3 

 
 
  
 
                                                 
– 

– 

The  acquisition  of  management  skills  by  academic  and  administrative  staff  thus  ensuring 
the  growing  demand  for 
institutions’  newly  granted  autonomy, 
balance  between 
accountability and the need for greater financial sustainability. 

The opening up of partner countries’ education systems to civil society. 

From  the  outset  the  programme  has  supported  partnerships  involving  networks  of  higher  education 
institutions  from  both  the  EC  and  partner  countries.  These  partnerships  (Joint  European  Projects  – 
‘JEPs’) have constituted the main type of project throughout the programme lifetime. They provide the 
framework for a wide range of specific outputs such as: 

– 

– 

– 

– 

– 

New curricula, teaching methodologies and didactic materials; 

Training for university management and staff; 

Policy papers and recommendations; 

New structures within institutions, such as services for international relations; 

Computerisation of university facilities, and libraries in particular. 

Next to the JEPs, Individual Mobility Grants (IMG) are a fundamental type of project within Tempus. 
IMGs have provided more than 15,000 staff with internships and periods of study abroad. 

2.2. 

From Tempus I to Tempus II 

Although its architecture and intervention logic have remained to a very large extent unchanged, the 
Tempus programme has significantly evolved, accompanying the changes in the political context and 
contributing  to  the  process  of  accession  of  the  CEEC  to  the  European  Union.  The  experience 
accumulated  by  the  programme  stakeholders  also  offered  significant  pointers  to  the  way  ahead  for 
Tempus in other geopolitical areas (see the recent extension of the programme to the Mediterranean 
countries). 

A  key  characteristic  of  the  programme  from  the  outset  was  its  «bottom-up»  approach  within 
Universities. The programme initially allowed working teams operating at Faculty or Department level 
to  submit  project  proposals  without  direction  from  the  top,  generating  a  high  motivation  to  engage 
with change, a motivation which was matched by a similar desire on the side of the Western partners 
to contribute to the reform process. A dynamic process was launched that at first was very positive in 
terms of getting the Programme off the ground. Later however, it became necessary to introduce into 
the programme a greater degree of planning at institutional level. 

As  a  result  the  Council  decisions  of  93/246/EEC  and  96/663/EC  on  Tempus  II  and  Tempus  II  (bis) 
provided not only for continuation of the basic approach and an extension of the programme to new 
eligible  countries  in  the  Tacis  and  Phare  area,  but  also  increased  the  expectation  levels  for  the 
programme. 

In  particular  Tempus  II  introduced  specific  national  priorities  and  this  brought  a  top-down  (or 
strategic) component into the programme. The national priorities gave the higher education authorities 
in the eligible countries and the Commission an instrument for guiding the Tempus applicants and the 
Tempus  activities  (funded  from  their  national  Phare  and  Tacis  budgets)  more  firmly  towards  the 
longer-term systemic reform needs of higher education as they perceived them. Tempus II aims thus 
went  beyond  the  mobility  objective  and  bottom-up  innovation  of  curriculum  development  and 
university management that characterised Tempus I. 

Tempus  II  bis  (see  Annex  I)  introduced  institution  building  as  a  new  field  of  work.  Under  Tempus 
Phare, "institution building" was linked to the accession process of the eligible countries. Institution 

4 

 
 
  
 
building projects were seen as instruments for using university co-operation and EU university know-
how  in  supporting  the  preparation  of  national  administrative,  economic  and  legislative  bodies  for 
operating  within  the  European  Union  by  training  their  staff  for  the  adoption  of  the  ‘acquis 
communautaire’,  the  accumulated  mass  of  European  legislation.  With  this  instrument,  the  Tempus 
programme opened up for involving non-academic institutions in the projects. 

Other  new  features  of  the  second  phase  that  were  introduced  to  help  the  programme  meet  high 
ambitions included the introduction of compact measures (Phare) or compact projects (Tacis). These 
were  two  types  of  activity  (next  to  JEPs  and  IMGs)  designed  as  new  instruments  for  more  targeted 
actions in the field of university management reforms and institution building. 

Despite  the  changes  introduced  during  the  second  phase  of  the  programme  and  its  ambitious 
reorientation  towards  a  more  strategic  approach  and  the  opening  to  the  civil  society,  the  bottom-up 
image  of  the  programme  remained  largely  unchanged  and  the  overall  majority  of  the  projects 
continued to address the reform of curricula. The reorientation might probably have been more visible, 
had  it  been  accompanied  by  an  enhanced  financial  contribution.  As  a  matter  of  fact,  during  the  six 
years of the second phase of Tempus (1994-1999), the annual budget was reduced to less than 40% of 
the initial figure (see Annex II).  

It  is  useful  to  emphasise  at  this  point  the  different  level  of  funding  in  the  years  1994-1999  in 
proportion to the size of the higher education sector in the respective partner countries. In the Phare 
countries  the  vast  majority  of  the  higher  education  institutions  have  participated  in  one  or  more 
Tempus projects. In Russia, on the contrary, only 100 out of some 800 higher education institutions 
have participated in Tempus. Out of a total of 1386 funded JEPs, 1120 were in the candidate countries 
of  the  Phare  region,  184  in  Tacis  countries  and  82  in  the  countries  of  former  Yugoslavia  (plus 
Albania). 

In terms of financing, Tempus Phare equalled some 30% of national spending on higher education in 
the partner countries. In Russia this percentage was less than 1 %. 

These  crude  financial  figures  should  always  be  kept  in  mind  in  what  follows  and  make  us  expect  a 
more  pervasive  impact  in  the  Phare  countries,  whereas  in  Russia,  for  instance,  one  might  expect  a 
more patchy effect. 

3. 

OVERVIEW OF THE EXTERNAL EVALUATION EXERCISE 

3.1. 

The Terms of Reference 

Since the adoption of Tempus I, the programme has undergone three evaluations: 

– 

– 

– 

The evaluation of the first phase 1990-1994; 

Mid-term evaluation of the second phase 1994-1996; 

Evaluation of the Tempus programme in the countries eligible under Tacis5. 

Pursuant  to  the  provisions  in  the  Council  decisions  on  Tempus  II  bis,  the  Commission,  (DG  EAC) 
launched an open call for tender for the provision of an evaluation report on the second phase of the 
Tempus programme (1994-1999). 

The tender was won by ECORYS Nederland BV and the contract for this assignment was signed in 
October 2002. The evaluation was carried out between October 2002 and July 2003 culminating in a 

5 

Council Regulation (Euratom, EC) No 1279/96 of 25/06/1996, OJ L 165, 04/07/1996, p. 1-11 

5 

 
 
  
 
                                                 
‘lessons  learned’  workshop  in  Brussels  involving  a  significant  representation  of  programme 
stakeholders. The final report was submitted to the Commission in October 2003 and finally approved 
in December 2003. 

This report sets out the findings of the external evaluation. 

The  Terms  of  Reference  for  the  assignment  specified  that  the  evaluation  aimed  to  serve  the 
Commission in formulating a well founded judgement on the medium-term impact of the programme 
and on the extent to which the second phase of the Tempus programme had produced the outcomes 
expected. 

In  the  light  of  the  provisions  of  the  relevant  Council  decisions  indicated  above,  the  evaluator  was 
requested to judge the extent to which the Tempus programme produced the following outcomes: 

1. The extent to which the programme has contributed to changes in legislation in eligible countries 
leading to: 

– 

– 

– 

(where appropriate) greater convergence of their education systems with those in the 
EC, 

education systems better adapted to the changing socio-economic reality, in particular 
market economies and multiparty political systems, 

(specifically  for  pre-accession  countries)  the  transposition  into  national  law  of  the 
acquis  communautaire,  particularly  in  so  far  as  the  regulated  professions  are 
concerned. 

2.  The  extent  to  which  the  programme  prompted  reforms  in  higher  education  institutions  (including 
polytechnics and technical schools) leading to: 

– 

– 

institutions' openness and preparedness for international co-operation, 

new  management  approaches  and  structures  in  line  with  more  independent, 
accountable higher institutions. 

3.  The  extent  to  which  the  Tempus  programme  has  brought  about  curriculum  development  and 
reforms leading to the acquisition by staff and students, of skills and qualifications necessary for the 
transition process and for the new socio-economic reality. 

4. The extent to which the Tempus programme has contributed to the training of decision makers in 
eligible countries and the extent to which staff with experience in Tempus played and continue to play 
a role in universities in partner countries. 

5.  The  extent  to  which  the  Tempus  programme  has  encouraged  sustainable  partnerships  in  higher 
education involving EC and partner countries. 

The evaluator summarised the above in the following eight expected outcomes: 

1.  Contribution  to  legislative  changes  leading  to  greater  convergence  of  education  systems  in  the 
eligible countries with those in the EC 

2. Contribution to legislative changes leading to better adaptation of education systems to the changing 
socio-economic  reality  of  market  economies  and  multi-party  political  systems  (curriculum 
development, mobility and ECTS) 

3. Contribution to the transposition into national law of the acquis communautaire, particular in so far 
as the regulated professions are concerned 

6 

 
 
  
 
4.  Reforms  prompted  in  Higher  Education  Institutions  (HEIs)  leading  to  institutions'  openness  and 
readiness for international co-operation 

5. Reforms prompted in HEIs leading to new management approaches and structures in line with more 
independent, accountable higher institutions 

6. Curriculum development and reforms leading to the acquisition by staff and students of skills and 
qualifications necessary for the transition process and for the new socio-economic reality (curriculum 
development and employability) 

7.  Contributions  to  the  training  of  decision-makers  (institution  building)  and  retention  of  Tempus 
experienced staff in universities (IMGs) 

8. The development of sustainable partnerships between HEIs in eligible countries and EU countries 

3.2. 

The evaluation approach 

The  evaluators  proposed  a  review  of  the  relevant  literature  on  the  programme.  This  included  the 
previous evaluation reports, the Tempus Impact Studies produced at the national level within the EC 
and  the  eligible  countries,  and  the  report  produced  by  the  European  Training  Foundation  Tempus 
Department on the Tempus contribution to Accession (a report limited to Hungary, Poland, Bulgaria 
and Romania). 

The  information  so  gathered,  however,  would  have  been  clearly  insufficient  for  the  purposes  of  the 
evaluation.  In  order  to  complement  it  with  more  structured  information  of  both  quantitative  and 
qualitative  nature,  the  evaluators  built  three  questionnaires,  which  focused  on  ex-post  assessment  of 
project outcomes and programme impact: 

– 

– 

– 

an online questionnaire for JEP co-ordinators; 

a questionnaire for structured interviews of JEP participants in the eligible countries; 

a  questionnaire  for  structured  interviews  with  higher  education  authorities  in  the  eligible 
countries 

The questionnaires were designed to provide the answers to the expected outcomes and the elements 
to  support  the  findings.  The  response  rate  to  the  online  questionnaire  was  43%  and  was  considered 
quite satisfactory by the evaluator. Due to the use of national experts engaged  by ECORYS for this 
assignment  the  structured  interviews  were  successfully  completed  in  100  percent  of  the  cases.  The 
contractor organised the meetings and conducted the interviews; likewise higher education authorities 
were interviewed in 20 countries. 

The approach chosen by the evaluator was certainly determined by the limits imposed in order to keep 
the study within the proposed budget. There are two comments on the selected approach. The first is 
about the choice of the evaluator of making limited use of the JEPs Final Reports. These provide first 
hand  information  on  the  results  achieved  by  the  single  projects  and  it  is  believed  that  an  in-depth 
analysis might have usefully complemented the findings of the questionnaires and of the interviews. 
This  deficiency  is  partially  remedied  by  the  use  the  evaluator  made  of  the  ETF  report,  which  was 
precisely  based  on  the  Final  Reports  but  focussed  on  accession  issues  only  and  was  limited  to  four 
countries.  The  second  consideration  is  that  the  respondents  were  not  necessarily  always  best 
positioned to give answers to all questions. Several examples of this will be given in the following. 

7 

 
 
  
 
4. 

THE MAIN CONCLUSIONS OF THE EXTERNAL EVALUATION 

4.1. 

The programme outcomes  

This section outlines the expected outcomes and the corresponding impact found by this evaluation. 
The Commission’s comments are presented as well. 

4.1.1. 

Impact on legislation 

The request contained in the Terms of Reference to judge the extent to which the Tempus Programme 
contributed to changes in legislation (Outcome 1) may appear surprising, as legislative reform is not 
and has never been the primary objective of the Tempus programme. As a matter of fact the lack of 
direct  influence  of  the  Programme  on  the  higher  education  reform  has  often  been  flagged  up.  This 
criticism  has  been  partly  echoed  in  some  previous  programme  evaluation  reports.  For  example  the 
1998 ‘Evaluation of the Tempus Programme in the NIS and Mongolia’, by FTP International stated: 
“So far, the Tempus programme has had no visible impact on the national higher education policies of 
the individual partner countries, nor on education legislation”. This lack of impact at the political level 
was generally considered as the inevitable downside of the bottom-up philosophy of the programme. 
With  faculties  and  departments  as  main  actors  –so  the  idea  was-  the  effects  of  Tempus  on  system 
reforms, if any, could only be long-term and indirect. Such opinion was shared by many programme 
stakeholders and taken for granted. In fact some even believed that, after all, this was the maximum an 
EU-funded  co-operation  programme  could  possibly  achieve.  In  fact  it  was  rather  doubtful  that  top-
down  measures  would  have  had  a  strong  and  immediate  effect,  given  the  widespread  scepticism 
towards authority and top-down initiatives in the aftermath of the fall of communism. 

However,  by  giving  the  evaluator  the  mandate  to  assess  this  outcome  it  was  the  Commission’s 
intention  to  investigate  whether,  in  the  medium  term,  a  more  explicit  link  than  was  previously 
acknowledged, might be established between the Tempus activities and the reform process ongoing in 
most  of  the  programme  partner  countries.  The  findings  of  the  final  evaluation  report  seem  partly  to 
change the picture described above. Of course it is tempting to ascribe this to the reorientation of the 
programme towards a  more strategic  approach. The  report gives some interesting clues that seem to 
confirm this view. Tempus appears to have been especially important for sensitising policy makers to 
the need for legislative and regulatory reforms. 

The  Russian  test  case  presented  in  the  report  deserves  a  closer  look.  Evidence  is  provided  here  that 
Tempus results were made available to the so called ‘Teaching and Teaching Methods Associations’, 
special bodies responsible for the development of education standards based on a bottom-up approach. 
With  the  help  of  further  supporting  evidence  gathered  through  structured  interviews  with  the 
competent authorities, the evaluator reached a conclusion apparently at odds with the one expressed 
above. The evaluation found that Russian academics who participated in Tempus, have subsequently 
been  able  (in  particular  through  the  academic  support  groups),  to  influence  the  new  strategy  papers 
and eventually orient the recent legislation characterised by a strong European flavour. This striking 
contradiction  between  the  current  and  previous  findings  may  have  several  explanations.  The  most 
obvious but probably also the most plausible is that in 1998 when the FTP report was issued, it was 
not yet possible fully to assess the extent of a ‘trickle-up’ effect, which is intrinsically slow and takes 
place under the surface. 

Nevertheless  some  other  elements  should  also  be  taken  into  account.  First,  the  conclusion  of  the 
positive  impact  of  the  programme  on  legislative  changes  was  reached  not  only  on  the  basis  of 
structured interviews but also on the basis of the online questionnaire. As the evaluator points out, all 
the  respondents  were  project  co-ordinators,  that  is  persons  directly  involved  in  the  projects  and 
responsible for the academic content. Some positive bias in the answers is therefore to be expected. 

Second,  despite  the  efforts  of  the  evaluator  to  avoid  misunderstandings,  respondents  interpreted  this 
particular  question  regarding  the  programme  impact  in  rather  different  ways.  Respondents  gave  a 
range of examples including changes in rules and regulations at faculty, university and national level. 

8 

 
 
  
 
This also explains an ancillary statement made by the evaluator that is that in the countries of former 
Yugoslavia  (plus  Albania)  no  influence  beyond  faculty  level  was  reported.  In  countries  where  the 
rector  is  basically  a  figure  head  and  deans  traditionally  negotiate  their  budgets  directly  with  the 
responsible  ministries,  this  comes  as  no  surprise.  Moreover  it  must  not  be  forgotten  that  in  all 
countries of former Yugoslavia the programme was  introduced well into the Tempus II phase and  a 
fair impact assessment is probably not yet possible (in very much the same way as it was premature to 
do it for Russia in 1998). 

All of the above allow us to conclude that the programme reorientation towards a more strategic 
approach has made the impact of the programme on legislation a reality, but (consistently with 
the  other  findings  hereafter)  it  is  fair  to  say  that  the  image  of  the  programme  characterised  by 
mainly a bottom-up approach has remained unchanged to a large extent. 

The  findings  related  to  the  influence  of  the  programme  on  the  adaptation  of  the  higher  education 
systems to the changing socio-economic needs (Outcome 2) are somewhat variegated. The influence is 
said to have been ‘major direct’ by the majority of the higher education authorities in the Phare area, 
in Russia the influence was found to be ‘minor indirect’. In the smaller Tacis countries both ‘minor 
direct’  and  ‘major  indirect’  were  popular  answers  in  the  survey.  It  is  difficult  to  give  a  unique  and 
straightforward  explanation  for  these  differences.  A  few  hypotheses  are  suggested  here.  First,  the 
distinction  between  direct  and  indirect  influence,  and  between  major  and  minor  influence  might 
simply  be  inappropriate  for  this  particular  question.  Second,  one  should  not  forget  that,  as  it  was 
already  mentioned,  project  co-ordinators  played  an  important  role  in  the  survey.  In  current  Phare 
projects,  according  to  Tempus  rules,  project  co-ordinators  may  belong  to  both  the  partner  countries 
and the EU, whereas in Tacis project co-ordinators need to belong to the EU. It is hardly surprising 
that  such  a  qualitative  judgement  turns  out  to  be  highly  dependent  on  the  cultural  and  political 
background and also on the role and the perspective of the respondents. 

The  Commission  agrees  with  the  evaluator’s  conclusion  (reached  by  averaging  out  all  respondents 
from  all  regions)  that  Tempus  II  had  a  major  indirect  impact  on  adaptation  to  socio-economic 
circumstances. 

The  contribution  of  the  programme  to  the  harmonisation  of  curricula  for  the  seven  regulated 
professions  covered  by  the  EC  directives  was  a  less  controversial  issue  (Outcome  3).  The  survey 
shows  that  almost  half  of  the  Phare  projects  dealt  somehow  with  regulated  professions,  mainly 
introducing best practices and curriculum reforms on the related subjects (in medicine and veterinary 
studies  more  than  others),  even  if  an  in-depth  analysis  of  the  Tempus  contribution  to  the 
implementation  of  the  EC  directives  was  not  performed  by  the  evaluator.  It  is  maybe  worth 
mentioning  that  even  if  the  ToR  explicitly  limited  the  question  to  pre-accession  countries,  the 
contribution  of  Tempus  to  veterinary  studies  (one  of  the  most  expensive  faculties  to  run!)  in  the 
countries  of  former  Yugoslavia  can  hardly  be  underestimated.  For  example,  in  the  former  Yugoslav 
Republic of Macedonia there was no faculty of veterinary previous to the break up of the Yugoslav 
Republic and the Faculty of Veterinary in Sarajevo was totally destroyed by the war. In both cases, the 
Tempus  contribution  in  the  set  up  and  in  the  reconstruction  of  the  Faculty  has  been  broadly 
acknowledged. 

The  Commission  agrees  with  the  evaluator’s  acknowledgement  of  the  Tempus  contribution  to 
the reform of the curricula leading to the regulated professions. 

4.1.2.  Contribution to HE reform 

The  substantial  contribution  of  the  Tempus  programme  to  the  openness  and  readiness  for 
international  co-operation  (Outcome  4)  is  one  of  the  undisputed  findings  of  the  external 
evaluation report. The project co-ordinators’ survey clearly spells out the importance of the second 
phase  of  the  programme  for  the  computerisation,  increased  access  to  and  use  of  information  and 
communication  technologies  in  classrooms;  for  the  exchange  of  visits  of  students  and  staff;  for  the 
access to international scientific literature; for the adoption of textbooks and teaching material in an 

9 

 
 
  
 
EU  language;  for  the  introduction  of  teaching  and  training  in  an  EU  language;  and  finally,  for  the 
establishment of international offices in higher education institutions. The impact has been defined as 
‘major direct’ by the majority of the respondents in all geographical areas. A non-negligible quota of 
respondents assessed the impact as minor only in the Phare countries and in Russia. This can easily be 
explained  in  Phare  countries  due  to  the  process  of  internalisation  starting  early  and  with  the  second 
phase  of  Tempus  offering  less.  In  Russia  on  the  other  hand,  the  perception  of  a  minor  impact  is 
certainly due to the lower level of penetration of the programme due to a proportionally lower budget 
in comparison with the size and needs of the higher education sector. 

The  Commission  considers  the  conclusions  of  the  evaluator’s  report  on  this  particular  issue  as  yet 
another indication of the validity of the overall programme design. 

The  impact  of  the  programme  on  university  management  (Outcome  5)  turned  out  to  be  a  relatively 
controversial issue and the feedback provided by the evaluator may appear as rather inconclusive. UM 
projects  represented  approximately  10%  of  all  Joint  European  Projects  and  therefore  the  evaluator 
perceived the clear risk of statistical insufficiency, if evidence were gathered only from the few UM 
projects  included  in  the  sample.  Therefore  the  option  was  taken  to  extend  the  survey  to  all  JEPs 
claiming  to  have  had  some  impact  on  university  management  (even  if  their  primary  focus  was 
different,  notably  curriculum  development).  The  Commission  tends  to  agree  with  the  evaluator’s 
conclusion that the impact was not negligible but that a systematic focus on university management 
might  have  multiplied  the  risks  of  clashes  between  old-fashioned  structures  and  pressure  for  reform 
and that the gentle approach chosen was probably the best to achieve the maximum relative impact. 

One has to take into account however, that the UM type of projects were unevenly distributed across 
the different geographic areas. Almost absent from the Tacis countries, they were particularly popular 
in the last two years of Tempus II in some Phare countries. For example in Poland 51 % of the total 
number of proposals submitted in 1998 was of the University Management type, whereas in Slovenia 
the percentage was 15 % only. This might be an indication of a different level of readiness to carry out 
reform in the various countries. One can also easily ascertain that rather different types of projects are 
in fact grouped under the UM label: from equipment provision to faculty or central libraries and the 
establishment  of  management  information  systems  to  the  development  of  strategic  planning  and  the 
introduction  of  democratic  principles  in  university  governance.  There  was  a  general  consensus  in 
acknowledging  the  impact  on  the  more  material  domains  (computerisation,  update  of  services  and 
facilities, provision of equipment, etc.). On the other aspects opinions tended to diverge. The higher 
education  authorities  emphasised  in  their  answers  the  Tempus  impact  in  their  own  field  of 
competence, that is the decision making process, whereas the JEP participants underlined the internal 
reform  aspects.  In  some  cases  the  impact  was  perceived  to  have  been  a  side  effect  of  Tempus 
activities. 

In conclusion, the findings confirm a mixed response partly due to the sensitivity of the issue for the 
people  involved  in  the  survey,  and  partly  for  the  objectively  heterogeneous  situations  in  which  the 
programme operated. 

Outcome 6 focussed on the programme impact on curriculum development. Curriculum development 
projects have been the core elements to Tempus and therefore the findings of the evaluation report can 
be  considered  of  the  highest  interest.  A  significant  component  to  these  projects  is  staff  and  student 
exchanges. The report clearly states the significance of this Tempus component: the mere exposure 
of  thousands  of  students  and  staff  to  new  ideas  and  social  environments  is  recognized  to  have 
tremendously contributed to raising the awareness of national and societal values, stimulating at 
the same time the dialogue between cultures, critical thinking and changes of attitude towards 
study, work and life in general. 

More specifically curriculum development projects had the objective of updating and restructuring the 
contents of existing courses and introducing entirely new curricula. In Tacis countries modernisation 
and  restructuring  of  courses  and  curricula  occurred  most  frequently.  In  Phare  countries  focus  was 
more  on  the  introduction  of  new  subjects,  with  a  marked  preference  for  interdisciplinary  subjects. 

10 

 
 
  
 
Regarding  the  relevance  of  degree  programmes  to  the  needs  of  the  labour  markets,  Tempus 
contributed  by  responding  to  the  emerging  demand  for  new  competencies,  even  with  limited 
involvement of the different economic actors. 

The perceived importance of Tempus in the different groups of eligible countries was highlighted by a 
specific question put by the evaluator. Asked whether the curriculum development changes introduced 
thanks to Tempus would or would not have occurred without Tempus, responses significantly varied 
depending  on  the  areas  where  the  programme  operated.  For  example,  in  Tacis  countries  a  large 
proportion of respondents indicated that many of the changes would not have taken place at all, while 
in the candidate countries of Phare the prevailing view was that reforms would have occurred sooner 
or later, but that Tempus speeded up the process. Finally, in the countries of the former Yugoslavia 
and  in  Albania,  people  thought  that  without  Tempus  reforms  would  probably  have  occurred  much 
later, with many adding that ‘later’ would probably have meant ‘too late’. 

The Commission is particularly pleased that the programme is broadly perceived as having had a 
positive  impact  on  the  employability  of  students  and  on  the  reputation  of  the  universities  and 
their ability to attract students. 

4.1.3.  Contribution to human resource development 

The  contribution  of  the  programme  to  the  training  of  decision-makers  was  to  be  analysed  under 
Outcome 7. The 2001 impact study of Tempus in Latvia – mentioned in the evaluation report - sheds 
light on the career development of a few participants in the programme. A list is provided that includes 
the head of the higher education council, the head of the Latvian Rectors’ Conference, several rectors, 
the head of department of higher education and research in the Ministry of Education and several other 
senior Ministry officials. Although an extended cohort study to track careers of former Tempus grant 
holders  was  never  undertaken,  the  evaluator  believes  (with  some  good  reasons)  that  Latvia  is  no 
exception  and  that  especially  in  small  countries  the  programme  had  a  tremendous  impact  on 
human  resource  development.  From  the  survey  it  clearly  emerges  that  the  experience  made  in 
Tempus  strengthened  the  academic  knowledge  of  the  participants  and/or  their  interpersonal  skills, 
contributing to their move to senior positions both inside and outside their academic institutions. This 
conclusion is important and it is a striking example of how the bottom-up approach can be effective in 
the medium term. 

The ability of universities to retain trained people was also investigated. Staff and student mobility has 
often been accused of increasing the risk of brain drain. No such evidence emerged from the survey. It 
must be observed however, that in this specific case the sample used (project co-ordinators and higher 
education authorities) was probably not very appropriate. A survey among former IMG grant holders 
could certainly have given more significant results. 

4.1.4.  Towards a sustainable partnership? 

The last outcome (Outcome 8) in which the evaluator was requested to provide a response was about 
sustainable  partnership,  a  key-issue  for  each  funded  co-operation  programme.  The  report  clearly 
indicates  that  enthusiasm  for  academic  co-operation  is  the  greatest  resource  of  the  programme.  The 
overwhelming majority of Tempus participants are prepared to stay in touch with their Tempus 
partners.  The  results  of  the  survey  are  a  strong  indication  that  more  than  half  of  the  JEP  consortia 
have indeed succeeded in building up a sustainable partnership. This is an impressive outcome by any 
standard. 

A caveat should perhaps be raised. In assistance programmes sustainability is defined as the ability to 
maintain  an  adequate  level  of  results  over  a  longer  period  of  time  after  termination  of  the  donor’s 
financial  support.  This  definition  needs  to  be  adapted  to  Tempus  specific  features.  As  a  partnership 
programme  Tempus,  to  a  large  extent,  can  be  described  as  a  co-operation  programme,  contributing 
more than just technical assistance. A  Tempus project can therefore be considered sustainable if the 
partnerships  (valuable  networks  of  academic  contacts  within  and  outside  the  EU  built  during  the 

11 

 
 
  
 
project)  remain  active  and  relevant  following  the  end  of  project  funding.  Among  the  forms  that 
sustainable partnerships have taken after completion of the projects, the most frequently mentioned in 
the survey were: academic networking and participation in conferences; exchange of academic staff; 
joint  research  and  publication;  and  regular  exchange  of  students.  Of  course  there  is  no  sustainable 
partnership  at  zero  cost,  but  the  evaluators  emphasised  that  even  if  nominally  funded  by  the  EC, 
Tempus projects have always been largely based on personal commitment and good will. 

A remarkable fact is that partnerships seem to have been most important in Tacis, where over 80% of 
respondents indicated to have maintained a form of sustainable partnership. A factor here may be the 
fact  that  in  this  area,  Tempus  was  and  is  the  only  programme  of  this  nature,  whereas  in  the  Phare 
countries,  universities  have  access  to  other  programmes  through  which  to  continue  international  co-
operation.  Many  in  fact  noted  the  Socrates  programme  as  the  most  natural  umbrella  under  which  to 
continue  the  co-operation.  University  budgets  were  also  indicated  as  a  main  source  of  funding  to 
continue co-operation (alongside bilateral programmes, multilateral organisations, private foundations 
and  new  Tempus  projects).  The  only  exception  in  this  rosy  scenario  is  perhaps  to  be  found  in  the 
countries of the former Yugoslavia (plus Albania) where higher education authorities do not seem very 
optimistic about sustainability of partnerships. 

An interesting complement to the above is the issue of incentives for EU universities to take part in 
Tempus, which was also investigated by the evaluator. The desire to assist the transition process was 
surprisingly ranked as high as the professional interest in joint curriculum development. Of course the 
interest for the institutions to receive EU funding was also quoted, suggesting that Tempus JEPs are 
apparently still considered a net source of income for EU universities. 

4.2. 

Programme management 

An assessment of programme management was not foreseen in the “Terms of Reference,” but with the 
Commission’s agreement, the evaluators decided to introduce a few questions on this aspect in their 
survey. 

With the online questionnaire, the following aspects were looked at: 

1. 

2. 

3. 

The information material provided to the applicants; 

The programme technical assistance and the project management reporting mechanisms; 

The  support  provided  by  the  National  Tempus  Offices  (NTOs)  or  Tempus  Information 
Points (TIPs). 

The  information  material  was,  in  general,  well  appreciated.  On  the  other  hand,  there  were  some 
criticisms  on  the  procedures  for  administrative  reporting  and  invoicing;  both  described  as  time 
consuming and excessively inclined towards control. Representatives from EU universities mentioned 
that  this  high  administrative  burden  was  perceived  as  one  of  the  main  barriers  for  new  universities 
participating in the programme. 

Appreciation for the support received from the National Tempus Offices varied significantly between 
the regions. High appreciation was marked in the ten acceding countries of the Phare area and in the 
countries of the former Yugoslavia (plus Albania). In the Tacis area, the answers show a lower level of 
satisfaction with 30% being dissatisfied with the support available. This result reflects the situation of 
the  Russian  Federation,  where  for  some  time,  Tempus  did  not  succeed  in  establishing  an  office. 
Having noted this however, it did not seem to have prevented an excellent response to the calls from 
the higher education institutions. 

In  general  these  results  are  in  line  with  expectations.  In  recent  years  there  has  been  a  significant 
effort to improve the administrative procedures and simplify the project cycle, and the effects of 
these improvements are anticipated to emerge gradually. 

12 

 
 
  
 
5. 

SUMMARY OF CONCLUSIONS 

The  Commission  agrees  with  the  evaluators’  findings  that  the  second  phase  of  Tempus  has 
demonstrated  the  flexibility  of  the  programme  and  its  ability  to  evolve  in  response  to  the  different 
socio-economic conditions and political perspectives in the various regions where it operates. It was 
also shown that dialogue with national authorities and the mechanism of setting national priorities is a 
valid  instrument  for  orienting  the  programme  towards  the  most  urgent  local  needs  and  making  sure 
that Tempus activities have a higher chance of embedding to sustain longer term reform. 

The external evaluators’ report clearly shows that Tempus II has significantly contributed to removing 
barriers to international mobility of students. The core element of the programme remains curriculum 
development. Tempus’s most durable impact has probably been on training people who will shape the 
higher  education  environment  for  decades  to  come,  and  on  providing  students  with  skills  and 
qualifications needed for employment and career development. 

The  Commission  is  particularly  pleased  that  the  programme  instruments  have  been  proved  to  be 
equally  appropriate  and  adaptable  to  different  environments.  In  Phare  countries  Tempus  II  has  been 
instrumental  in  opening  the  higher  education  sector  to  dialogue  and  co-operation  with  their  EU 
partners. It also addressed the needs of new skills linked to the process of economic reform and EU 
accession.  Finally,  it  has  contributed  to  positioning  them  in  the  best  situation  in  order  to more  fully 
reap the benefits of participating in other EU education programmes, notably Socrates. 

In the countries of former Yugoslavia (plus Albania) the participation in Tempus has offered a safety 
belt to education and science, during a period of sharply declining national resources for the sector. In 
former  Yugoslavia  in  particular,  outcomes  of  regional  co-operation  has  helped  to  mitigate  the 
devastating effect of the break-up of the previous state-based higher education system. 

Given  the  modest  financial  contribution  in  Tacis  countries,  the  Tempus  impact  on  curriculum 
development and reform is broadly recognised by project participants and national authorities to have 
gone beyond expectations in triggering momentum for change. 

This  report  is  meant  to  be  a  contribution  to  the  analysis  of  the  medium  term  impact  of  the  second 
phase of the Tempus programme. The Commission recognizes, however, that other aspects might be 
further investigated, for example the following: 

– 

– 

– 

– 

the process of embedding the newly developed courses into the regular curricula; 

the barriers opposing effective dissemination of Tempus results; 

the employability issue and the relations of Tempus with industry and the private sector; 

the impact of Tempus on the phenomenon of the ‘brain drain’. 

13 

 
 
  
 
LIST OF DOCUMENTS 

Final Report of the second phase of the Tempus Programme, by Ecorys 

Evaluation of the first two years of Tempus II, by Kehm, Mauworm, Over, Reisz, Steube, Teichler 

Evaluation of the Tempus Programme in the NIS and Mongolia, by FTP International 

Tempus @ 10, by Ard Jongsma 

From Association towards Accession, by Ewa Kolanowska, ETF (Turin) 

14 

 
 
  
 
Annex I 

Legislation 

Legal base 

Duration of the programme 

Council  decision  90/233/EEC 
of 7 May 1990 – Tempus I 

1990-1994 

Council  decision  93/246/EEC 
of 29 April 1993 – Tempus II 

1994-1998 

Council  decision  96/663/EC 
of  21  November  1996  – 
Tempus II bis 

Extends Tempus II until 2000 

Council  decision  99/311/EC 
of 29 April 1999 – Tempus III 

2000-2006 

Council  Regulation 
2666/2000 of 05/12/2000 

(EC) 

2000-2006 

15 

Partner  countries  (countries 
became  eligible  at  different 
times) 

Albania, Bulgaria, Czech 
Republic, Estonia, Hungary, 
Latvia, Lithuania, Poland, 
Romania, Slovenia, Slovakia, 
ex-DDR (1990 only), 
Yugoslavia (1991 only) 
Albania, Bosnia-Herzegovina, 
Bulgaria, Czech Republic, 
Estonia, Hungary, Latvia, 
Lithuania, Macedonia, Poland, 
Romania, Slovenia, Slovakia 
(Phare) 
Armenia, Azerbaijan, Belarus, 
Georgia, Kazakhstan, 
Kyrgystan, Moldova, 
Mongolia, Russia, Tajikistan, 
Turkmenistan, Ukraine, 
Uzbekistan (Tacis) 
Albania, Bosnia-Herzegovina, 
Bulgaria, Czech Republic, 
Estonia, Hungary, Latvia, 
Lithuania, Macedonia, Poland, 
Romania, Slovenia, Slovakia 
(Phare) 
Armenia, Azerbaijan, Belarus, 
Georgia, Kazakhstan, 
Kyrgystan, Moldova, 
Mongolia, Russia, Tajikistan, 
Turkmenistan, Ukraine, 
Uzbekistan (Tacis) 
Albania, Bosnia and 
Herzegovina, , the Former 
Yugoslav Republic of 
Macedonia 
Armenia, Azerbaijan, Belarus 
Georgia, Kazakhstan, 
Kyrgystan, Moldova, Russian 
Federation, Tajikistan, 
Turkmenistan, Ukraine, 
Uzbekistan 
Mongolia.  
Croatia and the Federal 
Republic of Yugoslavia 
become eligible under Tempus 
III by virtue of this Regulation 

 
 
  
 
Annex II 

Chronology and overall country allocations during 1994-1999 

PHARE
Bulgaria
Czechia
Estonia
Former GDR
Former Yugoslavia
Hungary
Latvia
Lithuania
Poland
Romania
Slovakia
Slovenia
CARDS
Albania
Bosnia and Herzegovina
Macedonia
TACIS
Armenia
Azerbaijan
Belarus
Georgia
Kazakhstan
Kyrgyz Republic
Mongolia
Moldova
Russia
Tajikistan
Turkmenistan
Ukraine
Uzbekistan

TEMPUS I

TEMPUS II

II bis

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

Funding
EUR MM
38,3
26,0
7,5
0,9
6,0
58,0
9,0
13,3
125,0
64,0
26,5
10,1

13,2
6,0
6,0

2,8
2,1
6,3
2,6
6,3
3,5
2,5
3,4
66,9
0,7
1,1
24,2
6,1

16