WORLD TRADE  
ORGANIZATION  WT/COMTD/W/125 
22 March 2004  
 
(04-1247) 
Committee on Trade and Development  
   
 
  
 
  
 
    
 
 
TECHNICAL COOPERATION AUDIT 
REPORT FOR 2003 
  
 
Note by the Secretariat
 
 WT/COMTD/W/125 
Page 2  
 
ABBREVIATIONS
 
 
 
 BTOR  Back to Office Report 
 CTD  Committee on Trade and Development 
 DDA  Doha Development Agenda 
 DFID  Department for International Development  IDB  Integrated Database  IF  Integrated Framework  ITTC  Institute for Training and Technical Co-operation 
 JITAP  Joint Integrated Technical Assistance Programme 
 LTPC  Long Trade Policy Course  M&E  Monitoring and Evaluation  MTS  Multilateral Trading System  PEF  Participant Evaluation Form 
 RTPC  Regional Trade Policy Course 
 SADC  Southern African Development Community  SPS  Sanitary and Phytosanitary Measures  STPC  Short Trade Policy Course  TA  Technical Assistance  TAMC  Technical Assistance Management Committee 
 TBT  Technical Barriers to Trade 
 TCA  Technical Cooperation Audit  TI  Training Institute  TPC  Trade Policy Course  TPR  Trade Policy Review  WT/COMTD/W/125 
 Page 3 
 
 
I. INTRODUCTION 
1. This is the third annual report of the Tec hnical Cooperation Audit (TCA) on the technical 
assistance (TA) and training activities in WTO.  It examines TA provided to Members during the period between 1 January and 31 December 2003.  The report draws upon and in a way adds to the 
evaluative information included in the Director-Gen eral's report on the implementation of paragraph 
41 of the Doha Ministerial Declaration submitte d to the Ministerial Conference in Cancún in 
September 2003.
1 
2. Members' needs for technical assistance and tr aining continued to be very high during the 
period under review. Last year more than one thousand requests for assistance were received, of 
which 441 were finally included in the Coordina ted WTO Secretariat Annual Technical Assistance 
Plan 2003 (Plan).2  The priorities reflected in the Plan we re addressed through intensive cooperation 
with all the stakeholders and partner agencies.   
3. The Secretariat continuously adapted the tr aining and technical assistance methods and 
products to the new circumstances and requirements.  During the period under review, building lasting 
capacities was a priority but the qua ntitative approach to benefit as many people as possible was still 
there.  The implementation of the TA and training  plan was supported by organisational changes that 
took place with a view to maximize synergies be tween training and TA, both aimed at enhanced 
institutional and human capacities of the Member s to address WTO issues.  The Technical 
Cooperation Division, and the Training Institute we re merged into the Institute for Training and 
Technical Co-operation (ITTC), which has become the driving force behind technical assistance and 
training. The Operational Divisions continued to be the main providers of technical assistance.  The 
annual report now reflects these organisational changes in the sense that it covers the TAs and training 
delivered both in the field and in Geneva (the la tter ones were formerly reported separately by the 
Training Institute). The relevant information is inte grated in the appropriate sections of the report. 
II. BACKGROUND 
4. TA and training activities as a capacity-buildi ng programme in 2003 were similar to those 
carried out in 2002.  TCA has compared the findings, conclusions and recommendations included in 
TCA Annual Report for 20023 with the analysis of the TA and training programme carried out in 
2003.  It has been found that most of the statemen ts made in the annual report for 2002 are still valid 
and also apply to 2003.  Therefore, the current report includes those conclusions and recommendations that represent a value added to the last annual report or that are worth repeating 
here in the hope that they will be considered during the implementation of the new TA and Training 
plan for 2004, which is new both in its approach, concepts and tools.  
5. This report focuses on the evaluation of TA and training delivered on the basis of the TA Plan 
for 2003.  Some activities, e.g. those for which the WTO does not assume responsibility for the full 
project cycle were not taken into consideration.  Th is is because it would be extremely difficult, if at 
all possible, to identify any effects and impact excl usively attributable to the WTO contribution to TA 
events where WTO's  only role was to deliver a presentation in a workshop, conference or seminar 
organised by other agencies, or to give an input  to a regional symposium or conference with hundreds 
of participants.  Therefore, some 270 TA and trai ning activities were subject to regular evaluations 
either by the TA providers or by TCA in 2003. 
                                                     
 
 1 Report by the Director –General: Paragraph 41 of the Doha Ministerial Decl aration- WT/MIN(03)/3 – 
14 August 2003. 
2 WT/COMTD/104/Add.1/Rev.2. 
3 WT/COMTD/W/111. WT/COMTD/W/125 
Page 4  
 
6. The system for Monitoring and Evaluation (M&E) of TA activities was introduced in 
April 2002 on an experimental basis for two years.   Although the M&E system is not yet working to 
its full potential, it has started generating an  increasing volume of important qualitative and 
quantitative information.  All this in formation is used in this report.  Furthermore, the report has also 
included observations and suggestions  from TA providers, obtained through regular work contacts, as 
well as the perceptions and recommendations of st akeholders, e.g. partic ipants and counterpart 
ministries and other agencies.  In the autumn of 2003 TCA carried out a mid-te rm review of the M&E 
system with a view to verify the application a nd use of the monitoring and evaluation concepts and 
tools and to suggest appropriate changes.  The findi ngs and recommendations of this review have also 
been included in the report.   
7. The strengths, weaknesses and opportunities are identified and examined in the planning, 
design, implementation (monitoring) and evaluation phases of TA and training. It is believed that this structured approach will help to make TA and tr aining more results-oriented. The core evaluation 
aspects, i.e. Effectiveness
4, Efficiency5, and to a certain extent Relevance6 and Sustainability7, are 
examined in the section analyzing th e evaluation phase of TA activities.  
8. During the period under review, continued em phasis was put on creating and strengthening 
local capacity.  Efforts have been made to gr adually transfer ownership of TA and training 
programmes to the field;  indeed, this is a key aspect of the regional trade policy courses.  By 
partnering with local academic institutions and e ngaging local resource persons in seminars and 
training activities, technical assistance has acquired a new dimension and enhanced  its regional focus.  
The evaluation reports emerging from TA and training  activities confirm that the level of knowledge 
of participants is increasing and the (overwhelmi ng) positive feedback from the participants points to 
the continued high quality of the TA. 
9. Furthermore, there is evidence of the positive impact of the TA and training, for example, 
some participants in WTO seminars are now acting as resource persons, many officials who were trained at the WTO's Institute for Training and Tec hnical Co-operation (ITTC) continue to liaise with 
the WTO and in many instances have become the local counterparts in the Ministries in capitals 
where they often occupy key positions; and many of th e former trainees have at some point returned 
to Geneva as delegates and/or negotiators.  In the light of the overall positive outcome recorded 
above, one should appreciate the review of th e results and the critical observations and 
recommendations included in the report, which we re made with a view to achieve further 
improvements in all phases of TA and training.  
10. Inter-agency cooperation continue d to serve as a core component of these programmes, and 
strengthened the coherence in policy making, techni cal assistance and capacity building. Special joint 
programmes, such as the Integrated Framework and JITAP have been reinforced and are generating 
results. 
11. During the period under review, DFID's financ ial support has proved to be invaluable.  
Without such assistance it would not have been possi ble to continue with evaluation work.  It should 
be noted, however, that by the end of June 2004 th is financial support is due to be phased out, with 
serious resource implications, and indeed a capaci ty deficit, in keeping up with monitoring and 
evaluation requirements of TA . 
                                                     
 
4 Effectiveness refers to the extent to wh ich the objective(s) of a TA was achieved. 
5 Efficiency considerations attempt to find ou t if the results of a TA justified the cost. 
6 Relevance refers to the extent to which the TA me t the needs of the participants and their countries. 
  7 Sustainability reflects if the benefits generated by the TA are maintained and further developed by the      
participants and their employers without further external assistance.  WT/COMTD/W/125 
 Page 5 
 
 
12. The annual report is intended to inform Memb ers, and to enable senior management and 
providers of technical assistance to take into account its conclusions, recommendations and lessons 
learned in future programming and operational decisi ons regarding trade-rela ted TA and training by 
the WTO.  Concerning the lessons learned, it shoul d be noted that organizational learning is an on-
going but slow process;  therefore it is not surprising that some of the lessons mentioned here are 
similar to those derived from similar exercises of pr evious years.  The follow-up to recommendations 
and the application of lessons learned are primar ily the responsibilities of TA providers and their 
organizational units, i.e. either ITTC or the Op erational Divisions.  Within its own means, TCA 
continues to conduct on-site eval uations, disseminate and encourage the application of lessons 
learned, keep up the established monitoring and evalua tion procedures, and further refine its tools.  
13. This report is based on a large number of eval uation reports.  One hundred and forty five of 
them, including eleven on-site eval uations conducted by TCA, were pr epared in the fo rmat of back-
to-office reports (BTORs).  In addition, we also received 31 memoranda prepared on field (outreach) activities and on TPRs, and 13 repor ts on training activities that took place in Geneva, prepared in the 
format used by the former Training Institute.  Th e total of 189 evaluation reports has provided a basis 
solid enough to draw conclusions, ma ke comprehensive recommendations  and learn lessons.  Overall, 
the number of reports received is a considerable improvement on the last  year's records (126) but still, 
there are a number of reports that do not reach TCA.  Some observations on this issue are made in the 
section on reporting below.  
14. In preparing this report TCA has also taken into account comments and suggestions made by 
the donors and Members concerned at both formal (in the CTD) and informal discussions of salient 
issues of monitoring and evaluation.  Annex 1 includ es the list of evaluation missions carried out by 
TCA in 2003.  
III. LINKAGES TO HIGHER LEVEL OBJECTIVES 
15. In response to some Members' suggestions, linka ges have been examined between the TA and 
training carried out during the period under review and the broad objectives set out for WTO technical 
cooperation as a whole.  The purpose of linking the evaluated TAs to higher level objectives was to 
see if the TAs were kept on the right track and impl emented in harmony with those broad objectives 
set out in WTO's Strategy for Technical Co-operation.
8 
                                                      
 8 There are seven broad objectives attached to th e New WTO Strategy for Technical Cooperation:  
(i) coordinate  effectively with key deve lopment partners to enable countries a nd customs territories, beneficiaries of 
WTO technical assistance, to participate and integrate more meaningfully in the multilateral trading system, in order to increase incomes for growth, pove rty reduction and development;  
(ii) assist  beneficiary countries to understand their rights, implem ent their obligations and impr ove their capacity to draw 
on the benefits of an open, rules-based, multilateral trading system.  To this end, technical assistance will focus on 
establishing partnerships with beneficiary count ries to better understand and use the rules;  
(iii) increase  the volume and quality of the technical assistance th at the WTO delivers in response to significantly 
increasing demand (from an increasing Membership);  
(iv) provide  a focused response to the priorities expressed by be neficiaries, within a coherent trade policy framework 
and companion policies;  
(v) support  national efforts and ownership in building durable institutional capacity for trade policy formulation, 
advocacy and negotiation;  
(vi) ensure  that WTO technical assistance is delivered within a coherent policy framework (not on a stand-alone basis) 
through careful coordination with bilateral and multilatera l development partners.  Delivery of WTO technical 
assistance within a coherent policy framework will seek to mainstream  trade priority areas of action into national 
development plans and strategies for poverty reduction, so that  trade will more effectively play its role as an engine 
for poverty reduction and development; and 
(vii) provide  legal advice and assistance in re spect of dispute settlement to deve loping country Members, pursuant to 
Article 27.2 of the Dispute Settlement Unde rstanding.   
 WT/COMTD/W/125 
Page 6  
 
16. The several rounds of discussions of the Pl an in the Committee on Trade and Development 
(CTD) have certainly contributed to keeping the Plan close to the technical cooperation strategy 
endorsed by the Members.  Furthermore, based on th e comments made by the participants of TA and 
training activities, and observations included in the self-evaluation reports, the following has been 
found, structured by the broad objectives.  The TAs included in this report: 
- were often coordinated and executed together  with the main development partners, e.g. 
UNCTAD, ITC, World Bank, OECD, IMF, USAID, Commonwealth Secretariat, DFID, etc, to mention a few (Obj. i);  
 - contributed to establishing partnership with the beneficiary Members and aimed at an 
improved understanding of the par ticipants regarding their coun tries' rights and obligations 
and strengthened their capacities to draw on the be nefits of an open, rules-based MTS.  This 
has been evidenced by the amount of feedback  and questions received from the participants 
and their employers, following the TA (Obj.ii).   
- the quantity of TA delivered in 2003 was so mewhat lower in number (486 in 2002 and 451 in 
2003) but carried out to the full satisfaction of th e beneficiaries.  This indicates that the "do 
fewer but better" consideration has started taking priority over the quantitative approach 
pursued in previous years (Obj. iii). 
 - WTO TA, particularly the JITAP and the IF TA programmes, has been delivered within a 
coherent policy framework seeking to mainstream  trade into national development plans and 
consolidate national ownership and capacities (Objs. iv, v, and vi), and finally, 
 - 12.4 per cent of the TAs included in this report delivered highly appreciated advice and 
assistance in respect of dispute settlement to developing country members (obj. vii). 
 
IV. OBSERVATIONS ON THE PLANNING, DESIGN, IMPLEMENTATION AND 
EVALUATION PHASES OF TECHNICAL ASSISTANCE AND TRAINING 
1. The Planning Phase 
17. The Doha Development Agenda (DDA) clearly reflects WTO Members' support of, and need 
for, technical assistance as an instrument for the integration of developing and least-developed 
countries into the multilateral trading system.  Th e DDA was the guiding fo rce behind the Members' 
considerations regarding the pla nning, the organization and delivery of WTO technical assistance in 
2003.  
18. Members oriented the secretariat's efforts to prov ide TA within the framework of a TA plan in 
the areas prioritized in the DDA.  The relevant paragr aphs of the DDA, to which a particular TA event 
was related were highlighted in the TA plan for 20 03.  This has facilitated the transparent allocation 
and use of the resources for technical assistance and training.  The Plan was based on a long list of 
national requests (1048) resulting in a great number of  individual TA activities structured in a tabular 
format without, however, explicitly linking them to one or more of the broad objectives set out in the 
strategy for WTO TA.  
19. With regard to the planning process, it was found that the Secretariat faced the same 
challenge as in 2002 in trying to make both ends meet, i.e. making the plan correspond to Members' 
needs and making it manageable within the available financial and staff resour ces. Although there was 
a large element of uncertainty involved, the Plan wa s approved in December 2002.  It listed a total of 
441 planned activities, marking 302 of them TBC (to be confirmed) (228 in the previous year) in  WT/COMTD/W/125 
 Page 7 
 
 
terms of dates, and sixty activities for which the pr ecision of timing had to remain within a range of 
three to six months.   
20. Clearly, at the time of preparation and approval of the TA Plan, neither the Secretariat nor the 
Members were in a position to be more precise, and the plan still reflected a fragmented approach; a clear shortcoming of the planning process.  The Plan became necessarily indicative and the 
management of its implementation should adapt the TA activities to the actual needs of the recipient 
countries.  
21. The table below shows an at-a-glance picture of  the implementation of the TA Plan for 2003. 
Type of Activity No. of TA implemented Relative to 441 Planned 
activities (%) 
Field-based 
→ Regional 
→ National 431 
245 
186 98 
56 
42 
→ Geneva-based 
→ Outreach         14 
6 3.1 
13 
Total no. of activities 
implemented 451 102.3 
Ad hoc 161 36.5 
 
22. Members will receive a separate report from ITTC that highlights details of implementation.  
From the point of view of this report, it is lauda ble that in terms of the number of TAs (98 per cent) 
the actual delivery was very close to the number ta rgeted. On the other hand, it is important to 
mention that the table shows a considerab le number of activities implemented on an ad hoc  basis 
(36.5 per cent or 161).  Indeed it is  much higher than in 2002 (27 per cent or 130).  Furthermore, it 
proves that the desired planning details could not be set out with the required precision and, as 
mentioned above, the implementation of  the plan had to be adapted to the actual needs of the recipient 
countries.  In other words, quite a few of the TA activities included in the original plan had to be 
deleted, delayed or reformulated and new ones added to meet new needs.  
Conclusions  
23. The concept of a Plan has both possibilities and limitations.  Ideally, it should include 
objectives, mutually agreed between  the Members and the Secretariat, that should be achieved, and 
can be measured by some pre-set benchmarks or performance indicators.  Furthermore,  it should 
reflect priority areas of concern and serve as a kind of management tool that helps transparent and 
accountable implementation.  The closer the actual deliv ery to the plan, the more realistic the planning 
process.  On the other hand if it is taken too rigor ously, the plan may become an obstacle to flexible 
reactions to emerging needs, particularly during a round of negotiations.  Therefore TCA considered 
the plan as an indicative instrument for implementi ng capacity-building activities.  It should also be 
mentioned that even the best plan cannot provide  a guarantee for the expected outcome of the 
activities executed, namely enhanced capacity of th e Members in WTO issues.  There are many other 
factors which will have an influence on this, no t least how effectively the Members receiving 
technical assistance can put into use what w as imparted and learned through the TA.  
24. The Plan for 2003 was still attaching impor tance to reaching the maximum number of 
countries and participants and disseminating inform ation and training covering all possible subjects 
falling into the competence of WTO.  This is pe rhaps not surprising given the demand-driven notion 
and the fact that most TA and training during the first eight months of 2003 was focusing on priority 
issues linked to the preparation of the Member s for the Ministerial Conference in Cancún.  WT/COMTD/W/125 
Page 8  
 
25. This approach, which has resulted in fragm entation and inflexibility has finally been 
recognized as an underlying weakness of the planni ng process; subsequently, the joint efforts of the 
Secretariat and the Members have resulted in a new approach, a new planning framework indeed, 
which, in the form of a core programme provides for what WTO can offer at its best.  Furthermore, it 
facilitates a flexible reaction to meet TA needs at the national level.  The TA and Training Plan for 
2004 is a welcomed development in the right dir ection, and TCA takes it as an important step 
forward, a sign of improved knowledge manageme nt and application of lessons learned. The 
management of the new TA and training plan shoul d meet the needs of the Members for information 
on the progress made in its implementation.  It is questionable, however, if the current reporting best 
serves that purpose.  Therefore, an examination is  suggested as to whether the rather frequent 
quarterly monitoring reports, introduced in connecti on with the implementation of the TA Plan for 
2003 still fit with the new TA Plan for 2004, or  a more practical, say, semi-annual progress 
(monitoring) reporting would do.  The planned mid- term review of the plan will hopefully give 
guidance on this issue. 
26. Another of the Secretariat's important tasks in  the planning phase is to match the TA Plan 
with the needs of the beneficiary Members.  Grow ing recognition of the necessity to make the TAs 
more oriented towards the problems/needs of th e Members has been noted as an important condition 
to make the TA even more user-oriented.  It is  commendable that the importance of the systematic 
assessment of trade-related TA and training needs has been recognized and carried out in several 
areas, although in a kind of decentralized manner, e.g. by JITAP, IF, the Committee on TBT, TPRs, 
trade clinics, etc.   
27. The Secretariat certainly has the capacities, alt hough more intellectual than financial, to help 
Members formulate and prioritize their needs for TA .  In meetings and cons ultations with donors and 
potential beneficiary countries or groups the Secretaria t has an opportunity to shape the programme of 
activities.  The TPRs should also provide an opportunity  to highlight the technical assistance needs of 
the target country or point to capacity deficits where training would be justified.  Inevitably, there will 
be situations where, for example, suggestions by committees that TA of a given kind should be 
provided, or an unforeseen request by a donor, benefici ary country or cooperating agency will have to 
be addressed.  Operational divisions have a particular role to play in this regard because of their in-
depth knowledge of the individual country s ituation with respect to each Agreement.  
28. Needs are more generic at the country level and more specific at the level of the participants. 
Therefore, the outcome of a TA activity can be improved if its programme is carried out in full 
awareness of what exactly the par ticipants expect from the TA provi der(s).  Failing this, if time and 
information permit, the best alternative would be to  identify and analyse the participants' needs prior 
to the TA and to adapt the programme to those n eeds.  Some courses and workshops have started by 
exploring the expectations of the participants a nd adapting the programme in due regard to those 
needs.  This approach, however, also has its limits since it is hardly feasible when there are more than 
thirty participants in a seminar or a workshop and thus the exercise becomes so  time-consuming that it 
would be detrimental to the de livery of the planned programme. 
29. The identified needs should, however, be met at the level of the participants.  To have the 
right group of participants in a TA or training event is crucial for its success.  With the exception of 
TA of a very technical character, where the particip ants are specialists in the subject discussed, many 
evaluation reports mentioned the heterogeneous composition of the participants, mixing beginners 
with advanced.  Since usually it is difficult to iden tify the individual expectations and carry out a due 
adjustment of the programme at the start of the TA , it often occurs that some participants complain 
that the presentations of the issues and related WT O Agreements were too de tailed, or not detailed 
enough.  It should be mentioned that the Secretariat, wherever it is feasible, makes systematic efforts 
to ensure that people with the right profile benefit from training.  For instance, fully-fledged selection 
applies for the twelve-week trade policy courses both in Geneva and in the regions.  Fully-fledged 
selection was also applied for the specialized course  on Trade in Services (two week long).  For the  WT/COMTD/W/125 
 Page 9 
 
 
other Geneva-based activities, selection criteria exist (and are followed) but they are based on a first 
come first served selection.  In many other cases, however, the actual composition of a group of 
participants is often beyond the control of the Secretariat.  
30. The ideal would, perhaps, be that once the TA needs of the Members are identified and 
prioritized, the subsequent actions lead to the development of individual country programmes of 
longer duration, say two to three years.  These programmes would pursue objectives mutually agreed 
upon between WTO and the Member and result in a kind of cumulative outcome benefiting the 
targeted human resources and contributing to the est ablishment of an enabling environment.  In case 
of similar priority needs of countries in the same region, TA may continue to be provided on a 
regional basis and the rest of the needs be met through TA and training at the national level. 
2. The Design Phase 
31. It should be noted at the outset that the Par ticipant Evaluation Forms (PEF) demonstrated the 
success of most of the TAs delivered, in spite of the fact that very often their objectives were set out  
in terms of activities by the TA provider rather than  of the expected end-results at the level of the 
participants.  It is evident that even a perfect d esign is no guarantee of success.  However, it makes the 
TA and training more transparent, accountable and pr ovides a better basis for evaluation.  Therefore it 
must be emphasized that having completed a needs/problem analysis and priority setting, the next 
most important task in the design phase is th e formulation of the objectives and performance 
indicators to measure their achievement.  TCA believes that clear objectives and indicators could further improve the performance of the TA and traini ng and certainly contribute to transparency and 
accountability.  In this regard the established guide lines and checklists shoul d be better known and 
utilised. 
32. The table below shows the situation re flected by the 145 evaluation reports. 
33. It has been found that relative to last year's findings the number of TAs setting out adequately 
formulated objectives has increased, totalling 48.3 pe r cent as opposed to 28 per cent in 2002.  Fifty-
one per cent of the objectives were still stated as act ivities of the TA providers.  Overall, this is a 
considerable improvement but also points to the wea kness that about a half of the TA providers still 
focus on what they intend to do in terms of lectur es and presentations, etc, rather than on the expected 
results.   
34. Another important design element that still recei ves insufficient attention is the performance 
indicators whereby the achievement of or progress towards the objectives should be measured.  Only 
a handful of TA providers (1.4 per cent) set out indicators prior to the delivery of the TA.  
35. The table below shows that most of the TA providers demonstrated the success of the TA by 
referring to how active the participants were in discussing the subject matter or asking questions 
(44 per cent), or stating positive views in the Partic ipant Evaluation Forms (PEF) (46 per cent) at the 
completion of the TA or training, instead of comparing these reactions with pre-set indicators.  On the 
other hand, a few TA providers, particularly in the field of dispute settlement, have started using the 
quality of the response of the participants to technical questionnaires or case studies, or technical reports prepared by them as targets,  set out before the TA takes place. Objectives Per cent Quantity 
Real objective  48.3 70 
TA providers' activity set out as an objective         51 74 
Output set out as an objective           - - 
No objective indicated                   0.7 1 WT/COMTD/W/125 
Page 10  
 
36. Regarding the rest of the tasks in the design phase, e.g., setting out the programme, defining 
the sequence of presentations, lectures, group work, discussions, etc., it has been found that the TA 
providers have the tendency to pack  too much into the TA programmes, which on the spot turns out to 
be too intense, resulting in comments from some of the participants that more time should have been 
allocated to learn about the subject matters.  TCA be lieves that such situations could be traced back to 
the insufficiently identified needs. 
37. Finally, very few of the TA providers left  room in the course/workshop programme for 
evaluation activities recommended by the M&E procedur es, e.g. measurement of the baseline at the 
beginning of the TA and an end-of-TA verbal evalua tion at the end, supported in between by personal 
interviews.  When questioned, the heavy programme is said to be the reason for leaving no time for 
evaluation.  
Conclusions 
38. The importance of good design is yet to be fully understood.  TCA believes that any TA 
activity should be designed with due regard to its evaluability.  Therefore, in addition to the well 
formulated objectives, performance indicators, outputs and activities, evaluation criteria and evaluation process have to be integrated into th e programme design in a consistent manner.  The 
reason it is not done systematically and more regularly is explained by the fact that most of the staff 
have no in-depth knowledge of the M&E concepts and their use, hence they appl y them to the best of 
their understanding of the evaluative questions included in the reporting formats.  
39. In a broader context, the office-wide introduc tion of monitoring and evaluation concepts and 
tools was put into effect with a view to induce a ppropriate changes in the management. It was a much 
needed decision but should have been complemented  by specific training for the staff concerned with 
TA.  The perception of "just do it" still persists.  Therefore, TCA, in the second half of 2003, 
introduced a service that assists the TA providers in  improving the design process, e.g. to formulate 
objectives and performance indicators and targets pr ior to the delivery of the workshop, seminar, 
course, etc, and the evaluation process is also explai ned.  This practice has led to positive results and 
the recipients now seem to feel more accountabilit y for achieving the jointly set out objectives and 
through them the expected results and outcome.  This  feedback encourages TCA  to continue with this 
service. Results Per cent Quantity 
Preset Indicators                        1.38 2 
Answers to PEFs                          39.31 57 
Active participation                     44.14 64 
End-of-TA verbal evaluation              2.76 4 
Quality of participants' presentations   4.14 6 
Formal test of K/S transferred                 2.07 3 
Feedback from participants               46.21 67 
Feedback from participant's employer     8.28 12 
Meeting legal obligations                0 0 
No indicators set out                    5.52 8 
Observations of the TA provider          4.83 7  WT/COMTD/W/125 
 Page 11 
 
 
3. The Implementation Phase  
40. The implementation of the TA plan for 2003 and meeting ad hoc  needs put great pressure on 
all staff involved in the provision of technical assi stance and training.  Almost all units of the 
Secretariat were involved in one or more aspects of  technical assistance.  The completion of the Plan 
necessitated almost 900 field missions.  
41. The feedback received from participants and delegates indicates that WTO staff are highly 
appreciated as experts on the issues involved in th e implementation of WTO Agreements.  They have 
a comprehensive, world-wide view of the current issues, know best the internal mechanism of WTO 
committees and possess the latest information.  TA  providers are well positioned to disseminate 
lessons learned, which may include best practices and/or situations that should be avoided.  The 
current way of TA implementation provides for a sufficient degree of flexibility, and it is the TA 
provider who can freely choose the modality of trai ning and information dissemination that is best 
suited to the actual needs, issues and target groups.  More specifically, s/he decides on the timely use 
of a variety of methods appropriate to the content and objective; the selection and use of audio visual aids, presentations, tests, exercises, case studies; the development or selection of background 
documentation, CD-Roms for distribution, etc.  
42. Overall, it has been observed that WTO staff has become, over the years, more used to 
providing TA and training.  The methods of lectur ing and presentations, the balance between theory 
and hands-on sessions, the choice of the training methods in general we re being given more 
importance in 2003.  The Training of Trainers course benefiting some 150 staff members of the 
Secretariat in 2002 is bearing fruit in terms of quality.  The use of the gap anal ysis, multimedia, slides, 
group exercises, tests, the involvement of the more experienced participants in the learning process, 
case studies and  simulations, where participants are ca lled on to play an active role, is more frequent. 
43. TA and training activities covered by the 145 evaluation reports benefited 5,110 persons 
(3,326 (65%) men and 1,784 wome n (35%)) representing government units such as ministries of 
trade, finance, foreign affairs,  agriculture, commerce, textiles and clothing, customs, as well as 
academia, private sector and NGOs.  
44. Evidently, this does not represent the total num ber of persons benefiting from WTO TA.  In 
the sample of the 145 evaluation reports the average num ber of participants totals thirty-two.  This, by 
experience, may be applied to the entire TA progra mme, which thus has benefited more than  14,000 
individuals.  A significant achievement, even if some  of the participants attended more than one TA 
event.  
45. The table on the following page shows the sector al coverage of the TAs and training reported 
about.  The percentages are higher than one hundred , since many TAs, national trade policy courses 
for example, address substantiv e issues in several sectors.  
46. From the table, services, SPS/TBT, market access,  agriculture and rules, emerge as areas that 
continued to be the most covered in 2003.  Howe ver, their proportion relative to other sectors has 
somewhat diminished and other sectors have to a certa in extent increased their share relative to 2002.  
In other words, the overall sectoral coverage tends to spread more evenly. 
47. The establishment of ITTC has started producing results in the implementation process.  
Regular and mutually advantageous contacts have b een established between the staff of some of the 
Operational Divisions and the ITTC requesting the la tter to hand out training aids, provide multimedia 
assistance, methodological help, develop tests and cases t ogether, etc.  All this demonstrates that TA 
providers gradually recognize the importance of inter active training, dialogue with the participants, 
testing their knowledge and measuring the transfer of  knowledge at the end.  However, some of the WT/COMTD/W/125 
Page 12  
 
TA providers still give priority to maximizing the volume of information included in a course or in a 
workshop, instead of considering doing fewer
 things better .  
48. Well organised documentation in ha rd prints and/or on CD Roms is usually distributed to the 
participants.  In a few cases however, problems wi th transportation, inadequate local reproduction 
capacities or unexpectedly high number of participants  have resulted in confusion or complaints.  
49. Some minor organizational and coordination diffi culties were also experienced.  They were 
attributed to short deadlines and stretched resources.  But all this has to be seen in the light of the 
delivery of an extremely large volume of TA and trai ning.  What would certainly be helpful in this 
regard is a continued systematic coordination between the Operational Divisions, ITTC and the 
Logistic Unit.   
50. Regarding the Geneva-based training activities ( not counting the Geneva Week events), more 
than 1,800 hours of training have been conducted (approximately 300 days), thousands of pages of 
training material have been deve loped and handed out to more than  300 participants.  Despite an 
effort being made in the selection process to ensure  an equal gender representation, the number of 
male participants remains higher than the number of female participants.  This is essentially due to the 
fact that beneficiary countries still tend to nominate more men than women.  Sector Percentage in 2003 Percentage in 2002 
Accession                                2.07 3.97 
Agriculture                              13.79 15.87 
Competition Policy                       7.59 2.38 
Customs Valuation                        4.83 7.94 
Dispute Settlement                       12.41 15.87 
Government Procurement                   2.76 3.17 
Implementation issues                    8.28 7.94 
IT/WTO Reference Centres                 7.59 2.38 
Integrated Framework/LDCs                0.69 4.76 
Investment                               12.41 5.56 
Mainstreaming                            0.69 6.35 
Market Access                            14.48 11.11 
Rules                                    13.79 13.49 
Services                                 26.21 29.37 
STPC                                     2.76 2.38 
SPS/TBT                                  16.55 17.46 
Textile &Clothing                         4.14 2.38 
Trade and Environment                    2.76 6.35 
Trade Facilitation                       7.59 3.97 
Trade Negotiation Skills                 2.07 5.56 
TRIPS                                    0.69 4.76 
DDA Advanced Negotiations Course         2.07 0.79 
Other                                    12.41 18.25  WT/COMTD/W/125 
 Page 13 
 
 
TA and training activities in Geneva Number of 
Projects Number of 
participants 
Trade Policy Courses (twelve weeks) 4 (2E*/1F**/1S***) 105 
Dispute Settlement Spec. Courses 3 (2E/1F) 84 
Introduction Courses 2  (1E/1F) 43 
Trade Negotiations Spec. Courses 2 (E) 25 
Trade in Services Spec. Courses 1 (E) 26 
Training week on WTO 1 (E) 15 
Introduction days 2 (E) 70 
 
* English ** French ***Spanish 
 
51. Regarding the implementation of capacity-build ing efforts in the regions in 2003, twenty-
three government officials from English-speaking African countries were trained in the Regional 
Trade Policy Course (RTPC) held in Nairobi, Kenya.  The participants benefited from fifty-five days 
of training.  It also contributed to the developmen t of partnership between the WTO and institutions in 
the region.  Twenty-five resource persons came from eight English-speaking African countries and 
were involved in the joint delivery of each of the topics covered by the course.  A key to the smooth 
collaboration between the resource persons from the Secretariat and those from the region was a 3-day 
preparatory workshop held in Geneva that include d discussions on joint research projects, methods of 
delivering the RTPC, and a series of bilateral me etings between WTO lecturers and the regional 
specialists anticipated to be involved in the join t delivery of the course.  Based on the favourable 
experience, it is already envisaged that for the 2004 Nairobi RTPC the regional resource people will 
play a greater leading role in the de livery of some areas of the course. 
52. This was the second regional trade policy course organised jointly with the University of 
Nairobi.  The University has played an increasing role in developing and structuring the course programme to adapt to the needs of the region.  In  addition, as evidence of a broadening partnership, 
at least one WTO Division has started to involve a regional specialist in their technical co-operation 
activities, outside the context of the RTPC.   
53. To foster and strengthen research links the Ec onomic Research and Statistics Division of the 
WTO has undertaken research projects involving some of the regional resource persons involved in delivering the Nairobi RTPCs.  Findings of one of these projects will be presented to the participants 
in the next Regional Trade Policy Course in Nairobi. 
54. The implementation of TA assumes the developm ent and availability of  outputs, such as 
lectures, presentations, slides, notes, overheads, vi deos, case simulations, tests, reports, books, etc, 
and related activities without which the objectives ca nnot be achieved.  Below are some important 
outputs that have proved to be very effective in transferring knowledge.  
 
 
  
  
 WT/COMTD/W/125 
Page 14  
 
 
Examples of major outputs used in capacity-building activities 
 
a) The study on “Market access issues related to pr oducts of export interest originating from least-
developed countries” particularly addresses the commitment contained in paragraph 16 of the Doha Declaration.  This analytical work complements previo us studies, such as a study entitled "Tariff negotiations 
and renegotiations under the GATT and the WTO: Proced ures and Practices", prep ared by the Secretariat. 
 b) The prototype of a “toolkit for negotiators”, that is in its completion phase (testing and validation), 
has been developed in cooperation with the Inter-American Development Bank and the Central-American 
Integration Secretariat (SIECA ), to assist in particular small delegations, notably from LDCs to carry out the 
analytical work involved in the preparation of offers and requests, as well as to get a better grasp of negotiation 
processes involving the drafting of consensus texts.   
 c) Considerable efforts have been made, with the fi nancial support of bilateral donors, to develop the 
Integrated Data Base (IDB) and adapt it to the particul ar needs of participants in tariff negotiations.  
Complementary work has also been don e with regard to the Consolidated Tariff Schedules (CTS) database.  
 
d) In response to a request by the Working Group on Trade and Competition Policy, a "Study on specific issues relating to a possible multilateral framework on competition policy" (WT/WGTCP/W228) was prepared 
with a view to assist developing and least-developed countries to better evaluate the implications of closer 
multilateral cooperation for their development polic ies and objectives and for human and institutional 
development (for example of Doha Declaration). 
 
e) A Handbook on Anti-dumping Investigations, develo ped jointly with Cambridge University Press as a 
practical tool to assist newly established trade remedies  units in conducting anti-dumping investigations is now 
available.  A Model Countervailing Duty Legislation was also developed in an effort to assist developing 
country Members in drafting co untervailing duty legislation.
 
 
Conclusions and Recommendations  
55. The above outputs demonstrate a cross-fertili zation between the regular work of the 
Secretariat and TA and training activities.  It w ould be useful if all these and other major notes, 
studies, papers, survey results, databases, tests, ex ercises, handbooks, videos, CD-Roms, etc. that are 
regularly used in TAs and training were documente d and put into a compendium which is regularly 
updated and made available to every TA provider as a major source of TA and training materials.  It 
would certainly help to avoid duplication of some training materials by TA providers, new to TA and 
training. 
4. The Evaluation Phase  
56. The evaluation function is enabling the WT O to meet its substantive accountability 
requirements.  Evaluations provide the Members wi th useful global information on the results and 
outcome of TA activities and the Secretariat with op erations and management related information to 
address weaknesses and subsequently improve TA and training programmes.  
Evaluation Procedures and Methods 
57. In accordance with the established M&E pro cedures, evaluation is a shared responsibility 
between the TA providers and TCA. In other words, most of the TA evaluations are decentralized to 
the TA providers and their units.  This is not onl y because each and every one of the hundreds of TA 
activities cannot be evaluated by the very limited r esources for evaluation in TCA, but also because a 
good results-based management assumes that monito ring and evaluation are recognized and exercised 
by the Secretariat as inter-linked management f unctions.  Once it is achieved, it will tremendously 
help the improvement of TA and training by WTO at no additional cost.  WT/COMTD/W/125 
 Page 15 
 
 
58. The most frequently applied form of evaluation is evaluation by the TA providers (so-called 
self-evaluations) on the basis of agreed formats and methods.  The evaluation process engages the 
providers' responsibility for achieving the expected r esults, effects and impact.  Self-evaluations are 
primarily based on the views and suggestions of the beneficiaries of the TA (the participants in a 
training course or a TA activity), and on the provi ders' own judgment.  These evaluations permit a 
comprehensive coverage of all TA projects.  The results of self-evaluations are included in back-to-
office reports (BTOR).  In addition to the comprehensive system of self-evaluation, TCA, which represents a modest but interna lly independent evaluation capacity , carries out on-site evaluations. 
59. Furthermore, the M&E procedures provide for ot her types of evaluation, such as thematic 
evaluations and ex-post evaluations , which have not yet been experimented with.  The first ex-post 
evaluation is planned to be conducte d in 2004 aiming at the review of  the impact of selected Geneva-
based twelve week Trade Policy courses, carried out  in 2002 and 2003, on the participants, their 
career path and their contribution to the effective pa rticipation of their country in WTO issues.  In 
2003 the Integrated Framework was eval uated by independent evaluators.
9  Pending the availability of 
resources, external evaluations may be conducted in th e future on selected aspects of TA and training. 
Key Evaluation Aspects Reviewed 
 
60. As mentioned above, the basic tool to analy se and transmit evaluative information is the 
internal evaluation report, the BTOR.  It is noted with satisfaction that the overwhelming majority of 
the TA providers made the effort to follow the est ablished format, and use the evaluation concepts and 
tools.  Nevertheless, there is still room for improvement as shown below.  
61. The TA's performance is examined through the r esults and outcome, more specifically if the 
objectives were achieved (Effectiveness), if the resources allocated were used economically 
(Efficiency), if the knowledge and skills imparted were relevant to the needs of the participants 
(Relevance) and if the results of the TA a nd training are sustainable (Sustainability). 
(a) Effectiveness  
62. For Members and the Secretariat it is equally important to have evidence of effects and 
impact that demonstrate if the resources allocated for TA and training had be en put into good use.  
The effects and impact should be examined in the light of both the broad strategic objectives and the 
immediate objectives set out for the individual TAs a nd training activities.  In paragraph 16 above, it 
has been described how the TA and training activiti es were guided by the broad strategic objectives 
and that progress has been made towards their achievement.  Below, the success of the TAs and 
training as reported by the TA providers and seen by TCA is examined. 
Results Per cent 
The objective was fully achieved         77.93 
The objective was partially achieved     14.48 
The objective was not achieved           0.69 
No reply                                 6.9 
63. The table above shows that more than 92 per cent of the TA providers stated full or at least 
partial objective achievement, measured by participan ts' reactions, tests, group exercises or feedback 
received from the employers.  
64. The high percentage of objective achievement is admirable but invites some comments.  In a 
strict sense, if one takes into account one of the b asic weaknesses of the TA designs outlined above in 
                                                      
9 See more on this in documents WT/IFSC/6/Rev.2 and WT/IFSC/7. WT/COMTD/W/125 
Page 16  
 
paragraphs 32 and 33, i.e. about half of the TA s set out immediate objectives as activities by the TA 
provider(s), then the high percentage of the objec tive achievement could simply mean that the TAs 
were implemented.  The fact that they were impl emented would not prove, however, that the desired 
effects were achieved.  
65. Nevertheless, most of the TA providers concer ned, when questioned, said that what they 
actually had in mind was that by the end of the c ourse or workshop the participants would be better 
off in terms of knowledge and skills , although formally they did not state it in this way.  Another 
argument that points to the good value of the above pe rcentages is that most of the evaluation reports 
were based on the opinion and feedback gathered fro m the participants, very often appreciating that 
the TA met their expectations and substantiati ng that the expected results were achieved.  
66. To improve this situation it is believed that ther e is a need for simple training of the staff in 
the proper design of TAs and training and in the use of the basic elements of the logical framework 
that is a widely used tool to develop TA and training projects.
10  Improved design would certainly 
increase the transparency of the TAs activities and accountability for what they intend to achieve.  
Clearly, improvements in the design of the TA a nd training activities would certainly lend the data 
and information on the achievement of the obj ectives (effects and impact) more credibility. 
67. It is important that the external resources a llocated to TA and training are used in a manner 
that generates positive effects and impact on the beneficiaries.  Below are some examples of effects typically generated by TA and training activities: 
-   improved understanding of, and practical e xposure to, rights and obligations emanating from 
the different WTO Agreements.  These effects are mostly linked to short, i.e. two to four day 
workshops focusing primarily on dissemination of information;  
 - operational knowledge and skills to collect a nd analyze the right information; up-to-date 
information on the negotiations.  This transf er of knowledge and de velopment of specific 
skills primarily result from two-week and longer training courses, and 
 
- in the case of regional and sub-regional technical assistance, an additional effect is 
networking with trade experts from other countries.   
68. Only in a very few instances did beneficiaries question the usefulness of the activity in which 
they participated.  On the whole, they were c onvinced that their participation had significantly 
enhanced their theoretical, legal, and analyti cal capacities, and through them that of their 
governments.   
69. Regarding the Geneva-based courses, the implementation of the December 2001 Members' 
decision to double the number of regular Trade Policy Courses already delivered by the former Training Institute went hand in hand with the expa nsion of other kinds of training and information 
dissemination activities.  At the same time the need to maintain and if possible further increase the 
quality of the training posed a real challenge to ITTC. 
70. In 2003, trade-policy or training courses of various length and scope as well as distance-
learning, trainers-training and uni versity-networking projects were implemented in Ge neva, benefiting 
some 300 officials mainly from developing countries. The courses focused on widening the participants' understanding of the multilateral trading system and international trade law, and of the 
                                                     
 
10 The logical framework is a tool for the planning, de sign, implementation and evaluation of TA and training 
activities. Its major com ponents are: Objectives; Indicato rs (means of verification); Out puts; Activities;  Inputs and 
Assumptions (External Factors).  It is a standard tool used by the entire donor community and the UN system in technical 
cooperation activities. 
  WT/COMTD/W/125 
 Page 17 
 
 
activities, scope and structure of the WTO and on enab ling them to improve the effectiveness of their 
work in their own administrations.  
71. Regular evaluations of the performance and obser vations of the participants of the different 
activities were conducted both during a nd at the end of the courses.  These evaluations continued to 
show a high degree of satisfaction with the types of responses given by the WTO to meeting the 
participants' training needs.  In terms of improveme nt of knowledge and skills the participants stated 
that they have improved both their understanding of the WTO rules and functioning and their ability 
to actively participate in WTO work.  The trainers who coordinated the training activities also shared 
this assessment. Moreover, knowledge self-evaluations performed by the participants both prior and 
after various training modules showed evidence that  knowledge and skills are efficiently transmitted 
by the trainers.  In summary it may be noted that these training activities have achieved their 
immediate objectives. 
72. It should however be noted that there is s till lack of a consolidated assessment by the 
beneficiary countries of the changes that the cours es have been able to generate in the working 
environment of the participants.  This issue will be  taken up by the ex-post evaluation planned to be 
carried out in 2004. 
73. Furthermore, the fact that the training ta kes place at the WTO headquarters certainly 
constitutes an asset to match the need of participan ts to better understand the functioning of the WTO. 
However, it should be noted that most of the c ourses delivered in Geneva cannot fully address the 
specific concerns of the beneficiary countries in partic ular due to the heterogeneity of the participants.  
74. More "tailor-made" training, or country-specifi c approach of training needs, may however be 
better conducted in the field in the context of Re gional TPCs, as well as through regional and national 
training and technical co-operation activities.  Geneva-based and field-based activities should 
therefore be considered as complementary.  Th e positive experiences with the organization and 
delivery of courses in Geneva had a very benefi cial impact on the regional courses, in terms inter alia  
of course material and quality of delivery of training , since they are the model for the regional courses 
and set the standard for quality, which is expected by the Members.   
75. With regard to the results in terms of tr ansfer of knowledge, the government officials 
participating in the Nairobi course expressed satisfaction with the training received and were convinced of its benefits and relevance to their wo rk upon their return home.  Current developments 
underline this evaluation; some former participants  have been promoted, one is very active in the 
Accession Working Party for his country and at least one other is being posted to his Mission in 
Geneva.  
76. After two courses organised for English-speaking African countries in Nairobi the counterpart 
university has acquired a wealth of experience that  can be capitalized in delivering training on WTO 
issues.  It has built up a core library dedicated to WTO issues and members of the law and economics 
faculties have participated in the co-delivery of the course.  Once the course is underway, the 
University is responsible for the administrative operation of the course, as well as ensuring the 
welfare of the participants.  Overall the potential of the partnership could be measured by the positive 
evaluation by participants of the substance of the two courses, as well as from the continuous interest 
and contribution by the regional specialists.  Othe r comments and suggestions of the participants 
regarding organization, location, academic coordination, etc will be taken into account in the planning 
and implementation of future courses organised in the regions. 
77. In summary, it has been found that taking the lo ng trade policy courses to the regions, closer 
to the Members, has been a positive experience and th e involvement of regional specialists resulted in 
added relevance and context that are generally more  difficult for WTO officials to offer, and which WT/COMTD/W/125 
Page 18  
 
are not generated to the same extent in Geneva c ourses that are delivered to participants from across 
different regions. 
(b) Efficiency
 
78. The TA providers in the period under review  rendered account of a better performance in 
terms of efficiency than in 2002. 
Results  2003 2002 
 % 
Very efficient                           55 33 
Satisfactory                             37 40 
Less than satisfactory                   8 27 
Useless                                  0 0 
79. TCA welcomes this development, which demonstr ates that TA providers have become more 
aware of the importance to deliver TA and traini ng with due account of the economical use of the 
resources.  
80. Wherever it was feasible TAs were organised on a cost-sharing basis with other agencies or 
back-to-back missions were arranged to reduce travel  time and costs.  Some of the TAs benefited 
from arrangements whereby resource persons were paid  by a donor or did not ask for compensation.  
There are, however, some areas in which ther e is still room for improvement.  The following 
suggestions and comments should, how ever, be taken in light of an overall more efficient TA and 
training programme. 
81. In terms of efficiency it should be noted that the financing of most of the Geneva-based TA 
and training activities comes out of the regular budget that makes their planning easier, well in advance and more predictable.  
82. Diversification of the activities also contributed to making them more effi cient.  New kinds of 
training activities were developed in 2003, for exampl e, the Specialized course on Trade in Services. 
The training sessions were made more practice-oriented and every effort was made to maintain a high level of excellence in the training delivered.  It s hould be mentioned that trainers, from the ITTC as 
well as from other Divisions, continued to improve their training abilities.  This, as is mentioned in paragraph 42, is in large part attributable to the in-house Training-of-Trainers programme. 
83. Time allocated for the TA programme shoul d be in proportion to the volume of the 
information to be delivered with due account to the participants' experiences and expectations.  It still 
happens that the course programme squeezes complex issues into 2-3 days when they need five, 
leading to comments by the participants that they  could not absorb all the information imparted.  
Repetitions between the presentati ons of resource persons, omitti ng references to local/regional 
issues, or not inviting dialogue with the participan ts, all reduce efficiency and detract from quality.   
84. There are some delivery aspects, which may c ontribute to a better use of resources, and 
should be considered for future TAs.  For exampl e, a more frequent use of video-conferencing could 
replace some on-site missions, often taking only 24 hours.  
85. Furthermore, videos produced with a view to changing the image of WTO should be used in 
TAs and training on a more regular basis.  However,  some TA providers say that the videos would 
reduce the time for lecturing and presentations, ther efore are not included in the course programme. 
On the other hand, those TA providers who starte d using these videos, e.g. "In the heart of WTO" 
reported a very favourable reception by the participants .  It is also true that video equipment and the 
required TV sets are not always available, not easy to  move, take time, etc.  Therefore consideration  WT/COMTD/W/125 
 Page 19 
 
 
should be given to investing some resources into  their conversion into DVDs or MPEG formats, 
which can be projected from the laptop of the TA providers. 
86. The following comments made last y ear still appear to be valid.  It is important that efforts to 
improve the efficiency of TA activities be system atic.  The development of standardized training 
packages, and improved logistical and administrative backup services are among the measures that 
can contribute to this.  The positive examples of the enhanced cooperation and coordination between 
ITTC and the operational divisions are encouraged to  be continued and certainly can help ensure the 
more efficient use of the human resources and avoid possible duplications or overlaps. 
87. Finally, one should mention the health factor as key to both effective and efficient delivery of 
TAs.  In this regard one should appreciate the huge amount of and always timely work of the Medical 
Service, which has taken every possible care (keepin g records, consulting files, giving the necessary 
treatment, verifying if all the vaccinations are done on time, issuing health status certificates before every mission, (some 900 in 2003 and almost two hundr eds already by 10 March 2004),  to make sure 
that the staff is fit for the missions.  Their worklo ad has been further increased by taking care of the 
participants of Geneva-based training courses. 
(c) Relevance and Sustainability
 
88. It is these evaluation concerns that are the least  substantiated with evidence under the current 
M&E system.  As a result of a TA or training activit y it is only the participants' immediate reactions 
that can be measured and possibly generate verifi able data on the knowledge and skills transferred. 
These reactions point both to qualitative and quantitati ve aspects, which are indispensable for judging 
on their value.  However, regarding the relevance of  the TA and training and the sustainability of the 
results, only ex-post evaluations could give a more  thorough answer.  Furthermore, it would not be 
economical to introduce a system which would provide for a follow-up of these evaluation concerns at the level of each and every TA or training activity.  Therefore any ex-post evaluation of relevance and 
sustainability issues should cover a larger group of TA activities.  It is exactly the intention behind the 
ex-post evaluation that has been mentioned (paragraph 72) to be undertaken for the first time in 2004.  
It will hopefully throw light, inter alia , on theses aspects of WTO TA.  
89. Although the above considerations  represent a constraint to a value judgment on the relevance 
of the TAs and the sustainability of the results, the following observations appear to be justified.  It 
should be noted that the priority areas included in  the DDA provided the "raison d'être", the major 
anchor, of most of WTO TA and training activities in  2003.  In examining relevance issues the first 
question should be if the kind of TAs and training provided were the right answer to the needs.  
90. The views of the participants and feedback from the recipient Members reflect that there is an 
overall satisfaction with the results and outcome of the TAs and training activities.  Most of the evaluation reports highlighted the WTO areas to whic h the particular TA contributed, either at the 
regional or national levels.  In a generic sense ther eby they pointed to their relevance.  Another 
yardstick that points to the relevance of WTO TA to the Members, is if they keep the trained people in 
jobs in which they can capitalize on the knowledge and skills acquired through WTO TA.  However, 
less information is available on the extent to whic h participants' employers have actually benefited 
from the improved knowledge and skills in terms of improved implementation of WTO agreements, 
meeting notification obligations, preparing so und positions for the negotiations, etc.   
91. The second question in examining relevance issues is if the Members still need this kind of a 
TA and training.  Many delegates in committee meet ings have asked for the continuation of technical 
assistance to help them bridge the gap in capacity  deficits.  Furthermore, the feedback through 
interviews and many of the recommendations includ ed in the BTORs demonstrate that the WTO TA 
is highly appreciated by the recipients and needs to  be continued.  Very often those who received a 
basic training asked for more adva nced, specialized techni cal assistance.  TCA is convinced that the WT/COMTD/W/125 
Page 20  
 
TA and training plan for 2004 provides an adequate framework for delivering even more relevant TA 
and training.   
92. The relevance of TA and training can be incr eased, as mentioned above, if the needs of the 
Members and more specifically those of the pa rticipants are known and adequately met.  The 
evaluation reports received demonstrated a good correspondence between the expectations of 
Members and participants.  The only potential concern is whether all aspects of the needs have been 
covered to the required depth, an d whether the varying circumstances of each country at a regional 
activity or the specific needs of the participants of a national activity have been adequately reflected.  
93. Most participants indicated their willingness to apply the newly ac quired knowledge and 
skills as well as pass them onto their colleagues.  This  shows their feelings immediately after the TA 
and how relevant they felt the TA was.  This is a laudable effect.  However, there is some scepticism 
that all the participants will, as they claim, be able to use the newly acquired knowledge and improved 
capacity in their jobs on a day-to-day basis, a nd share that knowledge and capacity with colleagues 
who have not attended the activity.  This is influenced by a number of  factors, such as institutional 
background, and by the knowledge-base that the par ticipants bring with them to the TA activity.  
Furthermore, the more uneven the level of expe rience, the knowledge and skill composition of the 
group, the higher the danger that the TA is  not equally relevant to all of them.  
94. It should be reiterated, however, that finding out the participants' individual expectations prior 
to the TA and having them reflected in the programme or giving them an opportunity to highlight their expectations at the beginning of th e TA, helps to improve its relevance.  
95. On sustainability considerations it is important  that TA should not be perceived and carried 
out as one-shot exercises.  They are supposed to ac hieve sustainable results, which are taken care of 
and further developed in the nati onal environment.  Good evidence of sustained results is if 
participants in WTO courses become part of the ne gotiating team of the country or members of the 
Geneva-based country mission.  
96. In a strict sense the larger projects, e.g.  LTPCs as a programme and JITAP as a regional 
project have sufficient resources, duration and methods  to build sustainable capacities.  In any case, 
there is little point in measuring the sustainability of individual activities, which are too short, between three to four days on an average, and speci alized, to have much long-term impact.  Assessing 
whether and how a recipient country has taken care of the results of all the TA activities in which a 
country has participated is more interesting;  as is assessing the impact of the TPCs, which will be 
verified through the above-mentioned ex-post eval uation.  Furthermore, another good example of 
sustained outcome of JITAP are the inter-institutional committees established by this regional 
programme. 
97. Although sustainability of the results of TA or training activities can best be verified through 
ex-post evaluations taking place some time after th e completion of the TA, it should not be addressed 
during an ex-post evaluation for the first time.  Th e issue has to be addressed at the design phase of 
the TA and training. Indeed, the products included in  the Technical Assistance and Training Plan for 
2004 were conceived with a view to generate sust ainable results. At the individual level, the TA 
providers must think about and provide for inte grating those components in the TA programme, 
which are known from experience to have a chance of  being sustainable, e.g. reference centres, 
suitably drafted legislation, databases, repor ting procedures, documen tation, and networking 
arrangements.  The importance of well-organised ha nd-out material cannot be overemphasized.  That 
material, either on CD-ROMS or in hard-copy,  should serve as a perman ent reminder and allow 
participants to refresh their knowle dge and skills as required, and cons olidate what has been learned at 
the activity.  Quick and reliable access to databases a nd Internet sites also facilitates sustainability.  
The application of participatory methods during th e workshops, seminars, courses, etc also facilitates  WT/COMTD/W/125 
 Page 21 
 
 
sustainability.  It has been found that in almost all TA events there are sustainable elements.  The only 
exceptions are activities concerned exclus ively with information dissemination. 
98. However, there are also some other conditions  in the implementation phase of a TA that 
facilitate the sustaining of results, e.g., if the right  participants are selected, if the message from the 
TA provider(s) effectively pass, if the participan ts with their improved kn owledge and skills do not 
leave their jobs soon after the TA is completed, they do get the necessary support and opportunity to 
apply their knowledge, then the results of the TA will not fade away. 
99. The lessons learned show that the participan ts should have backgrou nds relevant to the 
activity.  Training people in WTO matters whose work has in fact nothing to do with WTO, or technical skills to people who do not need them, is not a good use of their time or WTO's resources. In 
light of the costs involved, the distances travelled,  and the impact of the resource person's absence on 
work in Geneva, there is a need to reflect on how participants might be better "targeted".  It would 
seem reasonable that at least for regional projects, the WTO could seek to be advised who it is intended to send, since there is a substantial fina ncial investment involved.  All these examples show 
that the sustainability of the results is at the very least a shared responsibility, and lies to a large extent 
in the hands of the recipients.  
100. The Secretariat may set out certain conditions th at are considered necessary for results to be 
sustained, but it is the Members who determine who will participate in WTO TA and training and 
what happens after the activity.  Career plans in  the ministries that take WTO TA into account and 
give the participants opportunities to use the ac quired knowledge; keeping c ontact with WTO experts 
inside and outside the country, organized documen tation (reports, papers, exercises, tests, best 
practices, etc.) within the recipient ministries, etc, all this certainly contributes to sustaining the results 
and helps build institutional memory.  
101. Finally, networking is also worth mentioning.  Assuming that participants do remain in the 
trade business, the connections that they establish during TA activities, be it at the national level or 
the regional level, will also contribute positively to su stainability.  There is an additional advantage to 
this in terms of bringing together representatives of government, the private sector, academia and civil 
society, and in some instances, representatives of  different branches of the same government who 
would not normally come into contact with each ot her.  It is generally recognised that the more 
interaction is fostered at the national level, the greater the likelihood that the results of TA will be 
sustained. 
V. OBSERVATIONS ON REPORTING ON TA  
102. Reporting about TA and training activities provides the most important documented 
information of their implementation, strengths and weaknesses, eventual problems, effects and impact, recommendations and suggested follow-up actions .  It is important that the right information 
should reach the right people at the right time.  
103. On the positive side it has been noted with so me satisfaction, as mentioned above, that the 
reporting system on monitoring and evaluation has started generating a large volume of valuable 
operational and evaluative information that was not available before. TCA received 45 per cent more 
reports in 2003 than in previous years.  They ha ve provided a broader basis to highlight effects and 
impacts of the TA and training programme.   
104. Overall the reporting discipline is satisfactory, mo st of the BTORs have been prepared within 
five days after the completion of the TA event.  The quality of the reports is mixed, some of them are 
analytical others are descriptive.  The above  mentioned weaknesses regarding the inadequate 
understanding of the basic design terms, e.g. objec tives and indicators, certainly influence the quality 
of the evaluative information incl uded in the reports.  There ar e a number of reasons for this: WT/COMTD/W/125 
Page 22  
 
monitoring and evaluation concepts, tools and proce dures are still relatively new, and evaluation as a 
management function integrated into the delivery of TA activities is not yet generally understood, 
accepted or exercised by the TA providers.  Evaluation culture is yet to be developed.   
105. The only exceptions are the Geneva-based trai ning courses and the three months regional 
trade policy courses, which have been subject to re gular evaluations and reporting by the staff of the 
former Training Institute (TI).  Those evaluations  are primarily focusing on the feedback from the 
participants, which have been considered in th e organization and implementation of subsequent 
courses. The knowledge, experiences and observati ons with the evaluation of those courses 
accumulated in the TI have been an asset to this re port. As mentioned earlier, with the integration of 
the TI into the ITTC, these training activities have become subject to and are now being covered by 
WTO's monitoring and evaluation system.  
106. TCA has fed back comments on the quality of the reports to their authors with compliments 
when it was due or with suggestions for future reporti ng if the quality left much to desire.  The reports 
have provided the opportunity to introduce a kind of briefing service in which design, monitoring and evaluation concepts and tools were explained.  As  mentioned in paragraph 39, the positive reaction of 
the staff concerned encourages TCA to continue with this kind of a service in order to further improve 
the quality of the training activities and the reporting of them.  
107. It has also been derived from the BTORs that a better evaluation quality can be achieved if 
more detailed explanations are attached to the reporting formats and the evaluation reports become 
consistent office-wide.  Therefore both the BTOR fo rmat and the Participants Evaluation Form (PEF) 
are  being revised in order to make them applicab le to both technical assistance and long trade policy 
courses.  Furthermore, an organised mechanism of  the discussion of the recommendations included in 
the BTORs and the monitoring of the agreed upon follow-up actions are recommended.  
VI. TECHNICAL ASSISTANCE MANAGEMENT COMMITTEE (TAMC) 
108. The TAMC has become a recognised forum to di scuss management and other issues of TA.  
It has also become a forum where individual divisions share their TA experiences with other 
divisions.  It had nine meetings in 2003.  It is  encouraging that the higher management has given 
strong support to the consolidation of the M&E system in the Secretariat's work.  
VII. THE MID-TERM REVIEW OF THE M&E SYSTEM 
109. As mentioned earlier in the report the M&E syst em which was put into effect in April 2002 
on a trial basis for two years, was reviewed by TCA in order to see how it has been serving the purposes it was expected to.  The review gave an op portunity to initiate the required refinement and 
revision in monitoring and evaluation concepts, form ats and  tools.  A brief extract from the review 
highlights its main  findings and recommendations. 
Is the M&E System effective? 
 
 - The open and consistent support to evaluation by the senior management of the Secretariat 
has been vital in bringing about the necess ary conditions to have evaluation gradually 
recognized in the future as a management f unction both at the level of ITTC and the 
Operational Divisions. 
 - The M&E system has certainly contributed to  making WTO TA more effective by sensitizing 
TA providers to design and deliver high qualit y TA projects.  It should be mentioned, 
however, that less than satisfactory progress has been made in the systematic planning and 
design of TAs.  The emphasis is still on "Just do it" but the element "to do it better, with a view to deliver the right TA" appears to be one of the considerations behind the TA.  The  WT/COMTD/W/125 
 Page 23 
 
 
BTOR reporting on the success or failure with in the framework of self-assessments has 
contributed to making the staff more results-oriented. 
 - Although it should be further explored, it app ears that the programme managers as well as TA 
providers do not yet fully recognize evaluation as a management function (in addition to other 
functions such as planning and controlling the implementation of the work) nor do they use 
evaluation as a source of information or lessons learned that could/should be applied before 
taking programme or operational decisions.  On ly minimum progress has been made towards 
establishing an "evaluation culture" in the WTO Secr etariat. It should be noted that this is a 
long process and necessitates systematic effo rts and assistance and guidance from TCA and 
continuous encouragement by the senior management.  
 - During the years ahead every effort should be made to make the M&E system work to its full 
potential.  TCA should facilitate this and beco me more proactive in delivering briefing-
training services and feed back to TA staff and disseminate lessons learned. 
 
Is the application of the M&E System efficient?
 
 - The too many and small-size TAs of short duration do not appear to be conceived along some 
substantive and well-coordinated capacity-build ing framework, which may entail duplications 
and waste.  There is a need to reconsider the planning process of technical assistance building 
on what we know best, and putting priority to areas where WTO has comparative advantage 
over other organisations.  In this regard TCA considers the Technical Assistance and Training 
Plan for 2004 an important step forward.  It not only provides for linking the great number of small activities to bigger "products" but also help s the Members to focus the discussions more 
on strategic issues rather than on small details. 
 
- In due regard to the great number of TAs a nd limited evaluation capacities, it has been the 
right approach to share evaluation respons ibilities between TCA and TA providers.  The 
introduction of self-assessment has turned out to be very cost efficient and the BTORs have 
certainly benefited from the in-depth kn owledge of the TA providers of their 
training/Capacity Building activities.  And vice- versa, it is hoped that the TA activities, and 
even the regular work of the staff  will also benefit from the results-based approach promoted 
by TCA. 
 - No additional resources were put into staff training related to the newly established M&E 
system.  Lacking such supportive training, staff applied the procedures at their best level of 
understanding and experience.  However, this  does not appear to be enough.  A better 
understanding of objective setting and capabiliti es to formulate performance indicators to 
measure its achievement is needed.  Once learne d, not much additional time is needed to be 
invested in the planning and design process.  A nd the return will be a more transparent and 
logical TA and training programme which is easy  to evaluate and feedback the results to the 
senior management and to the Donors.  
 
- On the other hand, the delivery of the TA wo rkshops, courses, etc, has visibly improved by 
applying the training skills imparted through the Training of Trainers Courses organised at the Master Trainers Institute in Ferney-Voltaire in 2002.  
 
- The application of the PEFs and BTORs as ev aluation tools did not require much additional 
work from the TA providers.  The amount and quality of data included in BTORs have 
demonstrated the usefulness of evaluation and enabled TCA to produce annual evaluation 
reports of better quality.  The BTOR appears to be the main reporting format and has also 
been used for reporting about technical miss ions, which are not covered by the current 
evaluation system.  WT/COMTD/W/125 
Page 24  
 
 - Checklists for project design and the preparation of the BTOR are available, but are not well-
known and applied. 
 - Detailed explanations included in the guidelin es on self-assessments helped the TA providers 
to prepare BTORs but the explanations on objective setting and indicators require more 
refined reasoning, further adapted to WTO TA specificities. 
 - One full-time staff, a secretary and one consu ltant working on a short-term basis developed 
the M&E system, which can be judged as a modest investment. 
 
- The housekeeping and maintenance of the system ha s been kept at low cost.  The database has 
been developed in-house.  Further investment  in strengthening the capacities of TCA would 
result in more on-site evaluations, ex-post and thematic evaluations; would facilitate 
becoming more proactive and making the di ssemination and promotion of M&E concepts, 
tools and practices more effective; would resu lt in further developing the evaluation reports 
database and developing and evaluation web-site. 
 
Recommendations 
 
110. Based on the findings and conclusions included in this mid-term review, the following 
recommendations were submitted for consideration and implementation by the units concerned:  
- Future evaluation guidelines by TCA should reflect the organisational changes emanating 
from the merger of the Training Institute and the Technical Cooperation Division into the 
Institute for Training and Technical Cooperation (ITTC).  
 
- The revised BTOR format, should include bette r explanatory remarks for the formulation of 
objectives.  The term of indicators of achieve ment should be renamed and re-explained as 
performance indicators supported by targets.  The BTOR format should invite more analytical information about the contribution of the TA to the implementation of the DDA and also 
invite the views of the TA provider on the perfo rmance of external resource persons hired by 
WTO (performing outsourced functions).  Further criteria regarding the rating of the overall 
success or failure of the TA should also be a dded.  Accordingly, the PEF format has to be 
revised with a view to make it compatible with  the information required by the BTOR format. 
 - TCA should introduce regular briefing/training sessions for TA providers and continue to 
disseminate the lessons learned from the BTORs and its own on-site eval uations by its annual 
reports, briefing and training sessions.  
- TCA should make evaluative information (M&E  policy, strategy and procedures) available on 
WTO's Intranet, including major reports, best practices, checklists and lessons learned. 
- With reference to the TA and training plan for 2004, ITTC and the Operational Division 
should ensure that within every product in the Plan the objectives of the related individual TA 
and training activities are set out with due regard to the product objectives.  
- ITTC and the Operational Divisions should establish a monitoring mechanism, which 
provides for a regular follow-up on recommenda tions included in BTORs by TA providers 
and the application of lessons learned included in TCA's Annual Report to the CTD as 
appropriate. 
- ITTC and the Operational Divisions should ensu re that all TAs are reported about and a copy 
of the BTORs and the related PEFs are sent to TCA  for information and feedback.   WT/COMTD/W/125 
 Page 25 
 
 
- TAMC should provide a forum for discussing those recommendations of BTORs, which 
deserve follow-up or consideration by other ODs . Also within the TAMC the verbal account 
of TA activities by individual divisions should also include information about effects and 
impact. 
VIII.  SUMMARY CONCLUSIONS 
111. Some conclusions and lessons learned have al ready been drawn and suggestions put forward 
in earlier sections of the report.  This section a ttempts to summarize some more general conclusions 
and highlight some lessons emerged from the evalua tions carried out in 2003 and put forward some 
ideas about how to achieve further improvements.  I ndeed they are not much different from those put 
forward last year and from those resulting from the above-mentioned mid-term review. 
112. While it is ultimately for Members to judge, it can be said that the Secretariat has made a 
remarkable effort to fulfil the ma ndate it has been given to provide  technical assistance and capacity-
building.  Technical assistance and training activities were considered a priority, and continuous efforts were made to ensure that they were e ffective, well-targeted, and coordinated with other 
agencies to get the best results. 
113. The overall quality of the assistance delivered h as remained very high.  The resource persons 
in general demonstrated an impressive command of  their subjects, and a thorough professionalism in 
delivery.  Teaching skills of TA providers improved through participation in the training-of-trainers 
courses organized in 2002 were put into good use.  The TA activities reflected more conscious efforts 
in improving teaching methods, presentations and also  more attention has been paid to applying 
participative methods, although a common comment in  feedback is that still more can be done.   
114. The monitoring and evaluation  system and procedures introduced two years ago have 
started generating valuable data on the effects a nd impact of TA.  Further improvements can be 
achieved if: 
- the TA needs of the Members are systematically identified and analysed; 
- the design of TA projects follows the sugg ested guidelines, with clear, realistic and 
 achievable objectives and related performance indicators;   
- the delivery process is improved by cont inuous cooperation between ITTC and the 
 Operational Divisions; and 
- the information gathered through systema tic monitoring and evaluation is used in 
 programming and operational decisions. 
 
This means that not only will high quality TA be de livered, but it will be more results and outcome 
oriented.  For TCA the major lesson learned is th at the introduction of any concepts, tools and 
procedures that touch on management should be supported by appropriate training of the staff 
concerned.  WT/COMTD/W/125 
Page 26  
 
  
ANNEX 1
 
 
EVALUATION MISSIONS CARRIED OUT BY TCA IN 2003  
 
 
Botswana  Regional Seminar Trade and Invest ment for English-speaking African Countries, 
  03-06 February   
South Africa Workshop Agriculture and Services for SADC Member Countries, 03-07 February 
 St Kitts  Regional Workshop SPS & TBT, 22-24 July  & Nevis  
Burkina Faso Advanced Training Programme for se nior civil servants in French-speaking African 
  countries on the Doha Developm ent Agenda, 28 July – 08 August 
 Costa Rica Regional Seminar  on Trade Fac ilitation for Latin American and Caribbean  
  Countries, 26-28 August  
Austria  Short Trade Policy Course on Applie d Economic Policy, 29 September – 3 October 
 China  Advanced Course for Asian and Pacific Senior Government Officials:  The DDA - 
  Enhancing the Participation of Asian and Pacific Economies in the Multilateral  
  Trading System.  Discussions of WTO Trade Negotiation Issues, 27 Oct – 01 Nov 
 
Ecuador WTO BID INTAL Seminar, 17-19 November (Rules)  Hungary Workshop Trade & Environment for Cent ral, Eastern European & Central Asian  
  Countries, 8-14 December  
Qatar  Workshop on Trade Policy and the MTS, 6-11 December 
 Jordan  National Trade Policy Course, 14-18 December   
 
 
__________