WT/COMTD/W/199  
 
3 July 2013  
(13-3471)  Page: 1/18  
Institute for Training and Technical Cooperation   
 
  
 
ANNUAL REPORT ON MONI TORING AND EVALUATION 
OF TECHNICAL ASSISTANCE AND TRAINING  
2012 
 
 
Table of Content 
ACRONYMS ....................................................................................................................... 2  
1   EXECUTIVE SUMMARY ................................................................................................. 3  
1.1   Introduction of Results- Based Management (RBM) in  2012 .............................................. 3  
1.2   Monitoring & Evaluation of Performance in 2012 ............................................................. 3  
1.3   Lessons lear ned for 2014 -15 ....................................................................................... 3  
2   INTROD UCTION  .......................................................................................................... 4  
3   MONITORING & EVALUATION  OF PERFORMANCE IN 2012 .......................................... 4  
3.1   Introducti on of RBM ................................................................................................... 4  
3.1.1   Planning stage ........................................................................................................ 5  
3.1.2   Implementation stag e (Formative Ev aluation) ............................................................. 6  
3.1.3   Evalua tion st age ..................................................................................................... 6 
3.2   Immediat e result s ..................................................................................................... 7  
3.2.1   E- Learning ............................................................................................................ . 7 
3.2.2   Face-to-fa ce activities ............................................................................................. 9  
3.2.3   Academic Support Pr ogramme ................................................................................. 10 
3.3   Longer-t erm results .................................................................................................. 11 
3.3.1   Transparen cy obligations ........................................................................................ 11 
3.3.2   Count ry reviews .................................................................................................... 12 
3.3.3   Kenya ................................................................................................................. . 13 
3.3.4   Senegal ................................................................................................................ 14 
4   LESSONS LEARNED .................................................................................................... 15  
4.1   Management of WTO Technical Assistance .................................................................... 15 
4.2   WTO Technical Assistance  Offer .................................................................................. 16 
4.3   Implementa tion of RBM ............................................................................................. 17 
 WT/COMTD/W/199 
 
- 2 - 
 
  ACRONYMS 
ACP  African, Caribbean and Pacific group of states 
BTOR  Back-to-Office Report CD  Compact Disk 
CEECAC Central and Eastern European, Central Asian and Caucasus Countries 
CHF  Swiss franc CTD  Committee on Trade and Development 
CTS  Consolidated Tari ff Schedules Database 
DAC  Development Assistance Committee (OECD) DDA  Doha Development Agenda DVD  Digital Video Disk 
DVS  Kenya Department of Veterinary Services 
EIF  Enhanced Integrated Framework EU  European Union 
IDB  Integrated Database 
ITTC  Institute for Training and Technical Cooperation KEBS  Kenya Bureau of Standards 
KEPHIS Kenya Plant Health Inspectorate Service 
LDC  Least-Developed Country MA  Market Access M&E  Monitoring & Evaluation 
MTS  Multilateral Trading System 
NAMA  Non-Agricultural Market Access NGTF  Negotiating Group on Trade Facilitation 
NTB  Non-Tariff Barrier 
PEF  Participant's Evaluation Form PLS  Progressive Learning Strategy 
NPDIP  National Programme of Development of Intellectual Property 
RBM  Results-Based Management RTPC  Regional Trade Policy Course SIDA  Swedish International Development Cooperation Agency 
SPS  Sanitary and Phytosanitary 
TA  Technical Assistance TAME  Technical Assistance Monitoring and Evaluation Unit 
TBT  Technical Barriers to Trade 
TCA  Technical Cooperation Audit Unit TPR  Trade Policy Review 
TRIPS  Trade-related aspects of Intellectual Property Rights 
TRTA  Trade-related Technical Assistance WCP  WTO Chairs Programme WIPO  World Intellectual Property Organization 
WTO  World Trade Organization WT/COMTD/W/199 
 
- 3 - 
 
  1  EXECUTIVE SUMMARY 
1.1  Introduction of Results- Based Management (RBM) in 2012 
1.1.  The introduction of Results-Based Management (RBM) for Technical Assistance (TA) activities 
led to the establishment of a monitoring and evaluation team within the Institute for Training and 
Technical Cooperation (ITTC).  The existing evaluation system was reviewed and some new 
elements introduced to have a complete RBM Monitoring & Evaluation (M&E) system.  Implementation of the system is on-going.  The sy stem will keep relying on self-evaluation of the 
immediate results by the trainers.  Future re views and ex-post evaluations of programmes to 
measure longer-term results would have to be based on a deeper analysis of facts and use key indicators of performance. 
1.2  Monitoring & Evaluati on of Performance in 2012 
1.2.  The performance of E-Learning courses could be measured on the basis of a set of objective 
performance indicators.  The E-Learning courses were very successful in 2012, with an average success rate of 81%.  E-Learning courses on Sanitary and Phytosanitary (SPS) and Trade and 
Environment have traditionally been the most successful amongst them, whilst some others (e.g. 
Multilateral Trade Agreements, Services or Market Access (MA) for Goods) were more challenging for participants.  French language E-Learning co urses used to have a lower success rate, but had 
been catching up with English and Spanish courses in 2012. 
1.3.  For face-to-face TA activities, there were hardly any objective performance indicators to help 
measure their immediate results, as most Back-to-Office Reports (BTORs) kept relying almost exclusively on the opinion of participants to evaluate the activities.  A pilot on the use of a final 
exam in two Regional Trade Policy Courses (RTPCs) confirmed that more objective indicators could 
also be used in face-to-face acti vities, as has always been the case  for E-Learning.  This would also 
require improvements in the quality and content of BTORs. 
1.4.  The mid-term review of the WTO Chairs Programme (WCP) showed that this programme had 
been successful in stimulating the academic activity on trade-related issues, but less so in monitoring the quality of this production and reaching out business communities or policy makers.  
The current WCP will reach its end in 2013.  Th e overall support from WTO to the academic 
institutions including the WCP was CHF1.8 million  in 2012, representing approximately 11% of the 
total expenses on Trade-Related Technical Assistance (TRTA).  A decision is needed on the form and substance of the future involvement of the WTO with the academic world, and on the human 
and financial resources that should be allocated to it, taking other TA priorities and the lessons 
learned into account. 
1.5.  Certain results could be measured over the years in the area of agricultural notifications.  The 
rate of outstanding notifications had been  slowly going down since 2009 when the WTO 
programme was launched at the request of Members.  A number of changes could also be noted since 2006 in the domains in which WTO TA had concentrated in Kenya and Senegal.  However, 
these two pilot country reviews showed the difficulty  of gathering precise enough data a posteriori.  
Synergies with Trade Policy Reviews (TPRs) need to be explored to overcome this challenge. 
1.3  Lessons learned for 2014-15 1.6.  Continue to have contingency plans to deal with the current human and financial challenges 
and strive to do more with less; benchmark each of the three levels of progressive learning to 
ensure progressivity in the training activities; improve needs assessment to ensure that TA activities are designed to better respond to the identified needs, while also promoting quality 
instead of quantity.  WTO should continue to en sure that effective modes of delivery of TA are 
employed and that the Organization moves from measuring outputs to measuring outcomes in the next three years, with the full participation and support of the stakeholders. WT/COMTD/W/199 
 
- 4 - 
 
  2  INTRODUCTION 
2.1.  In order to implement RBM for its technica l assistance and training activities in 2012, the 
Technical Cooperation Audit unit (TCA) was integrated into the Institute for Training and Technical Cooperation (ITTC), where it became the Technical Assistance Monitoring & Evaluation unit 
(TAME). 
2.2.  For the sake of concision, the information on Input and Output presented by the Secretariat 
in its report on the implementation of the Techni cal Assistance and Training Plan (TA Plan) has not 
been repeated here.
1  It has been assumed that Member s were already acquainted with it. 
2.3.  The report is divided into three parts of equal importance.  The first one presents the work 
done in 2012 to introduce RBM for TA activities and the changes introduced in the Monitoring & 
Evaluation methodology resulting from it.  In the following chapter, the immediate and longer-
term results of the WTO TA that could be measured in 2012 are presented to Members.  
Recommendations for improvement are then summarized, presented as lessons learned during the year under review. 
3  MONITORING & EVALUATION OF PERFORMANCE IN 2012 
3.1.  The design of the M&E system for WTO TR TA was completed and approved by management 
for implementation in December 2012.  The online pl atform of the system is being developed.  To 
ensure effectiveness of the system, clear roles and responsibilities and formal lines of authority 
have been established.  The sections and staff who will be in charge of collecting, analysing, and 
reporting performance information have been define d.  There is need to ensure ownership of the 
M&E system by all stakeholders to increase its use, reach and effectiveness. 
3.1  Introduction of RBM 
3.2.  The Monitoring and Evaluation (RBM M&E) framework is the strategic tool for achieving 
technical assistance targeted results (outcomes).  The system, once fully operational will provide a 
continuous flow of information useful, both internally and externally, to measure the progress 
being made towards achievement of the expected re sults, strengthen ITTC's capacity to improve 
performance and management of TRTA activities and enable all TA services and their results to be 
measured against stated objectives, so as to pr ovide accountability to all relevant stakeholders. 
3.3.  Further to the decision to start implemen ting RBM for TA activities in 2012, the Technical 
Cooperation Audit unit (TCA) was subsumed into the ITTC on 1 March 2012 and became the Technical Assistance Monitoring & Evaluation Unit (TAME).  Its main responsibility in 2012 was the 
introduction of RBM. 
3.4.  The pre-existing evaluati on system progressively developed by TCA over the years already 
i n c l u d e d  m a n y  o f  t h e  m a i n  c o m p o n e n t s  o f  a n  RBM system.  With the support of an outside 
specialist, it was reviewed to fill any remaining gap and an RBM Monitoring & Evaluation toolkit 
was developed. 
3.5.  The diagram below sets out the designed RBM M&E system premised on three key stages, 
namely Planning, Implementation (Monitoring) and Evaluation which also builds upon the existing 
tools for data collection and dissemination. 
                                               
1  See document WT/COMTD/W/197. WT/COMTD/W/199 
 
- 5 - 
 
  
 
3.6.  The measurement of results at the heart of  any RBM system, presupposes the identification 
of the set of performance indicators relevant to measure the output and outcome of the TA provided by the WTO and for which reliable information is available or can be obtained.  The Secretariat worked in 2012 on the development of a set of such indicators, by subject or by 
programme.  Some of these indicators were test ed during the pilot country reviews conducted 
during the year (see below), thanks to the active contribution of some beneficiaries.  Further work is required in this domain to arrive at a reliable and more stable set of indicators for the various TA 
programmes of the WTO. 
3.7.  The RBM system could not fulfil its aims wi thout relying on a database to collect and analyse 
the data on the activities, on their intended output s, on performance indicators and on results.  It 
had been decided in 2011 to move to electronic fo rms and on-line work-flows for most of the key 
steps in the process, from request to authoriz ation, implementation and assessment of a TA 
activity.  By saving paper, streamlining procedures  and feeding the relevant data directly into the 
ITTC TA database, this should contribute to increase the efficiency of the Secretariat.  The 2012 
review confirmed the validity of this choice.  These electronic forms are currently being developed 
and the database of TA activities is being adju sted accordingly.  The content of the database 
should also be adjusted to the new needs of the RBM system in terms of analysis and reporting.  
This will require a clear segregation with the historical data already stored in the database.  The 
intention is to have the whole system operational in the second part of 2013. 
3.8.  Finally, the 2012 review focused on the logi cal framework on which the RBM system is based 
and on the planning of the activities during the year.  The measurement of the resources dedicated 
to each activity now implemented by the Secretaria t should also provide a more precise picture of 
the inputs used and help make a link between resources employed and results achieved. 
3.1.1  Planning stage 
3.9.  During the Planning stage, stakeholders analysis and needs assessments will be undertaken, 
lessons learnt from the previous period, and a ne w Plan for TA activities is to be developed. 
3.10.  The various forms that are used to initiate, organize and assess a TA activity have been 
reviewed and reformed in 2012.  A new Activity Request form was designed, which requires the 
organizers to define already at this early stage in the life cycle of the activity its intended objectives (outputs), the performance indicators which will be used to measure their achievement, and the assumptions that cannot be dealt with through the TA activity, but must be taken into WT/COMTD/W/199 
 
- 6 - 
 
  consideration when designing and implementing  it and may impact the achievement of the 
immediate results of the activity. 
3.11.  The form used by beneficiaries to reques t national activities was reformed following the 
same logic.  The requesting country needs to firs t explain the problem that the requested activity 
is intended to solve, or the situation it should try to modify.  In consultation with the beneficiary, 
this then leads to a definition of objectives corresponding to the needs identified in the request.  The performance indicators that should be used to measure if the activity fulfilled its intended 
objectives also have to be specified in the request. 
3.1.2  Implementation stag e (Formative Evaluation) 
3.12.  In the Implementation stage the information generated in the Planning stage (the 
benchmarks) will be used as the basis for appraising, monitoring and evaluating the delivery of the 
TA services.  The benchmarks underpin the implementation and monitoring process. 
3.13.  The Formative Evaluation reporting will fo cus on achievement of outputs and the realization 
of assumptions, with the Development Assistance  Committee (OECD) (DAC) criteria of Relevance, 
Equity, Efficiency and Effectiveness used to  provide overall coherence and analysis.  
3.14.  The Formative Evaluation reporting is geared towards internal use, providing unit managers, 
ITTC Director, and senior managers within the WTO with evidenced based performance data on 
how resources have been used and progress towards the delivery of TA activities achieved.  Once 
the internal reporting and review has been comple ted, the final Formative Evaluation Report will 
be disseminated more widely to external stakeholders and Members through the Committee on Trade and Development (CTD). 
3.15.  The system will remain largely based on self -evaluation of the immediate results of activities 
by the trainers themselves, for obvious reasons of cost efficiency (formative evaluations).  However, future self-evaluations will need to be based on more objective performance indicators 
identified prior to each activity, and not exclusively on the opinion of participants as used to be the 
case until now.  For example, as most activities ha ve as one of their primary objectives to increase 
participants' knowledge of one or several WTO ag reements, basing the assessment of the results 
on quizzes or exams is a much more objective measure of how much they learned than the 
participants' own subjective assessment. 
3.16.  The Back-to-Office Report (BTOR) has traditionally been the instrument through which the 
o r g a n i z e r s  o f  t h e  W T O  T A  a c t i v i t i e s  h a v e  r e p o rted on what they organized, implemented and 
achieved.  This report also undertook a facelift  to comply with RBM requirements.  The revised 
report should provide a better quality of information on the measurement of the immediate results achieved at the end of an activity, and estab lish a clearer link between these measures and the 
assessment of the activity. 
3.1.3  Evaluation stage 3.17.  The Evaluation stage focuses on analysing results and learning the lessons to be taken 
forward into future plans.  It is also a time when the opportunities for sustaining the TA service’s 
benefits should be investigated, either by crea ting on-going support for them or mainstreaming 
them into partner organizations.  The Evaluation stage will be undertaken biennially and focused 
on outputs to outcomes and assumptions. 
3.18.  The ex-post evaluations of certain progra mmes conducted in the past by TCA every year 
would need to become more systematic.  However,  this will depend on the resources dedicated to 
such evaluations in the future, which will in turn depend on the evolution of the overall resources 
made available to the WTO to finance TA activities. 
3.19.  The country reviews conducted by TCA since 2009 would need to be based more 
systematically on a set of agreed performance indicators to better measure the longer-term results 
of the TA provided by WTO.  To that effect, TA ME initiated in 2012 a pilot project designed to see 
the extent to which such results could be measured in a small sample of countries.  The initial intention was to include four countries covering Africa, Asia and Latin America, with one or two WT/COMTD/W/199 
 
- 7 - 
 
  least-developed countries (LDCs) amongst them.  Kenya and Senegal agreed to participate in the 
experiment, but efforts to enlist countries from other regions remained unsuccessful. 
3.20.  An agreement was reached with the Kenyan and Senegalese authorities to focus on a list of 
only six or seven subjects on which approximat ely ¾ of the WTO TA had concentrated over the 
past six years.  For each of these subjects, a list of performance indicators - quantitative and 
qualitative - was then agreed.  However, capturing retrospectively precise enough data corresponding to these indicators has proven elus ive to date, and it has not been possible to go 
beyond generalities in most cases.  Work will continue in 2013 to see if more precise data could be 
gathered to support the analysis of the results of TA, at least in certain domains.  Coordinating such analysis with the TPR of the country may also be tested to see if this gives access to better information.  Synergies with the M&E programmes set up in certain LDCs in the context of the 
Enhanced Integrated Framework (EIF) could also be explored. 
3.2  Immediate results 3.2.1  E-Learning 
3.21.  Measuring the immediate results of E-Learning courses is much easier than for most other 
activities, since the results of each participant can be monitored individually and completion of an on-line training always requires passing an exam  at the end.  The performance indicators below 
were used to measure the immediate results of a course: 
 Success rate (proportion of those who passed the exams amongst those who 
actually completed the course) - the passing rate was 50% on average of correct answers in the final exam at introductory level, and 60% on average at intermediate 
level, without failing more than one exam. 
 Distinction rate (proportion of successful participants who passed the exams with 
more than 90% of correct answers on average) 
 Rate of correct answers in the exams, kn owing that the passing rate is 50% on 
average in the course " Introduction to WTO ," and 60% on average in the other 
courses. 
 Dropout rate (proportion of those who did not complete the course within three 
months) 
Course Attended Successfully 
completed Success 
Rate Drop 
out Dropout 
Rate Correct 
Answers Distinction Distinction 
Rate 
Agriculture 281 178 81% 61 22% 82.6% 49 28% 
Development 159 124 84% 11 7% 86.8% 48 39% 
Environment 325 264 88% 25 8% 87.2% 132 50% 
Introduction to 
WTO 1,500 1,056 78% 144 10% 80.6% 251 24% 
Multilateral Trade 
Agreements 409 206 72% 121 30% 83.9% 48 23% 
Trade Economics 122 104 90% 6 5% 82.9% 42 40% 
Market Access for 
Goods 193 112 80% 53 27% 78.8% 14 13% 
MA Databases 23 22 96% - 0% 84.0% 6 27% 
Rules 228 183 85% 12 5% 85.5% 73 40% 
Services 355 189 78% 113 32% 83.1% 52 28% 
SPS 504 371 83% 58 12% 88.4% 188 51% 
TBT 278 217 84% 20 7% 87.4% 95 44% 
TRIPS 258 181 82% 38 15% 83.9% 50 28% 
My Course 3 2 67% - 0% 80.0% N/A N/A 
Total 4,638 3,209 81% 662  14% 83.8% 1,048 33% 
 WT/COMTD/W/199 
 
- 8 - 
 
   
3.22.  These results speak for themselves.  An average success rate of more than 80% is a very good result, in slight progression compared to 2011 (+ 2 percentage points).  The standard deviation from this average (6%) is limited, as success rates in the various courses was fairly 
homogeneous.  The lowest rate of success (72%) was registered in the intermediate level course 
for generalists (multilateral trade agreements).  This is a continuation of the trend in previous years, despite a marginal improvement.  The ne w courses on Market Access databases and trade 
economics had very high rates of success, but these figures may not be entirely representative 
given their limited number of participants in 2012 (they were first offered on 1 October and 10 August respectively).  Otherwise, Trade and Environment continued to be the most successful 
course in 2012 as in 2011. 
3.23.  The average rate of dropouts was reduced from 17% in 2011 to 14% in 2012.  Services, 
Multilateral Trade Agreements and Market Access for Goods courses had dropout rates of around 
30%, whilst a number of other courses registered rates of a few per cent only.  The rate of 
distinction amongst successful participants gave an equally contrasted picture, with rates as high 
as 50% (SPS and Environment) and as low as 13% (Market Access for Goods), a continuation of the trend observed in 2011. 
3.24.  Overall, this analysis shows certain differe nces between the courses, as some of them have 
traditionally had higher rates of success (e.g. trade and environment, SPS), whereas some others have been more challenging for participants (e.g. Multilateral Trade Agreements, Services or Market Access for Goods), even if they remained successful. 
3.25.  The same performance indicators can also be  used to analyse the results of the courses by 
linguistic version, since they reach different publics. 
Language Attended Successfully 
completed Success 
Rate Drop 
out Dropout 
Rate Correct 
Answers Distinction Distinction 
Rate 
English 2,746 1,927 82.5%  368 11.3% 84.5% 641 33.3% 
French 915 582 79.3% 184  20.8% 81.7% 137 23.5% 
Spanish 977 700 81.9% 110 12.6% 84.9% 270 38.6% 
Total 4,638 3,209 81.4% 662 14.4% 83.8% 1,048 32.7% 
 
3.26.  There are small success rates differences between the courses in the three languages.  Participants in French-language versions of the courses tended to have slightly lower results than 
their counterparts using the English or Spanish version of the same course. 
3.27.  However, French-language courses impr oved on all accounts in 2012 compared to the 
previous year: the growth of their success and di stinction rates was respectively three and ten 
times that for English courses, whilst their dropout rate went down much more rapidly (-13% 
compared to -2% for English courses).  The inte rest in the French-language E-Learning courses 
grew by 14% in 2012, compared to 7% and -1% for English and Spanish courses respectively, in 
spite of the lower growth of the number of cour ses offered in French (+2 courses when 4 new 
courses were offered in English in 2012).  Overa ll, though the performance remained somewhat 
lower in French courses, it was progressively catc hing up with the other two linguistic versions.  
The evolution of the situation will be closely monitored in 2013. 
3.28.  An analysis of these results by gender showed that women's average success rate was a 
few percentage points higher than men's one.  The difference was 2 percentage point in 2010; 4.5 in 2011; and 3.6 in 2012.  However, these averages hide significant variations between the 
courses, as women were far from bein g always more successful than men. WT/COMTD/W/199 
 
- 9 - 
 
  
 
3.2.2  Face-to-face activities 
3.29.  The performance indicators 
available to monitor the results of face-
to-face activities are certainly less 
precise than in the case of E-Learning: in 2012, only two activities used a final 
exam to measure the performance of 
their participants.  The immediate results of the other activities should be measured in the BTOR by the trainers 
themselves.  As BTORs are consequently 
the main evaluation tool for face-to-face activities, following past 
recommendations from TCA, an effort 
was made since 2009 to make them more systematically available.  The 
measures taken in 2012 in this regard 
have produced good results and the reportin g rate for the year edged closer to 100%. 
3.30.  In reality, a fully-fledged BTOR was availabl e for only three quarters of the TA activities in 
2012.  The remaining 25% of activities were reported through simple memos or notes, which in 
most cases deprived the Secretariat some essential information, such as the objectives of the 
activity and an assessment of their achievement at  the end.  In 2012, the format and content of 
BTORs was revised to align them with the DAC criteria and make sure that in the future BTORs 
would include the information required by the WTO reporting guidelines. 
3.31.  In 2012, the use of an exam was tested in the RTPCs for the Central and Eastern European, 
Central Asian and Caucasus countries (CEECAC) and Middle East region.  The result for each 
participant took into account participation during the course (30%) and the final exam results 
(70%).  Of the 21 participants in the CEECAC RT PC, four received a certificate with distinction 
(19%), 15 received a certificate of success (71%) and two who did not reach a combined pass mark in both the final exam and participation in the course received a certificate of attendance 
(10%).  For the 28 participants in the Middle East RTPC, these figures were respectively 4 
certificate with distinction (14%), 16 certificates of success (57%) and 8 certificates of attendance (29%).  The overall success rate of the courses was 90% and 71% respectively.  This 
experimentation with exams showed that the te sting mechanism needed several improvements. 
3.32.  This experiment proved very valuable for the Secretariat, as it gave a much more objective 
measure of the actual results achieved by participan ts at the end of the course.  It gave a strong 
incentive to the participants to work hard during the course and learn as much as they could.  As 
such, it could be a powerful tool to maximize the immediate results of the WTO activities.  It also showed that the approach used for E-Learning courses could be replicated in face-to-face activities, with some adjustments.  Finally, it broke a taboo about the possibility to fail a 
participant at the end of a face-to-face activity. 
3.33.  Most other face-to-face activities were evaluated by the trainers themselves on the sole 
basis of the feedback from Participants' Evaluati on Forms (PEFs).  At the end of each activity, 
participants are requested to assess on a scale of 1 (minimum) to 5 (maximum) different aspects, 
including in particular whether each of the objectives assigned to the activity at its beginning had been achieved.  The opinion of the participants sh ould normally be only one of the elements taken 
into account by the trainers when evaluating the immediate results of the activity they organized 
and delivered.  In practice, other more objective criteria, such as the results of quizzes or exercises, relevance of questions raised, etc., were very rarely used in the BTORs in 2012, even when they were available.  As a result, most face-to-face activities were mainly evaluated by the 
participants only. WT/COMTD/W/199 
 
- 10 - 
 
  3.34.  An analysis of the BTORs revealed that the activities organized in 2012 had on average 2.5 
objectives each (one third of the activities had two objectives, another third had three and the last 
third was split almost equally between the two extr emes).  Three quarters of the activities for 
which a BTOR was available had as one of their objectives to improve the knowledge of 
participants about one or several of the WTO Agreements.  The improvement of the analytical or 
negotiating skills of the participants came next (3 8%), followed by the improvement of the general 
understanding of the WTO (29%), the mastering of  the provisions related to transparency and 
notifications (24%), and the reinforcement of the au tonomy of participants in finding the relevant 
provisions, information or documents on a particular subject (21%). 
3.35.  When evaluating their own achievement of  the immediate objectives of the activities, 
participants gave an average mark of 4.5 out of a maximum of 5.  Am ongst the 413 immediate 
objectives assigned to TA activities for which data was available in 2012, only 25 were given a 
mark 3 and two a mark 2 (5 being the maximu m and 1 the minimum).  This means that the 
achievement of 93% of the objectives was rated as satisfactory or outstanding in 2012.  Whilst 
there is no doubt that the TA provided by the WT O was of the highest quality and that participants 
got the most out of it, these figures look too good to be entirely true.  As demonstrated when more objective testing methods are used (for E-Learning courses or in the pilot conducted in 2012 
in the RTPCs for the CEECAC and Middle-East regions), the actual success rate must probably be 
somewhere between 70% and 80% on average. 
3.2.3  Academic Support Programme 3.36.  Under the Academic Support Programme, th e main event of the year was the Mid-Term 
Review of the WCP, for which an assessment mission  by independent external evaluators was sent 
to each of the 15 WTO Chairs.  The evaluators examined relevant documents, data, research 
papers, receipts and records in order to assess whether key components of the WCP had been 
developed and whether the funds provided were us ed appropriately to achieve the objectives of 
the programme.  The review addressed the enhanced research capabilities created on trade policy, the development of enhanced teaching and curricu lum, the enhancement of the outreach of the 
Chairs, their ownership of the Programme and its long-term sustainability, their financial 
management and any knowledge sharing they wo uld have engaged in.  The evaluations were 
presented to the WCP Annual Conference early July.  The Mid-Term Review gave a rather contrasted picture of the situation. 
3.37.  Some Chairs performed better than others, producing significant amounts of outputs of high 
quality and strengthening their future sustainability  potential.  Out of the 15 Chairs, the mid-term 
review showed that two were significantly underpe rforming.  This represents a success rate of 
more than 85%.  However, the main challenge iden tified by the Review was the attribution of the 
various activities announced by the Chairs to the Programme, as in several instances it was not entirely clear if they had resulted from the WTO support or not.  The value actually added by the 
Programme to the activities regularly carried out by the various Chair holders could not be 
measured.  In this regard, it seemed that the "WTO" brand - more than the financial grant - could be the main asset of the programme, as it increased significantly the national and international standing and visibility of the Chair holders on the academic scene. 
3.38.  The Programme acted as a catalyst in generati ng a significant amount of activity in the area 
of curriculum development, research and outreach, the three pillars of the Programme.  It was felt 
that, on average, it had contributed to enhancing institutional capabilities in the universities 
holding a Chair. 
3.39.  The overall volume of activity 
remained fairly stable compared to 2011, 
but the mix between the three pillars of 
the programme changed significantly.  The number of research publications 
increased by 30%, which might result from the efforts made by the Chairs in the previous years to 
stimulate research.  Activities for the development of new curricula and new courses went down in 
almost the same proportion, which seemed natural since new courses cannot be multiplied and existing ones revised indefinitely.  Finally, the volume of outreach activities also went down by 
more than 10%, which was cause for concern on the part of the evaluators. WCP Activities 
Year Research Curriculum Outreach  Total 
2010 47 31 58 136 
2011 140 50 122 312 
2012 183 37 106 326 WT/COMTD/W/199 
 
- 11 - 
 
  3.40.  The evaluators raised several concerns about the kind of research ascribed by the Chairs to 
the Programme.  Whilst the overall volume of research may look satisfactory, the absence of 
quality control on the outputs was an important caveat, especially given the high value attached to the WTO brand.  It was recommended to focus less on quantity and more on quality.  This would 
lead, as recommended, to give priority to publications in English (a significant proportion of the 
research was published in local languages) in peer-reviewed periodic, if possible of international standing (only a small minority of the research projects financed by the programme actually met 
these criteria). 
3.41.  Progress in the development of new or updated curricula was more uneven amongst the 
Chairs.  The courses offered tended to be multidisciplinary, which could limit the opportunities for students to specialize on trade issues.  Competition with other universities offering similar 
programmes was spotted in certain places.  The evaluators noted that the sustainability of some of 
these courses was directly linked to charging fees to students, a practice adopted by most but not all Chairs. 
3.42.  The evaluators also pointed to the limited evidence of impact of the research produced on 
national or regional policies in the area of international trade.  Here again, if the total volume of outreach activities might look satisfactory, ther e was limited evidence that they addressed the 
appropriate publics and had any influence on them.  Only a small minority of Chairs seemed to 
have established appropriate connections with governmental authorities.  These links actually predated the WCP.  Chair holders remained entirely  free, for example, to define the priority areas 
of their work, and WTO did not interfere in matters re lating to research topics.  If this gave strong 
ownership of their work to the Chairs, this might also have limited its political relevance.  
Emphasis was placed on linking research and ou treach with policy-makers towards the end of 
2012. 
3.43.  As a result, in its present form, the WC P appeared to have been quite successful in 
stimulating academic activity on trade-related issues, but less so in monitoring the quality of such production and in influencing policy makers. 
3.44.  These results have to be assessed against the significant resources dedicated to the 
programme.  In 2012, out of approximately CHF 1. 8 million dedicated to the Academic Support 
Programme (representing approximately 11% of the total WTO resources dedicated to TA),
2 the 
WCP used CHF 1.3 million.  This Programme is supported by an Advisory Board comprised of 
23 academics acting on a pro bono  basis and benefited 15 academic institutions.  A number of 
WTO staff members also supported the WCP upon request from the Chairs. 
3.3  Longer-term results 
3.45.  The measurement of the longer-term resu lts of the TRTA provided by the WTO to its 
Members and Observers is not the least of the ch allenges ahead of the Secretariat, for both 
methodological and practical reasons.  This has been abundantly documented in the literature on 
the subject.  Leaving this important caveat aside, the Secretariat has undertaken since 2009 to 
measure some of the results produced in the domains in which it provides TRTA to beneficiaries.  When requesting the Secretariat to manage its TA programme on the basis of results, Members 
have indeed agreed that it would be assumed that, should results be measured in those areas in 
which the WTO had provided TRTA, part of the merit would be attributed to it. 
3.3.1  Transparency obligations 3.46.  Every year, when presenting the provis ions of the various agreements, many WTO TA 
activities insist on the transparency obligations  imposed on Members.  In 2012, a quarter of the 
face-to-face activities even had as one of thei r main objectives a proper understanding by the 
participants of these obligations or their mastering of the notifications' technicalities. 
                                               
2  By comparison, this is more than the resources allocated to the RTPC programme which financed 189 
days of training for 176 participants in five regions in 2012. WT/COMTD/W/199 
 
- 12 - 
 
  
3.47.  For example, the Secretariat has been orga nizing workshops specifically dedicated to the 
notification obligations under the agreement on Ag riculture since 2009.  In the discussions on the 
timeliness and completeness of notifications held by the Committee on Agriculture in the spring of 2009, many Members had mentioned their need fo r specific training in this domain.  The 
Secretariat performed a survey on this subject, in which Members expressed their ideas on ways 
to improve compliance with their notification obligations and to enhance their institutional capacity to analyse the information notified by other Members and participate in its review by the 
Committee.
3  As a result, the Committee on Agriculture requested on 2 July 2009 that a series of 
notification workshops be organized by the Secretariat. 
3.48.  However, well-trained officials do 
not necessarily guarantee a subsequent 
outpour of notifications, as many 
political or logistical externalities may stand in the way.  Taking this important 
caveat into account, the Secretariat has 
measured the evolution of the rate of outstanding notifications under the 
Agreement on Agriculture.  The measure 
focused on five main notification obligations of Members under the Agreement on Agriculture.
4  Since 2009, 
after four workshops on agriculture 
notifications, the overall rate of 
outstanding agriculture notifications has progressively decreased from 38% down to 32% in March 
2013.  Progress occurred even for notifications on domestic support and export subsidies,5 which 
have traditionally been the most problematic for many Members.  Only 40% of them are still outstanding today, instead of 50% four years ago. 
3.49.  It can reasonably be assumed that the trai ning provided by the Secretariat in this domain 
p l a y e d  a  r o l e  i n  t h i s  e v o l u t i o n .   H o w e v e r ,  t h e decrease of outstanding notifications is very 
progressive for the moment, which means that the support provided by the Secretariat to Members will need to be sustained over a long period of time to produce more meaningful results.  
This should be combined with a continuous focus of the Committee on Agriculture on this particular 
aspect of the implementation of the Agreement. 
3.3.2  Country reviews 
3.50.  As indicated above, the Secretariat initiated two pilot country reviews in 2012 in Kenya and 
Senegal.  The objective of these reviews was to beef-up the country reviews conducted earlier by TCA and which had been mainly based on interviews, in order to ground the assessment of the 
longer-term results produced by the TA provided by the WTO on more objective and factual 
elements. 
3.51.  As will be seen below, these two pilots  have been partially successful.  They gathered 
factual information which can objectively support the measurement of certain results.  However, 
the volume and precision of the data to be collected a posteriori proved a real challenge.  Some of 
the chosen indicators did not produce meaningful  information and had to be dropped; others - 
such as the participation in the meetings of WTO bodies - which involved regional groupings were 
actually impossible to measure with a sufficient level of reliability.  As a result, the harvest could 
not be comprehensive enough in a number of respects.  The Secretariat also lacked resources to investigate more thoroughly on its own, as it does in the context of TPRs. 
                                               
3  See document G/AG/GEN/85 of 15 June 2009. 
4  These notifications related to Market Access (tari ff-rate quotas), Domestic Support (AMS) and Export 
Subsidies.  Figures on outstanding notifications are regularly presented to the Committee on Agriculture in 
document G/AG/2 and its subsequent revisions. 
5  These are the two areas in which all Members have  a notification obligation.  Other notification 
requirements are largely dependent on the specific commitments assumed by each Member under the Agreement on Agriculture. WT/COMTD/W/199 
 
- 13 - 
 
   
3.3.3  Kenya 
3.52.  A review of the TA provided by the 
WTO to Kenya over the 2006-11 period 
showed that 85% of the activities and 90% of 
the participants had dealt with seven main subjects: General training on WTO and its 
agreements, SPS, Market Access for Goods, 
Trade Facilitation, Technical Barriers to Trade (TBT), Agriculture and Trade-related aspects of Intellectual Property Rights (TRIPS).  For 
each of them, a list of performance indicators 
was then agreed with the Kenyan Authorities to try to measure the extent to which any 
material change could be noticed in these 
domains during the period under review.  In 2012, the TPR of Kenya was being performed 
as part of the review of the East African 
Community.
6  The Kenyan Authorities 
undertook to collect the required evidence and make them available to the Secretariat once 
their TPR was over.  Unfortunately, this proved only partially possible, as the authorities could only 
provide very general data on four of the chosen  subjects (Agriculture, Market Access for Goods, 
SPS and TBT).  However, the information was complemented with the information available to the 
Secretariat in the various WTO documents and databases. 
3.53.  Kenya has a National Committee on WTO, with subject-specific sub-committees, which meet 
periodically to review issues pertaining to the multilateral trading system (MTS).  During the period 
under review, Kenya's negotiating positions were ge nerally aligned with those taken by the African 
and African, Caribbean and Pacific (ACP) groups, which presented and defended these positions.  
These groups have been particularly active in the Non-Agricultural Market Access (NAMA) negotiations, in which they presented each year a number of proposals (on issues related to Non-
Tariff Barrier (NTBs), non-reciprocal preferences, international standards, etc.) or in the 
Negotiating Group on Trade Facilitation (NGTF).  This means that the contribution of Kenya to the negotiations or the operation of the WTO bodies could not be observed directly in most cases.  In 
2011, Kenya was the coordinator of the African Group and was provided an intern under the 
Regional Coordinator Internship Programme to support its Geneva mission.  It also had a Mission intern in 2006 and 2010. 
3.54.  According to the Authorities, the annual provision of national data to the Integrated 
Database (IDB) and Consolidated Tariff Schedules Database (CTS) was very timely, as was the verification of the transposition of the national  schedule of concessions.  The HS96 schedule of 
concessions of Kenya was circulated to the Committee on Market Access in June 2010. 
3.55.  Kenya seems fairly well organised to deal with SPS issues.  Meetings of the SPS Committee 
are attended by representatives of the inquiry points Kenya Plant Health Inspectorate Service (KEPHIS) and Kenya Department of Veterinary Se rvices (DVS).  Notifications were regularly 
presented when necessary (e.g. when emergency measures were taken in 2005-06 due to the 
outbreaks of avian flu) or required (e.g. requir ements for importation of plants in July 2010 - 
G/SPS/N/KEN/30).  Stakeholders seem to be regularly informed and involved through the National 
SPS Coordinating Committee.  Kenya benefited from specific TA from the European Union (EU) in 
this area to beef-up the facilities of its Plant Health Inspectorate Service (G/SPS/GEN/1020) and 
implemented an electronic pest surveillance mechanism for horticultural products (G/SPS/GEN/1019).  A National Biotechnology Awareness Strategy was developed in 2008 leading 
to the adoption of a Biosafety Act in 2009 and th e establishment of a National Biosafety Authority 
in 2010. 
                                               
6  See document WT/TPR/S/271 Topic  Activities  Participants
Agriculture   8  15 
Dispute Settlement   3  3 
Environment   5  6 
General  57  119 
Gov. Procurement   2  4 
MA for Goods  6  49 
Rules  7  9 
Services  4  10 
SPS  18  53 
TBT  12  17 
Trade Facilitation   11  21 
TRIPS  10  15 
Total  143  321 WT/COMTD/W/199 
 
- 14 - 
 
  3.56.  The management of TBT issues by the Na tional TBT Consultative Committee seemed equally 
effective.  During the period under review, Keny a notified more than 200 technical regulations 
(e.g. on food products, chemicals, textiles, water,  electric equipment, etc.) to the Committee on 
TBT, in which Kenya is regularly represented thanks to the sponsorship of Swedish International 
Development Cooperation Agency (SIDA) (mentorship programme).  Standardization issues are 
managed by the Kenya Bureau of Standards (KEBS), which is the national enquiry point on TBT.  
Kenya raised one specific trade concern in 2010 concerning the effects of Canada's Tobacco Act on 
Kenya (G/TBT/W/330). 
3.57.  In the area of Trade Facilitation, Kenya in troduced the Simba electronic system in 2005 to 
process customs declarations.  This helped reduce progressively the number of days necessary for overall procedures.  However, 24 days were still needed to that effect according to the World 
Bank. 
3.58.  In 2008, Kenya associated itself with a group of developed and developing countries led by 
the EU to present a proposal on the extension of the additional protection for geographical 
indications to products other than wines and spirits (TN/C/W/48).  It also passed the Anti-
Counterfeit Act and took actions to eradicate counterfeit and piracy, in particular for software with the support of Microsoft.  The Anti-counterfeit Agency was established in June 2010. 
3.59.  These results are encouraging.  At this stage, it can reasonably be assumed that the TA 
provided by the WTO to Kenya contributed to th em, if only by drawing the attention of the 
authorities to required improvements and possible solutions. 
3.3.4  Senegal 
3.60.  A review of the TA provided by the 
WTO to Senegal over the 2007-11 period showed that 72% of the activities and of the 
participants had dealt with four main 
subjects: General training on WTO and its agreements, SPS, Trade Facilitation and 
T R I P S .   F o r  e a c h  o f  t h e m ,  a  l i s t  o f  
performance indicators was then agreed with the Senegalese authorities to try to measure the extent to which any material 
change could be noticed in these domains 
during the period under review.  The authorities undertook to collect the required 
evidence and make them available to the 
Secretariat.  The information provided by the authorities covered all the agreed areas, 
sometimes in a rather general way.  It was complemented by the information available to the 
Secretariat in the various WTO documents and databases. 
3.61.  Senegal has a National Committee on Inte rnational Trade Negotiations with six subject-
specific sub-committees since 2001.  According to the authorities, the committee functioned 
regularly throughout the period.  The authorities also confirmed that a large number of the officials 
trained by the WTO during the period (48 in 2011) were still working for the government on WTO-related matters, such as the Doha Developmen t Agenda (DDA) negotiations.  One former 
participant was posted at the Senegalese mission in Geneva throughout the period under review.  
Senegal had submitted itself to a TPR in 2009.
7  The authorities undertook to follow the 
recommendations generated by the review.  They  stated that the two WTO Reference Centres 
established in Senegal remained operational thro ughout the period, but could certainly attract 
more visitors. 
                                               
7  See Document WT/TPR/S/223/Rev.1 Topic Activities Participants  
Agriculture 6 10 
Dispute Settlement 3 3 
Environment 2 11 
General 44 73 
Gov. Procurement 1 2 
MA for Goods 4 8 
Rules 4 4 
Services 5 6 
SPS 10 16 
TBT 5 8 
Trade Facilitation 13 17 
TRIPS 9 23 
Total 106 181 WT/COMTD/W/199 
 
- 15 - 
 
  3.62.  SPS issues were dealt with by the Sub-Committee on Trade in Agricultural Products, which 
met regularly during the period.  Three notifications were presented by Senegal in 2010.8  It had 
previously presented only four other notifica tions under the SPS Agreement back in 1996 and 
2000. 
3.63.  In the area of Trade Facilitation, th e National Committee on International Trade 
Negotiations established a seventh Sub-Committee to deal with this subject in 2009.  According to 
the authorities, the sub-committee efficiently prepar ed Senegal's positions prior to the meetings of 
the NGTF in Geneva, in which it participated regularly. Around four officials trained by WTO kept 
working on these issues during the period.  In  2011, Senegal summarized its positions on the 8th 
revision of the negotiating text in a document sent to the negotiating group, and it associated itself 
with Egypt and Saudi Arabia to present a joint negotiating proposal concerning the time for the 
next round.  In 2009, in its report on " Doing Business ," the World Bank ranked Senegal as the first 
most reformist African country and the 5th worldwide.  It also ranked Senegal as the world most 
reformist country for trading across borders. 
3.64.  The sub-committee on TRIPS met regularly every year during the period to coordinate 
Senegal's positions in this area.  In October 2007, a national brigade to fend off counterfeit and pirated goods started to operate inside the national police.  It launched a number of actions, for 
example against sellers of pirated Compact Disk s (CDs) or DVDs (250 arrests per year between 
2007 and 2009, 8 tons of pirated CDs/DVDs and production equipment destroyed).  A new law on Copyright and related rights was adopted in 2 008.  The 2009 TPR found that the Senegalese 
legislation conformed to the TRIPS agreement.  In January 2011, Senegal adopted the protocol on 
patents and public health amending the TRIPS agr eement.  The criminal code was also reformed 
that year to take into account the 2008 law on copyright and a new National Programme of Development of Intellectual Property (NPDIP) was developed with World Intellectual Property 
Organization's (WIPO) support (still waiting implementation).  In June 2011, Senegal 
communicated to the TRIPS Council its priority needs for technical and financial co-operation in a 21-page document (IP/C/W/555).  Finally Senegal participated in three meetings of the TRIPS 
Council in 2010 and 2011, an improvement on the pr evious years during which it did not attend 
such meetings. 
3.65.  These evolutions are encouraging.  At this stage, it can reasonably be assumed that the TA 
provided by the WTO to Senegal contributed to th ese evolutions.  The presence during 2011 at the 
Geneva mission of Senegal of an intern financed by the WTO certainly played a role in the 
increased presence of the country in meetings of  the WTO bodies, such as the TRIPS Council, as 
did the WTO programme to finance the participation of LDC delegates in the meetings of the NGTF. 
4  LESSONS LEARNED 
4.1.  This chapter regroups the various lessons learned when monitoring and evaluating the TA 
activities implemented in 2012.  The review performe d led to the identification of a number of best 
practices, which should be extended to other ar eas of TRTA.  Both the lessons learned and the 
identified best practice should be taken on bo ard by the Secretariat when preparing future TA 
Plans. 
4.2.  The various lessons learned and best practice identified during the year under review can be 
regrouped around three main themes: improvemen ts required in the way TA is organised and 
managed in the TA Plan 2014-15; adjustments to the TA offered by the Secretariat; and actions needed for the successful implementation of RBM. 
4.1  Management of WTO Technical Assistance 
4.3.  Traditionally, the TA offered by WTO has been managed collectively by Members through the 
TA Plan.  Since a large part of this TA is financed through voluntary contributions, an increase in 
earmarking of these voluntary contribution s could have implications on the DDAGTF. 
                                               
8  See documents G/SPS/N/SEN/5 to 7. WT/COMTD/W/199 
 
- 16 - 
 
  Recommendation 1:  Members may wish to analyse the possible implications of the current 
increase in the earmarking of the voluntary contributions received from donors to help finance the TA provided by the WTO.   
4.2  WTO Technical Assistance Offer 
4.4.  A new approach to face-to-face activities in general, and regional ones in particular, is 
required in the TA Plan 2014-15.  A particular atte ntion will need to be paid to the actual value 
added by each type of activity, compared to more efficient alternatives such as E-Learning or 
national activities. 
Recommendation 2:  In the TA Plan for 2014-15, the Secretariat should focus face-to-face 
activities, in particular at regional level, on the areas and content on which they can provide a real added-value, given their relatively high cost 
compared to E-Learning or national activities. 
Recommendation 3:  The Secretariat should dedicate more resources in the future to E-
Learning to widen its coverage of the various WTO subjects and improve 
the presentation, pedagogy and interactivity of the courses when 
possible, since the return on such investment would be proportionately much higher. 
4.5.  To maximize the use of WTO staff time, a particular attention should also be paid to the 
existence and identification of a commonality of needs in a region or at global level.  In a context where resources are scarce, it seems pertinent to reassess the composition of the WTO "product mix" to eliminate potential duplications between national and regional activities on the same 
subjects and maximize the results produced with the resources available. 
Recommendation 4:  In the TA Plan for 2014-15, the Secretariat should examine carefully the 
value added by regional activities, given their relatively high cost.  The TA 
Plan should provide the Secretariat with  the flexibility to adopt a regional 
or national approach, responding fully to the identified needs of beneficiaries and taking into account the relative cost of the immediate 
results that can be reasonably expected from the activity.  This should 
apply on a case-by-case basis. 
4.6.  Under the Academic Support Programme, the Mid-Term Review of the WCP conducted in 
2012 showed that a number of adjustments were needed in this domain. 
Recommendation 5:  In the TA Plan for 2014-15, the Secretariat should consider ways of 
improving the efficiency of the Academic Support Programme.  Its resources should be better aligned with its potential to generate results. 
4.7.  To better respond to the TA needs and prio rities of beneficiaries, the Secretariat should 
continue its work on the structure of its TA offe r, in particular on the most appropriate balance 
between the three Progressive Learning Strategies (PLS) levels of training and between the various 
subjects.  However, there is still a need for a clearer definition subject by subject of the 
curriculum, concepts and knowledge which has to be mastered at each of the three PLS levels.  The progressivity achieved thus far in WTO training resulted mainly from the generalization of prerequisites to participate in intermediate and advanced activities.  This increased the level of 
knowledge and competence of the average partic ipants about the WTO and its agreements and 
allowed trainers to go deeper into the substance during the TA activities when covering the same 
subjects.  This way, a certain progressivity was introduced in the TA offered by WTO at regional or 
global level. 
Recommendation 6:  The Secretariat should define the substantive content of each of the three 
training levels instituted by WTO subject by subject.  It should then 
adjust the content of the training offered by WTO, on-line and face-to-
face, accordingly.  It should inform future participants about the substantive knowledge required in any activity to which they are invited, so they can undertake to bridge in advance any gap they would detect. WT/COMTD/W/199 
 
- 17 - 
 
  Recommendation 7:  In the TA Plan for 2014-15, the Secretariat should consider whether the 
current balance between levels and training paths in the TA offer of the WTO, responds to the needs of Memb ers.  Corrective actions should be 
proposed if necessary. 
4.3  Implementation of RBM 
4.8.  The active involvement of all TA stakeholde rs (beneficiaries, donors and the Secretariat) is 
essential if RBM is to be successful.  RBM for the monitoring and evaluation of TA activities is being 
introduced incrementally and systematically by the Secretariat. 
4.9.  Whilst progress was made on this front in 2012, more is needed to measure as objectively 
the immediate or longer-term results achieved by the TA provided by WTO to beneficiaries.  The 
use of relevant and objective performance indicators, such as tests of the participants' acquired 
knowledge or skills at the end of face-to-face activities, is critical in this perspective.  At a time 
when the Secretariat is expected to report more comprehensively on the results of the TA activities it organises, past recommendations on a credible measurement of performance have to be 
repeated. 
Recommendation 8:  The immediate results achieved by the TA activities organized or 
delivered by the WTO should be measured as objectively as possible by 
the Secretariat.  Credible performance indicators should be identified in 
advance by the trainers to that effect, such as exams, quizzes, exercises results, etc.  BTORS should report systematically the measure shown by the indicators chosen.  The opinion expressed by the organizers in the 
BTOR should be grounded on these measures. 
Recommendation 9:  For 2014-15 TA activities, the Secretariat should recommend the 
introduction of new means of testing participants' knowledge at the end 
of face-to-face TA activities organized by WTO.  Such means could take 
into account the engagement by each participant in the activity and be based on objective evaluation methods as far as possible (e.g. quiz, test, 
exercise, exam, etc.).  They should focus on the essential content of the 
activity according to the benchmarks to be developed for each subject at each of the three levels of progressive learning. 
4.10.  As BTORs will remain the key instrument  for the evaluation of the immediate results 
(outputs) of the TA activities under the new RB M methodology, the success of the new approach 
rests on the capacity of the Secretariat to improv e significantly on the availability and quality of 
BTORs.  It is worth repeating earlier recommendations in this regard. 
Recommendation 10:  A BTOR from the organizers should become systematically available for 
each and every TA activity organized or delivered by WTO staff members.  The BTOR should include the information required by the WTO reporting 
guidelines, such as immediate objectives, performance indicators, 
assessments of the immediate results, etc. 
4.11.  The measure of the longer-term results that can be associated with the TA provided by the 
WTO would require developing new synergies with the TPRs conducted by the Secretariat.  The 
two pilot reviews conducted in 2012 in Kenya an d Senegal demonstrated that certain results, 
evolutions and changes did occur during the peri od under review in the areas in which the TA 
provided by the WTO had concentrated.  However, one possible lesson learned in the process was 
that such country reviews duplicate in several resp ects the work done during a TPR.  There would 
be obvious merit in combining the two exercises an d performing them together.  This might make 
available a better, more precise and more comprehensive set of data and could leverage the 
resources of the Secretariat and of the country und er review.  This would also allow going beyond 
the mere inventory of recent Aid for Trade support and needs currently included in the TPRs of LDCs and some other developing countries. WT/COMTD/W/199 
 
- 18 - 
 
  Recommendation 11:  Members should consider including future country reviews designed to 
measure the longer-term results of the TA provided by the WTO as part of the TPR of these countries. 
 
 
__________