WT/COMTD/W/255  
 
 
12 May  2020 
(20-3558) Page: 1/30 
Institute for Training and Technical Cooperation    
 
 
 
TECHNICAL ASSISTANCE  MONITORING AND EVAL UATION PLAN  
 
1   BACKGROUND  ................................ ................................ ................................ ...............  2 
2   TECHNICAL ASSIST ANCE  ................................ ................................ ..............................  2 
2.1   TA offer and Target Audience  ................................ ................................ .........................  3 
2.2   Partnerships  ................................ ................................ ................................ .................  3 
2.3   Budget  ................................ ................................ ................................ ........................  4 
3   EVALUATION  ................................ ................................ ................................ ................  4 
3.1   Key Result 1  ................................ ................................ ................................ .................  4 
3.1.1    Onsite Evaluations  ................................ ................................ .............................  5 
3.1.2    Country Assessments  ................................ ................................ ........................  5 
3.1.3    Ex-post Evaluations  ................................ ................................ ...........................  6 
3.1.4    TA Annual Report  ................................ ................................ ..............................  6 
3.1.5    External evaluations  ................................ ................................ ..........................  6 
3.2   Key Result 2  ................................ ................................ ................................ .................  6 
3.3   Key Result 3  ................................ ................................ ................................ .................  7 
3.4   Key Result 4  ................................ ................................ ................................ .................  7 
4   THE LOGICAL FRAM EWORK  ................................ ................................ ..........................  8 
4.1   Indicators  ................................ ................................ ................................ ................... 14 
5   DATA FLOW  ................................ ................................ ................................ ................  19 
6   DATA ANALYSIS AN D REPORTING FLOW  ................................ ................................ .... 21 
ANNEX I: ONSITE EVAL UATION OF TA ACTIVIT IES ................................ .........................  22 
ANNEX II: COUNTRY -BASED TA EFFECTS AND I MPACT ASSESSMENT  .............................  25 
ANNEX III: EX -POST E VALUATIONS  ................................ ................................ ................  28 
 
  WT/COMTD/W/255  
 
- 2 - 
 
  1  BACKGROUND  
1.1.  Trade -related Technical Assistance (TRTA) is a core function of the WTO.   Its main purpose is 
to enhance the human and institutional capacities of beneficiaries to take full advantage of the rules -
based Multilateral Trading System (MTS), meet their obligations and enforce their rights as Members, 
and deal with emerging trade -related challenges.  
1.2.  WTO Technical Assistance (TA) activities are carried out in accordance with the Biennial 
Technical Assistance and Training Plan (TA Plan), prepared by the Secretariat in close consultation 
with beneficiaries and partner institutions.  
1.3.  The TA activities are coordinated by the Secretariat  which operates through a Result -Based 
Management (RBM) approach, focusing on achieving measurable results through improved planning, 
decision -making, transparency and accountability.  
1.4.  The design, management, coordination  and delivery of the TA activities are  a shared 
responsibility across the Secretariat .  They frequently involve partners and outside experts . 
1.5.  The TA Monitoring & Evaluation Unit (TAME) is responsible for coordinating the Secretariat  
monitoring,  evaluation and quality assurance functions, and for producing each year an annual report 
on the implementation of TA activities  and their results . 
1.6.  This M&E Plan has been developed taking into account recommendations 11, 18, 22, 23 and 
26 of the TA Externa l Evaluation performed in 2016.  
1.7.  As a dynamic tool, the M&E Plan should be referred to and updated on a regular basis to reflect 
Member’s priorities and evolving needs as well as global trade developments over time.  
1.8.  This M&E Plan provides an overall structu re for the monitoring and evaluation of TA activities 
managed by TAME , using R BM and Logical Framework approaches . 
1.9.  The purpose of the M&E Plan is to help TA funders, beneficiaries and trainers to develop a joint 
understanding of the M&E Framework for the Secretariat TA  operations.  In particular, the Plan 
intends to:  
a. Describe TAME's  approach and system to assess progress and impact of the overall TA  
activities ; 
b. Explain the overall purpose and rationale  of TAME’s M&E Framework ; 
c. State roles and responsib ilities as well as the information flow s; 
d. Promote a better integration of M&E a t all the stages of the TA offer;  
e. Provide accountability to TA donors , beneficiaries and other key stakeholders;  
f. Preserve the WTO institutional memory.  
2  TECHNICAL ASSISTANCE  
2.1.  As stated in the background , the TA Plan represents the backbone for the WTO Secretariat's 
approach to the delivery of Trade -related Technical Assistance.   The TA Plan is principally a policy 
framework that identifies priorities, anticipated res ults, mechanisms for implementation and sources 
of funding.  
2.2.  Ever since the introduction of RBM for TA, the TA Plan of the WTO has been structured around 
four main Key Results (KRs) that TA should contribute to achieve.  Today, these KRs are defined as 
follows: 
• KR 1. Government officials are implementing WTO Agreements and fully realising Members' 
rights and obligations;  WT/COMTD/W/255  
 
- 3 - 
 
  • KR 2. Acceding governments are participating in accession negotiations;  
• KR 3. Academic institutions and other stakeholders are analysing WTO issues and reaching 
out to policy makers; and  
• KR 4. Non -governmental stakeholders are aware of and knowledgeable about WTO work and 
issues.  
2.3.  The TA Plan for 2020 -21 explains how the above four Key Results will be pursued over the 
biennium, tak ing into consideration Members’ needs and the outcome of the most recent Ministerial  
Conferences.1  In addition, the TA Plan identifies the performance indicators and targets that will be 
used to measure the attainment of the Key Results.  
2.1  TA offer and Targ et Audience  
2.4.  Training activities are delivered either  face-to-face or online .  Face-to-face activities take place 
at national, regional and global levels covering a variety of trade topics.  Distance l earning courses , 
where participants can learn at their o wn pace, are available on the WTO eLearning platform on a 
24/7 basis.  
2.5.  In addition to the above training offer, the Secretariat proposes  other learning by doing 
programmes (internships, Young Professionals Programme and other targeted schemes).  
2.6.  A key object ive is to maintain a geographical balance in the delivery of activities, but priority 
is given to least -developed countries (LCDs)  and Africa .  Overall, the TA target audience is divided 
into two main categories:  
Government officials  
2.7.  TA is primarily geared  towards government officials from developing countries, LDCs, 
economies in transition and countries in the process of joining the WTO.   Government officials include 
those working on trade -related issues from different arms of government.  
Non-Government officials  
2.8.  The audience can also include representatives from parliament, civil society, journalists, 
academic institutions , the private sector  and the general public.  
2.2  Partnerships2 
2.9.  The WTO collaborates with many  inter-governmental and non -governmental organizations to 
deliver TA activities.  These partnerships allow to take advantage of synergies and leverage financial 
and human resources.  
2.10.   The nature of the partnership varies widely, depending on the institution,  the content of any 
signed undertaking, where they exist, and on the agreed objectives in the partnership arrangements.   
For example, the WTO participates in events organized by, or in cooperation with, other agencies 
and in trade -related conferences, symp osia and high -level meetings.  The WTO makes use of the 
services of external consultants who contribute to  the delivery of regional and national activities and 
who undertake research.  Also, university professors and researchers often participate as lectur ers. 
2.11.   It is expected that partnerships will : 
• Enrich the quality of training and TA activities by using existing capacity, expertise and 
facilities, where available, at the regional level and/or in Geneva.  Such cooperation will 
facilitate the delivery of TA and the dissemination of the WTO training programmes;  
 
1 See WT/COMTD/W/248/Rev.1  
2 Please visit WTO’s website for a comprehensive list of partners: 
https://www.wto.org/english/tratop_e/devel_e/teccop_e/partnerships_e.htm . WT/COMTD/W/255  
 
- 4 - 
 
  • Achieve greater coherence among providers of WTO -related training and TA; 
• Gradually transfer ownership and build partnerships on TA; and  
• Optimize the use of resources available . 
2.3  Budget  
2.12.   The implementation costs for the TA 
Plan are around CHF 1 9 million  per year .3  
The WTO training activities are funded from 
two main sources: the WTO regular budget 
(CHF 4.5 Mio) provided by the WTO 
membership, and voluntary contributions 
from certain WTO Membe rs into trust funds  
(CHF14.5Mio).4 
2.13.   Contributions from WTO Members 
have declined in recent years owing to the 
lasting impact of the 2008 global financial 
crisis.   Through a c areful management of its 
TA resources  the Secretariat was able to 
maintain a high l evel of training activities 
despite its  decreasing resources.  
2.14.   Today, o ne of the key challenges for the WTO is to stabilise the funding of its TA  activities.  
3  EVALUATION  
3.1.  The evaluation of training activities can be addressed as a strategic function contributing to  
increase quality, accountability and transparency to wards Members, donors, partners and other 
stakeholders .  The monitoring and evaluation (M&E) of WTO TA is important in assessing the 
efficiency, effectiveness , impact  and continued relevan ce of courses, workshops or seminars, and 
any potential for improvement .5 
3.2.  The e valuation approach involves the use of standard ised tools whose implementation is then 
tailored to the specificities of each activity, programme or beneficiary .6 
3.3.  As mentioned  above, the TA Plan of the WTO is structured around four main Key Results (KRs).   
The relative importance of each of the KRs in the TA Plan is  quite variable.  KR  1 has traditionally 
been the most significant, representing approximately 87% of the activities,  participants or 
expenses.  The share of KRs 2 and 3 was approximately 5% each and KR 4, 3%.  To the extent 
possible, the M&E Plan will take this distribution into account.  This will be pursued over several 
years.  This means that the large majority of th e evaluation work will focus on TA activities primarily 
designed to achieve KR 1, reason why the evaluation methodology for KR 1 is significantly more 
sophisticated than for the other three KRs.  
3.1  Key Result 1  
3.4.  PEFs.   The Participant Evaluation Form (PEF) is the main instrument to measure participants' 
reaction to a training activity.  PEFs must be completed – online or on paper – by participants at the 
end of each activity to give their feedback on a set of predefined issues (closed questions) and share 
their comments and suggestions for improvement (free text).  TAME process the PEFs and provide s 
 
3 Please see “Financing of TRTA” : 
https://www.wto.org/english/tratop_e/devel_e/teccop_e/financing_trta_e.htm  and the Annual Report . 
4 Please consult the list of contributions here: 
https://www.wto.org/english/tratop_e/devel_e/teccop_e/financing_trta_e.htm#contributions .  
5 TAME uses the DAC Criteria for Evaluating Development Assistance – OECD , to structure each data 
record form for comparative analysis and for evaluation report conclusions .  
6 Please see Annexes I, II and III in this document.  
 - 5 10 15 20 25Millions CHFTechnical Assistance Resources
Earmarked Trust
Funds
Non-earmarked
Trust Funds
Regular BudgetWT/COMTD/W/255  
 
- 5 - 
 
  their statistics to the organisers of the activit ies.  These statistics and participants' comments should 
be analysed by the trainers in the BTORs, in its section on the assessment by participants.  
3.5.  BTORs.   The Back -to-Office -Report (BTOR) is a standardised instrument through which the 
organisers and trainer s assess various parameters of the training activity they have delivered, such 
as the extent to which its learning objectives have been reached by participants, its immediate 
objectives, the contribution of partners, etc.  Therefore, BTORs are an exercise of self -evaluation by 
the trainers, whose credibility depends entirely from the intellectual honesty of the organisers.  TAME 
has three objectives concerning BTORs:  
a. Maintain a high level of reporting;  
b. Accelerate the production of BTORs; and  
c. Improve the quality of the data gathered through BTORs.  
3.6.  The data contained in BTORs accumulate in the TAMS database and constitute the basis on 
which many indicators identified in the TA Plan logframe will be measured at the end of each year.  
3.7.  The graph b elow shows  the 13 successive steps of the workflow through which the Secretariat 
implements TA activities , from the initial request to the approval of the BTOR .  In the example, the 
steps in green  have been completed by the responsible person ; the process is currently a t the step 
in yellow; and future steps still to be completed appear in grey.  This example corresponds to an 
approved activity not yet implemented, as its BTOR is not yet in preparation.  
 
3.1.1   Onsite Evaluations7 
3.8.  Onsite evaluations are a powerful tool to obser ve training dynamics, through participants' body 
language and interaction with the trainers and peers; the change in knowledge and skills of the 
participants; whether the programme meets the learning objectives; whether participants' prior 
knowledge is ade quate; the level of interaction between participants and trainers; the various 
training methods used; whether the diversity of the language offer meets participants' needs, etc.  
They allow the evaluator to engage in informal discussions and interviews thr oughout the training.  
3.9.  They also give a higher standing to the final evaluation session in which specifically focused 
questions can be addressed collectively by the participants.  This first -hand feedback enables the 
evaluator to measure participants' react ion to the training.  
3.10.   Onsite evaluations also contribute to lending a higher level of credibility to the self -evaluations 
performed by the trainers in their BTORs.  A recent comparison showed that the trainers tended to 
inflate the level of success of their  activities in their BTORs  by 20% compared to TAME's evaluation.  
Whilst not insignificant, this margin remains limited  and confirms that the self -evaluations in the 
BTORs can be relied upon to assess  whether TA activities are reaching their learning objectives . 
3.11.   According to its well -established practice, TAME aim s to carry out around six onsite 
evaluations per year in ord er to diversify the subjects and regions covered, so that all the WTO 
subjects and regions are reviewed over a two to three -year cycle.  
3.1.2   Country  Assessments8 
3.12.   Country  assessments were initially launched in 2009 on an opportunistic basis.  Taking 
advantage of  TAME's presence in a country for an onsite evaluation, meetings are organised with 
the authorities of the host country to collect evidence on the changes of behaviour observed among 
 
7 See Annex I for a detailed methodology.  
8 See Annex II for a detailed methodology.  
WT/COMTD/W/255  
 
- 6 - 
 
  their officials who attended WTO activities over a given period of time, long enough to measure 
changes and short enough to keep track of the participants (typically five years).  As demonstrated 
by the 2016 External Evaluation,9 the anecdotal information gathered through these exchanges is 
very useful, especially if TAME is ab le to gather a sufficient volume of it over time.  
3.13.   To evaluate the concrete results that the TA provided by the WTO may have contributed to  
support , country assessments are based on informal interviews with key representatives from the 
national authorities.   The anecdotal evidence gathered on these occasions ha s the potential to build 
over time a critical mass of facts and opinions.  
3.14.   According to its well -established practice, TAME aims to perform a country assessment on the 
occasion of each of the onsite eva luations undertaken outside Geneva during the year.  
3.1.3   Ex-post Evaluations10 
3.15.   Ex-post evaluations aim to identify how far former trainees have changed their behaviour 
following their participation in a specific programme , and the influence this had on the capac ity and 
competence of their institution .  Ex-post evaluations  cover a certain period of time long enough to 
measure changes and short enough to keep track of the participants (typically five years).  These 
evaluations follow a rigorous approach to collect former beneficiaries' and other identified 
stakeholders' feedback.  
3.16.   Ex-post evaluations have traditionally allowed TAME to measure some concrete results 
produced by a specific training programme in the various countries which benefited from it over a 
period  of time.  
3.17.   TAME aims to perform one ex -post evaluation per year, targeting in priority the most 
significant programmes in terms of participants or budget.  
3.1.4   TA Annual Report  
3.18.   At the level of the overall WTO TA portfolio, TAME assesses the results of TA activit ies in the 
Annual Report which presents for each of the indicators identified in logframe of the TA Plan, the 
evidence that could be gathered.  The TA annual report is the document in which the WTO reports 
back to Members on how the TA logframe was impleme nted and the results this led to, based on  the 
data sources identified in the TA Plan.  
3.1.5   External evaluations  
3.19.   External evaluations provide the most rigorous approach to collecting evidence on the impact 
of WTO TA for beneficiaries.  However, they are too complex, time consuming and expensive to be 
conducted annually .  So far, external evaluations have been conduct ed at the request of donors  and 
approved by Members .  Their periodicity has been every ten years ; the last one t ook place in 2016.  
3.20.   In the absence of a clear signal from the Donors, it has been assumed that no such external 
evaluation would take place in th e foreseeable future.  
3.2  Key Result 2  
3.21.   Most of the TA activities designed to achieve KR 2 are similar to those designed to achieve 
KR 1.  The only difference comes from their public, since under KR 2 the participants come from 
Observers, whereas they come from Members under KR 1.  This means that the evaluation 
instruments and methodologies used under KR 1 equally apply under KR 2.  
3.22.   The two main differences compared to KR 1 will be the following:  
3.23.   As most of the TA activities held in Observers have a national  scope, TAME is unlikely to travel 
regularly to observe such activities, especially given the limited volume of TA dedicated to KR  2.  
 
9 See WT/COMTD/89.  
10 See Annex III for a detailed methodology.  WT/COMTD/W/255  
 
- 7 - 
 
  This means that the reaction of trainees to a TA activity and the achievement of their learning 
objectives will be primar ily measured through PEFs and BTORs.  
3.24.   Since TAME's travel to an Observer country will remain rare, so will the performance of a 
country assessment in an Observer.  This means that any change in Observers' participants 
behaviour and the results this led to w ill have to be primarily measured through ex -post evaluations 
and the TA annual report.  
3.3  Key Result 3  
3.25.   The nature of the TA provided under KR 3 is significantly different from the rest of the WTO 
TA portfolio.  As the public is comprised exclusively of stude nts and academics, like any investment 
in public education, the TA provided by the WTO is not supposed to produce any direct and 
measurable immediate result for Members and Observers, even when it takes the form of training 
activities.  
3.26.   KR 3 is also the onl y domain in which the WTO is sponsoring through the allocation of grants 
other institutions' activities, programmes or projects instead of implementing them itself.  This 
means that the data on the results of such activities, programmes or projects will no t be directly 
available to the WTO, as they will come through the filter of its academic partners and will be  difficult 
to verify.  Only the quality of the management of the grant -giving project by the WTO staff can be 
assessed first -hand by TAME.  
3.27.   Finally,  the nature of the activities, programmes or projects under KR 3, when they do not 
pertain to training and education (e.g. academic research or curriculum development), is very 
different from other more traditional WTO TA, reason why their WTO coordinators  have generally 
claimed that TAME was ill -equipped to assess their results.  
3.28.   Therefore, TAME has so far  avoided to assess the results of the activities under KR 3.  It 
focused occasionally on the management of the programme.  A comprehensive external evalua tion 
of the WTO Chair P rogramme (which constitutes the bulk of the WTO TA under KR 3) was conducted 
in 2019 by an outside consultant to cover the TA provided over the eight years elapsed since its 
launch.  
3.29.   The evaluation of the TA results under KR 3 will fo llow the same approach as in the past.  
3.4  Key Result 4  
3.30.   The nature of the TA provided under KR 4 is also different: it addresses a different kind of 
public too (journalists, MPs, NGOs, private sector representatives, etc.) and its objectives are only 
to reach out to different stakeholders and not to train them on anything, according to IERD, as if 
organising TA activities with these publics was an objective in itself.  IERD claims for example that 
it would be politically unacceptable for such participants to fi ll in a PEF.  Ever since 2012, IERD has 
resisted the implementation of RBM for these activities, claiming that it was impossible to measure 
any result in this domain and dragging its feet to cooperate for the definition of relevant indicators 
and the colle ction of related data.  As a result, KR 4 has traditionally been the part of the TA Plan 
with the highest rate of data not available when evaluating performance in the TA annual report.  
3.31.   Against this challenging background, TAME is relying on a combination of BTORs and onsite 
evaluations, and to a lesser extent on the TA annual report to assess the results achieved under 
KR 4.  This should continue for the foreseeable future.   
  WT/COMTD/W/255  
 
- 8 - 4  THE LOGICAL FRAMEWOR K 
4.1.  The Logframe  is the main TA planning tool within the RBM M&E Framework.  The Logframe is a dynamic tool and will be used to record actual 
progress against the plans and can be used to re -assess and revise the activities as they develop and as circumstances change duri ng implementation.  
•  Results  Indicators  Baseline  Target  Evidence  Assumption  
Impact11 Developing and LDC 
Members are benefiting 
from their active 
participation in the WTO  • Share of developing Members' 
total trade (imports and exports)  • 2015-17 average: 
40.4%  • > 40%  • WTO statistics  • 1. Global economic and 
political stability or 
improvement  
• 2. The number of LDCs 
remains constant over 
the biennium  • Share of LDC Members' total 
trade (imports and exports)  • 2015-17 average: 
1.07%  • > 1.00%  • WTO statistics  
Key 
Result  1: Beneficiary 
government officials 
are implementing 
WTO Agreements and 
fully realizing 
Members' rights and 
obligations  • Number of documents submitted 
by beneficiary developing and 
LDC Members to WTO Bodies  • Annual average 2016 -
18: 
• Developing: 537  
• LDCs: 81  • Increase  • WTO Documents 
Online database  • 1. Political support for 
WTO-related issues  
• 2. Trained staff retained 
in relevant positions  
3. Human and financial 
resources available  • Number of outstanding 
notifications by TA beneficiary 
Member  • 28 per beneficiary 
Member on average in 
2016-18 • Decrease  • Notifications 
database (CRN)  
Output  1.1: Government 
officials have enhanced 
knowledge about the 
WTO Agreements, 
formulation of trade 
policies and conduct of 
trade negotiations  
 
 
 
  • Number of participants 
completing each PLS level  • Annual average 2016 -
18: 
• Level 1 - 4,271  
• Level 2 - 12,408  
• Level 3 - 1,802  • Stability for 
levels  1 and 2  
• Increase in level 
3 • TAMS database  • 1. WTO negotiations 
remain active  
• 2. Trained staff retained 
in relevant positions  
• 3. Members present 
suitable candidates  
• 4. WTO administrative 
measures do not reduce 
the access to TA  
• 5. Availability of training 
facilities in Geneva   • Number of national activities 
requested  • 2016-18 average: 147  • Increase  • TAMS database  
 
11 In accordance with the RBM approach, the impact is the highest -level result to which WTO contributes jointly with other stakeholders. It is not an exc lusive 
attribution to WTO TA, but an indication of the contribution the WTO TA is making to a common impact.   
  WT/COMTD/W/255  
 
- 9 - •  Results  Indicators  Baseline  Target  Evidence  Assumption  
  • Number of beneficiary Members 
and Observers participating in 
TA courses  • 2016-18 average: 174  • Stability  • TAMS database  • 1. WTO negotiations 
remain active  
• 2. No LDC graduation  
• 3. WTO administrative 
measures do not reduce 
the access to TA    • Number of beneficiary LDCs 
participating in TA courses  • 2016-18 average: 46  • Stability  • TAMS database  
  • Interactive training methods' 
share of time in each activity 
(exercises, case studies, 
simulations, Q&A, participation 
in Committee meetings, etc.)  • 2016-18 average:  
• - 43% lectures  
• - 57% interactive 
methods  • Increase share 
of interactive 
training 
methods to an 
average of 
60%.  • TAMS database  • Enough  WTO staff 
undergo training on 
pedagogical skills  
  • Success rate in 8 -week RTPCs 
and ATPCs  • 2016-18 average 
(RTPC): 95%  • Stability  • BTORs  •  
  • Distinction rate in 8 -week RTPCs 
and ATPCs  • 2016-18 average 
(RTPC): 32%  • Stability  • BTORs  •  
  • Percentage of participants giving 
a top mark to the overall content 
and results in RTPCs, ATPCs and 
advanced thematic courses  • 2016-18 average:  
• - RTPCs: 61%  
- ATPCs: 83%  
- advanced thematic 
courses: 87%  •  
• Increase to 80%  
• Stability  
• Stability  • BTORs  •  
  • Number of participants trained in 
advanced trade negotiating skills 
courses  • 2016-18 average: 290  • Stability  • TAMS database  • 1. Enough demand for 
training activities  
• 2. Availability of 
resources  
• 3. Availability of training 
facilities in Geneva    • Number of advanced thematic 
courses using pre -diagnostics, 
coaching, leadership, action 
plans or follow -ups • 2018: 4  • Stability  • BTORs  
• TAMS database  
  • Number of interns trained in 
various Internship programmes  • 2016-18 average: 
FIMIP: 16; NTP: 16; 
RCI: 7 • Stability  • BTORs  
• TAMS database  • Funding remains 
constant   
  WT/COMTD/W/255  
 
- 10 - •  Results  Indicators  Baseline  Target  Evidence  Assumption  
  • Final evaluation of each intern 
by their supervisor  • 100% of at least fully 
satisfactory in 2016 -
18 • Stability  • Evaluation by the 
supervisors  •  
  • New, revised or translated online 
courses released during the year  • 2016-18 average: 10  • Stability  • BTOR  • 1. Appropriate e -Learning 
resources  
• 2. Absence of external 
disturbance in e -Campus    • Percentage of online courses in 
SCORM format  • 56% in 2018  • Increase  • BTOR  
  • Percentage of participants rating 
the quality of the online courses 
content with the two top marks  • 2016-18 average: 
85% • Stability  • BTOR   
  • Percentage of participants rating 
the effectiveness of the 
e-Learning website with the two 
top marks  • 2016-18 average: 
93% • Stability  • BTOR  • Absence of external 
disturbance in e -Campus  
  • eLearning success rate  • 2016-18 average: 
67% • Stability  • TAMS database  
• BTOR  •  
  • eLearning drop -out rate  • 2016-18 average: 
28% • Stability  
  • eLearning distinction rate  • 2016-18 average: 
51% • Stability  
Output  1.2: Effective 
partnerships established  • Proportion of activities with the 
involvement of a partner  • 2016-18 average: 
54% • Stability  • TAMS database  • Potential partners are 
interested and have the 
required resources  
• Proportion of partners' 
contribution to the TA activities  • 2016-18 average: 
46% low contributions  • Decrease the 
proportion of 
low 
contributions  
Output  1.3: Members' TA needs 
and priorities identified  • Number of responses to TA 
questionnaires.  • 2017-19 average: 76  • Stability  • BTORs  
• Secretariat 
questionnaires  • Members can identify and 
prioritize their needs  
• No of TPR Follow -ups • 2017-18 average: 8  • Stability   
  WT/COMTD/W/255  
 
- 11 - •  Results  Indicators  Baseline  Target  Evidence  Assumption  
Output  1.4: WTO data and 
databases accessed by 
Members  • Visits by Members' logging in to 
the WTO website  • 49,927 in 2018  • Increase  • WTO Webmaster  • Members' willingness to 
use available WTO data 
sources  
• Number of TA activities including 
training on WTO databases as 
one of their objectives  • 2016-18 average: 63  • Increase  • TAMS database  
• BTORs  • Enough demand for 
training on WTO 
databases  
Key 
Result  2: Acceding 
governments are 
participating in 
accession 
negotiations  • Number of formal or informal 
Working Party meetings held for 
acceding beneficiary 
governments  • 2016-18 average: 6  • Increase  • DG Annual 
Report on 
Accessions  • Accessions are active  
• Governments submit 
required input to their 
Working Parties  
Output  2.1: Acceding 
governments 
participants' knowledge 
on WTO substantive and 
procedural matters 
enhanced  • Number of participants from 
acceding governments attending 
TA activities focused on 
accessions  • To be collected in 
2019 • Stability  • BTORs  
• TAMS Database  
• DG Annual 
Report on 
Accessions  • 1. WTO administrative 
measures d o not reduce 
the access to TA  
• 2. Trained staff are 
retained in relevant 
positions  • Number of roadmaps and 
endgame plans designed  • 2016-18 average: 5  • Stability  
Output  2.2: Accession 
Internship programme 
participants' knowledge 
enhanced  •  100% of at least fully 
satisfactory in 2016 -18 • 100%  • Stability  • Evaluation by the 
Supervisors  • Members present suitable 
candidates  
Output  2.3: Interested 
Members' expertise on 
GPA rules, accession 
negotiations enhanced  • Average no. of relevant 
accession -related documents 
circulated per GPA acceding 
Member  • 2016-18 average: 6.8  • Stability  Documents 
online  GPA accession 
negotiations remain 
active  
Key 
Result  3: Academic 
institutions and other 
stakeholders are 
analysing WTO issues 
and reaching out to 
policy makers  Number of consultative 
processes between WTO Chairs 
and Policy makers  • Consultative 
processes in 2016: 13  • Stability  • Chair holders’ 
annual reports  
• BTORs  • Members' willingness to 
engage with WTO Chairs 
and other stakeholders  
Number of National TA Requests 
for Academic institutions  • To be collected in 
2019 • stability  • BTORs  • Members' willingness to 
engage with Academic 
stakeholders   
  WT/COMTD/W/255  
 
- 12 - •  Results  Indicators  Baseline  Target  Evidence  Assumption  
Output  3.1: Research on trade 
policy and WTO -related 
matters generated  • Number of publications in 
internationally peer -reviewed 
journals  • 13 in 2016  • Increase  • Publications  
• Chair holders’ 
annual reports  
• WCP final 
evaluation  • Compliance with 
programme guidelines  
• Quality of research papers as 
measured by SJR and SNIP 
ratings of the journals in which 
the Chairs' research is published  • Average SNIP rating 
in 2016: .69  
• Average SJP  
rating in 2016: .38  • Increase  • Publications  
• Chair holders’ 
annual reports  
• WCP final 
evaluation  
Output  3.2: Courses with WTO 
content developed by 
the WCP  • Number of students in WTO -
related courses or getting a 
degree per year  • 2,091 students 
trained by the Chairs 
in 2016  • Stability  • Chair holders’ 
annual reports  •  
Output  3.3 Students trained on 
WTO work and issues  • Number of requests from 
Academic institutions not 
designated as WTO Chairs  • 985 students and 
academics trained by 
the WTO in 2016  • Stability  • BTORs  • Financial resources are 
available for these 
activities  
Output  3.4: WTO Chairs 
research papers 
disseminated to other 
stakeholders and policy 
makers  • Number of outreach events per 
year • 54 in 2016  • Increase  • Chair holders’ 
annual reports  • Chairs remain active  
• Involvement of key policy 
makers  • 383 in 2016  • Increase  • Chair holders’ 
annual reports  
Key 
Result  4: Non -governmental 
stakeholders and 
legislators are aware 
of and knowledgeable 
about WTO work and 
issues  Diversity of WTO related topics 
covered by statements, reports, 
and papers produced by this 
group of stakeholders  • In 2017: 26 reports?  • Stabilit y • Media reports 
when available  
• Statements 
made by 
legislators, 
business 
associations, 
NGOs when 
available  • 1. Stakeholders remain 
interested in WTO issues  
• 2. Partners remain 
interested in trade issues   
  WT/COMTD/W/255  
 
- 13 - •  Results  Indicators  Baseline  Target  Evidence  Assumption  
Output  4.1: Understanding of 
WTO work and issues by 
non-governmental 
stakeholders improved  • Number of seminars, workshops 
conducted  • 2016-18 average: 26 
activities  • Stability  • TAMS database  • Legislators, journalists 
and civil society remain 
interested in WTO issues  
• Number of key non -
governmental stakeholders 
reached during the year  • 2016-18 average:  
• 437 legislators  
• 66 journalists  
• 32 civil society  • Stability  • BTORs  
Output  4.2: Improved access to 
WTO issues through 
online training material 
for self -study  • Number of training material 
consulted  • 2016-18 average: 
3,157  • Stability  • Online form  • Public interest in WTO 
issues  
Output  4.3 Information on WTO 
trade -related TA 
accessed  • Number of views of the web 
pages on technical assistance 
including news items on this 
subject  • 2016-18 average: 
59,105  • Stability  • WTO webmaster  • Public interest in WTO 
issues  
Output  4.4: Young 
professionals are 
exposed to WTO work 
and issues  • Number of Young Professionals 
engaged annually by the WTO  • 15 in 2018  • Stability  • TAMS Database  • Enough suitable 
applications received  
• Final evaluation rate of each 
intern's performance by their 
supervisor  • 100% at least fully 
satisfactory in 2017 -
18 • Stability  • Evaluation by the 
supervisors   
  WT/COMTD/W/255  
 
- 14 - 4.1  Indicators  
Indicator  Purpose of indicator  Baseline  Target  Data collection  Expected timeline  Responsible  
• Share of developing 
Members' total trade 
(imports and exports)  • To assess whether 
beneficiary countries are 
improving their capacity 
and use of trade policy -
making, legislative 
compliance, 
multilateral/regional trade 
negotiations and 
implementation of 
international trade 
agreements.  • 2015-17 
average: 
40.4%  • > 40%  • WTO Statistics  
• Interviews/focus group 
discussions with target 
former trainees, their 
supervisors, head of 
national agencies that 
benefited for past 
trainings, and any other 
relevant national 
authorities  By December of each 
year  • WTO Economic 
Research and 
Statistics Division  
(focal point)  
• TAME Director and 
Evaluation 
Specialists  
• Share of LDC Members' 
total trade (imports and 
exports)  • To assess whether 
beneficiary countries are 
improving their capacity and 
use of trade policy -making, 
legislative compliance, 
multilateral/regional trade 
negotiations and 
implementation of 
international trade 
agreements.  • 2015-17 
average: 
1.07%  • > 1.00%  • WTO Statistics  
• Interviews/focus group 
discussions with target 
former trainees, their 
supervisors, head of 
national agencies that 
benefited for past 
trainings, and any other 
relevant national 
authorities identified by 
TAME as a useful 
informant  By December of each 
year  • WTO Economic 
Research and 
Statistics Division  
• TAME Director and 
Evaluation 
Specialists  
• Number of documents 
submitted by beneficiary 
developing and LDC 
Members to WTO Bodies  • To assess whether 
beneficiary countries are 
improving their capacity and 
use of  legislative 
compliance , notifications 
and implementation of 
international trade 
agreements.  • Annual average 
2016-18: 
• Developing: 
537 
• LDCs: 81  • Increase  • WTO Documents online 
database  • Ongoing 
collection of data  
• Statistical 
Reviews and 
Reports  • Languages, 
Documentation and 
Information  
Management 
Division   
  WT/COMTD/W/255  
 
- 15 - Indicator  Purpose of indicator  Baseline  Target  Data collection  Expected timeline  Responsible  
• Number of outstanding 
notifications by TA 
beneficiary Member  • To assess whether 
beneficiary countries are 
complying with their 
transparency obligation 
requirin g to report trade 
measures to the relevant 
WTO body  • 28 per 
beneficiary 
Member on 
average in 
2016-18 • Decrease  • Notifications database 
(CRN)  Ongoing process and 
according to 
Notifications deadlines  • List of focal points 
on notification 
matters12 
• Number of participants 
completing each PLS 
level • To evaluate the level of 
overall participation in TA 
activities and the proportion 
of participants in each of 
the three levels  • Annual average 
2016-18: 
• Level 1 - 4,271  
• Level 2 - 
12,408  
• Level 3 - 1,802  • Stability for 
levels 1 and 
2 
• Increase in 
level 3  • TAMS D atabase  By December of each 
year (the information is 
analysed in the Annual 
Report  which reflects 
the previous year ) • TAME Director  and 
Database 
Specialists  
• Number of national 
activities requested  • To assess the needs of 
beneficiary countries on 
specific topic(s)  • 2016-18 
average: 147  • Increase  • TAMS database  By the end of the year  • TAME Director  and 
Database 
Specialists  
• Number of beneficiary 
Members and Observers 
participating in TA 
courses  • To measure the overall 
number of participations  
by Members and 
Observers  • 2016-18 
average: 174  • Stability  • TAMS database  By the end of the year  • TAME Director  and 
Database 
Specialists  
• Number of beneficiary  
LDCs participating in TA 
courses  • To measure the overall 
number of participations  
by LDCs Members  • 2016-18 
average: 46  • Stability  • TAMS database  By the end of the year  • TAME Director  and 
Database 
Specialists  
• Interactive training 
methods' share of time 
in each activity 
(exercises, case studies, 
simulations, Q&A, 
participation in 
Committee meetings, 
etc.) • To assess the level of 
interaction in each activity 
by type of learning 
method. Interactivity is a 
key component of adult 
learning.  • 2016-18 
average:  
• - 43% lectures  
• - 57% 
interactive 
methods  • Increase 
share of 
interactive 
training 
methods to 
an average 
of 60%.  • TAMS database  
• ITTC trainer(s)  By the end of the year  • TAME Director  and 
Database 
Specialist s 
• ITTC Trainer(s) and 
organizers  
• M&E Specialists   
 
12 Please  see: https://www.wto.org/english/tratop_e/devel_e/tecc op_e/tctdnt2_e.htm    
  WT/COMTD/W/255  
 
- 16 - Indicator  Purpose of indicator  Baseline  Target  Data collection  Expected timeline  Responsible  
• Success rate in 8 -week 
RTPCs and ATPCs  • To evaluate the number of 
participants who 
succeeded in two relevant 
products: RTPC and ATPC  • 2016-18 
average 
(RTPC): 95%  • Stability  • BTORs  Within one month after 
the delivery of the 
activity  • ITTC O rganisers 
and trainers  
• Distinction rate in 8 -
week RTPCs and ATPCs  • To evaluate the number of 
participants who obtained 
a Certificate with 
distinction in two relevant 
products: RTPC and ATPC  • 2016-18 
average 
(RTPC): 32%  • Stability  • BTORs  Within one month after 
the delivery of the 
activity  • ITTC Organisers 
and trainers  
• Percentage of 
participants giving a top 
mark to the overall 
content and results in 
RTPCs, ATPCs and 
advanced thematic 
courses  • To evaluate the extent of 
transferring an appropriate 
level of knowledge and 
skills on WTO matters to 
participants  • 2016-18 
average:  
• - RTPCs: 61%  
- ATPCs: 83%  
- advanced 
thematic 
courses: 87%  •  
Increase to 
80% 
• Stability  
• Stability  • BTORs  Within one month after 
the delivery of the 
activity  • ITTC Organisers 
and trainers  
• Number of interns 
trained in various 
Internship programmes  • To capture the number of 
interns involved in various 
learning -by-doing 
programmes  • 2016-18 
average: FIMIP: 
16; NTP: 16; 
RCI: 7 • Stability  • BTORs  
• TAMS database  Within one month after 
the delivery of the 
activity  • ITTC Organizers 
and trainers  
• TAME M&E 
Special ists/Analys ts 
• Final evaluation of each 
intern by their 
supervisor  • To assess the quality of 
intern's contributions to 
their unit and evaluate 
their match with the 
selection criteria   • 100% of at 
least fully 
satisfactory in 
2016-18 • Stability  • Evaluation by the 
supervisors  End of the internship  • TAME M&E 
Specialists/Analys ts 
• New, revised or 
translated online 
courses released during 
the year  • To assess the level of 
novelty and availability of 
courses in the three WTO 
languages  • 2016-18 
average: 10  • Stability  • BTOR  
• eCampus Platform  Within one month after 
the delivery of the 
activity  • eLearning officers 
and platforms 
holders  
• Percentage of online 
courses in SCORM 
format  • To assess the level of 
interactivity and 
compatibility of eLearning 
courses  • 56% in 2018  • Increase  • BTOR  
• eCampus Platform  Within one month 
after the delivery of 
the activity  • eLearning officers 
and platforms 
holders   
  WT/COMTD/W/255  
 
- 17 - Indicator  Purpose of indicator  Baseline  Target  Data collection  Expected timeline  Responsible  
• Percentage of 
participants rating the 
quality of the online 
courses content with the 
two top marks  • To measure the approval 
rating of online courses by 
the two top marks  • 2016-18 
average: 85%  • Stability  • BTOR  
• eCampus Platform  Within one month 
after the delivery of 
the activity  • eLearning officers 
and platforms 
holders  
• Percentage of 
participants rating the 
effectiveness of the 
e-Learning website with 
the two top marks  • To assess the level of 
interactivity and 
compatibility of the 
eLearning platform  • 2016-18 
average: 93%  • Stability  • BTOR  
• eCampus Platform  Within one month 
after the delivery of 
the activity  • eLearning officers 
and platforms 
holders  
• eLearning drop -out rate  • To measure the number of 
participants who 
completed the entire 
online course  • 2016-18 
average: 28%  • Stability  • BTOR  
• eCampus Platform  Within one month 
after the delivery of 
the activity  • eLearning officers 
and platforms 
holders  
• Proportion of activities 
with the involvement of 
a partner  • To assess the share of  TA 
partners involved in WTO 
TA activi ties • 2016-18 
average: 54%  • Stability  • TAMS database  By the end of the year  • TAME Director and 
Database 
Specialists  
• Visits by Members' 
logging in to the WTO 
website  • To measure the quantity of 
visits by Members' logging 
in to the WTO Website  • 49,927 in 
2018 • Increase  • WTO Webmaster  Ongoing monitoring  • WTO Webmaster  
(Information and 
External Relations 
Division – IERD)  
• Number of formal or 
informal Working Party 
meetings held for 
acceding beneficiary 
governments  • To capture the rate of 
activity during the 
'accession process' by 
beneficiary acceding 
governments  • 2016-18 
average: 6  • Increase  • DG Annual Report on 
Accessions  
• Accession Working Party 
Meetings  • Ongoing  
• Coordination with 
Accession by the 
end of the year  
 • Accession Division 
(focal point)  
 
• Number of participants 
from acceding 
governments attending 
TA activities focused 
on accessions  • To assess the relevance 
and target of TA activities 
focused on accessions  • To be 
collected in 
2019 • Stability  • BTORs  
• TAMS Database  
• DG Annual Report on 
Accessions  
• Accession Working Party 
Meetings  • Ongoing  
• Coordination with 
Accession by the 
end of the year  • Trainers and 
organizers  (ITTC)  
• Accession Division   
  WT/COMTD/W/255  
 
- 18 - Indicator  Purpose of indicator  Baseline  Target  Data collection  Expected timeline  Responsible  
Number of consultative 
processes between 
WTO Chairs and Policy 
makers  • To assess WTO outreach to 
WTO Chairs and Policy 
makers  • Consultative 
processes in 
2016: 13  • Stability  • Chair holders’ annual 
reports  
• BTORs  Within one month 
after the delivery of 
the activity  • WTO Chairs 
Programme Unit  
• WCP Coordinator  
Number of National TA 
Requests for Academic 
institutions  • To assess the extent of 
involvement of Academic 
institutions in WTO TA 
activities  • To be 
collected in 
2019 • stability  • BTORs  Within one month 
after the delivery of 
the activity  • WTO Chairs 
Programme Unit  
• WCP Coordinator  
• Trainers and 
organizers  
• Number of publications 
in internationally peer -
reviewed journals  • To assess the quantity of 
Journals in which WTO 
Chairs published articles  • 13 in 2016  • Increase  • Publications  
• Chair holders’ annual 
reports  
• WCP final evaluation  By the end of the year  • WTO Chairs 
Programme Unit  
• WCP Coordinator  
• Quality of research 
papers as measured by 
SJR and SNIP ratings 
of the journals in which 
the Chairs' research is 
published  • To assess the quality of 
Journals in which WTO 
Chairs published articles  • Average 
SNIP rating 
in 2016: .69  
• Average SJP 
rating in 
2016: .38  • Increase  • Publications  
• Chair holders’ annual 
reports  
• WCP final evaluation  By the end of the year  • WTO Chairs 
Programme  Unit 
• WCP Coordinator  
• Number of outreach 
events per year  • To measure the outreach 
level of the Secretariat in a 
specific year  • 54 in 2016  • Increase  • Chair holders’ annual 
reports  By the end of the year  • WTO Chairs 
Programme  Unit  
• WCP Coordinator  
• Outreach 
Counsellor  
• IERD 
Diversity of WTO 
related topics covered 
by statements, 
reports, and papers 
produced by this group 
of stakeholders  • To measure the diversity 
of WTO related topics  • In 2017: 26 
reports?  • Stability  • Media reports when 
available  
• Statements made by 
legislators, business 
associations, NGOs when 
available  • Ongoing  
• By the end of the 
year • Information and 
External Relations 
Division   
  WT/COMTD/W/255  
 
- 19 - Indicator  Purpose of indicator  Baseline  Target  Data collection  Expected timeline  Responsible  
• Number of seminars, 
workshops conducted  • To measure the outreach 
level of the Secretariat  • 2016-18 
average: 26 
activities  • Stability  • TAMS database  By the end of the year  • TAME's Database 
Specialists and 
Support Officers  
• Outreach 
Counsellor  
• Number of key non -
governmental 
stakeholders reached 
during the year  • To measure the outreach 
level of the Secretariat 
targeting non -
governmental stakeholders  • 2016-18 
average:  
• 437 
legislators  
• 66 journalists  
• 32 civil 
society  • Stability  • BTORs  Within one month 
after the delivery of 
the activity  • Trainers and 
organizers  
• Outreach Counsellor  
• Number of training 
material consulted  • To assess the number of 
WTO training material 
consulted by the large 
audience  • 2016-18 
average: 
3,157  • Stability  • Online form  • Ongoing  • WTO Webmaster  
• IERD 
• Number of views of the 
web pages on technical 
assistance including 
news items on this 
subject  • To measure the number of 
views and interest by topic 
of the web pages on 
technical assistance  • 2016-18 
average: 
59,105  • Stability  • WTO webmaster  • Ongoing  • WTO webmaster  
• IERD 
• Number of Young 
Professionals engaged 
annually by the WTO  • To assess the number of 
YPs engaged by WTO in a 
specific year.  • 15 in 2018  • Stability  • TAMS Database  By the end of the year  • TAME M&E 
Specialists/Analists  
• HR Division  
 
5  DATA FLOW  
5.1.  The information that follows in the next page is a graphical representation of a flow chart showing how the data input will flow from the place 
where it is collected up to the management team, leading to the final evaluations and reports.   
  WT/COMTD/W/255  
 
- 20 - 5.2.  The methodology used  as well as the source of information  is detailed in Annex es I, II and III of th e M&E Plan . 
WT/COMTD/W/255  
 
- 21 - 
 
  6  DATA ANALYSIS AND REPORT ING FLOW  
6.1.  It is recommended that M&E progress information, results and accountability are reported in 
the following order:  
1. Unit Managers and Staff  
M&E results should feedback to managers and their staff with comments on forward actions 
before reports are finalised for wider distribution.  
Time should be allowed to see the results of any change before reporting to a wider audience.  
 
 
 
 
2. WTO Management  
WTO Management should receive the M&E reports once the internal review by ITTC staff has 
been conducted.  
WTO Management may wish to comment before the report is circulated further afield; a period 
of time, say one week, should therefore be allocated to them for making comments before 
circulating the report widely.  
 
 
 
3. Beneficiaries  
Beneficiaries must apply wh at they have learnt in a course in their work situation.  They need 
short-term close interaction with the Unit who delivered the service as they may need further 
assistance.  
 
 
 
 
 
4. Government Ministries and other Stakeholders  
Government Ministries and other Stakeholders in their capacity of applying agreements and 
rules need to be kept informed, in order to support officials who have received TA training in 
applying their learning.  
They may also be involved with management in making changes if plans are  not achieved.  The 
more they know the more involved they can be.  They are responsible for supporting the 
achievement of Outcome to Impact level objectives and must feel part of the process of 
strengthening their institutions to apply WTO agreements and r ules. 
 
 
 
 
 
5. Members and Donors  
Members and d onors need to know how the funds were spent, how the funds were managed, 
and how WTO management dealt with the minor obstacles during implementation, and if the 
use of fun ds led to sustainable benefits.WT/COMTD/W/255  
 
- 22 - 
 
  ANNEX I: ONSITE EVAL UATION OF TA ACTIVIT IES 
1  BACKGROUND AND PURPO SE 
1.1.  As part of its mandate to promote RBM across WTO  TA activities, TAME undertakes regular 
reviews to assess the effectiveness of TA provided to Members and to identify good 
practice, innovative approaches, and areas for further improvement.   These reviews are 
carried out using a variety of instruments (e.g. activity -based on -site evaluations, 
activity/programme -based ex -post evaluations, country -based impact assessments, etc.) that 
measure different levels of training effectiveness according to the Kirkpatrick's Training Evaluation 
Model.  This model identifi es four levels:  
a. Reaction:  this level measures how trainees reacted to the training;  
b. Learning:  this level measures what the trainees have learnt from the activity;  
c. Behaviour:  this level evaluates how far the trainees have modified their behaviour as a 
result of the training received;  
d. Results:  finally, this level tries to assess the outcome of the training for the teams or 
organisations in which the trainees operate.  
1.2.  Onsite evaluations are used to assess the Reaction and Learning levels:  
a. To assess the learner's Reaction  to training, the evaluator observes the training's 
dynamics through participants' body language and interaction with the trainers and peers.  
The evaluator also engages in informal dialogue (guided interviews) with the participants 
throu ghout the training (usually during the breaks) to collect their impressions and 
investigate any attention -grabbing behaviour observed (e.g. difficulty with the training 
language).  Where possible, the evaluator leads the final evaluation session (using foc us-
group discussion technique) to gather first -hand participants' feedback on the training.  
b. To assess the Learning achieved by participants during the course, the evaluator 
considers factual evidence (results of pre - and post -course quizzes, individual pro jects, 
participants' self -reflection and takeaways sessions, etc.) but also observes the change in 
knowledge and skills of participants (ease of moving to higher level of difficulty, pertinence 
of the discussions, etc.).  Factors affecting the ease of Lear ning are assessed by the 
evaluator: adequacy of expected learning outcomes (objectives) with the level of training, 
adequacy of participants' prior knowledge and thus homogeneity of the audience, 
appropriateness of training methods used, preparation of the  training, etc.  
1.3.  Several TA activities are selected annually by TAME for onsite evaluation taking into account a 
set of objective criteria: rotation of the WTO topics and of TA regions, mix of PLS levels (2 or 3), 
representative coverage of the various Key Results of the TA plan, features of the activities (duration, 
standalone or module in larger activity, etc)….  In some cases, an activity can be selected for 
evaluation outside of these criteria for various reasons including:  in response to a substantiate d 
request from the organizers; for pilot -testing; to explore a feature of special interest (e.g. an 
innovative approach); as part of a meta -evaluation on a given topic (i.e. multiple activities on the 
same topic can be evaluated successively for comparativ e analysis), etc.  
2  IMPLEMENTATION OF ON SITE EVALUATIONS  
Upon selection of a TA activity for an onsite evaluation:  
• TAME contacts the officers responsible for the activity (trainers and ITTC responsible officer) 
to collect preliminary information about the ac tivity: background to the request (for regional 
or national activities), expected outcomes, participants profiles, selection process, how is 
the activity being organized (partners, shared -responsibilities, potential outside trainers, 
etc.).  WT/COMTD/W/255  
 
- 23 - 
 
  • TAME observes t he preparation of the activity by being copied on relevant email exchanges, 
attending key preparatory meetings (with partners when relevant), by being updated on 
selection process; and by being given access to any documents' repository (GoogleDrive, 
Virtua l Classroom, etc.) shared with partners and students.   As an observer, TAME does 
not intervene in the decision -making process but only takes notes of the 
organizational arrangements for subsequent analysis.  
• TAME explores with the organizers the best way to  integrate the evaluation into the 
programme without disturbing the natural flow of the activity (e.g. when possible TAME is 
provided with a few minutes slot during the opening session of the course to briefly introduce 
the evaluation and seek participants ' collaboration, also ideally a final evaluation session of 
at least 1 hour is included in the programme and facilitated by TAME).  Activity's PEF can be 
modified by TAME in some cases.  
During the activity:  
• TAME observes the activity and engages dialogue w ith the participants on the margin of the 
activity (e.g. during the breaks) and with the trainer(s) and external speakers as needed.  
• TAME may interview local partners, when relevant.  
• TAME moderates the final evaluation of the activity.  
• TAME considers the t ools used by the trainers to test the acquired knowledge.  
• At the end of the activity, TAME may hold a debrief session with the trainer(s) and partners 
to share initial feedback.  
After the activity:  
• TAME analyses the data gathered during the evaluation and when necessary triangulates the 
findings with historical data (review BTORs and programmes of past editions of the same 
activity).  
• TAME reviews trainers' BTOR and analyses the appropriateness of RBM indicators and 
sources of verification (results of pre - and post -course tests, final exam, project work, etc.) 
used by the trainers to assess activity's performance.  
• TAME prepares a draft evaluation report and shares it with the organizers (trainers and ITTC 
responsible officer) for correction of possible factual errors or clarification.  
• TAME finalizes the report based on input received and circulates it internally to selected 
recipients (activity evaluation reports are for restricted circulation within WTO).  
• Non-sensitive information (e.g. generic  data from the activity, recommendations applicable 
to other activities, good practice, lessons learned, etc.) is disseminated by TAME through 
the TA annual performance report and other means, as applicable.  
3  CONTENT OF THE EVALU ATION REPORT  
3.1.  There is no sta ndard outline for the activity evaluation report.  The structure of the report will 
depend on the areas of emphasis identified for each activity.  However, throughout its various 
sections, it will seek to address three of the five OECD/DAC criteria featured in the evaluation 
framework for TA: relevance, effectiveness and efficiency.  
3.2.  A typical report would address:  
a. the background to explain the histor y of the activity when relevant, its context and general 
features; and the evaluation methodology;  WT/COMTD/W/255  
 
- 24 - 
 
  b. the preparation of the activity and the efficiency (mainly when involving external 
partner(s));  
c. the relevance of the activity to the participants and the ade quacy of the objectives;  
d. the delivery of the programme (content and methods) and its alignment with for the 
objectives; and  
e. the conclusions arising from the analysis (highlighting lessons learned and good practice) 
and recommendations for future improvemen ts, if any.  
 
Annex  A 
Participant's Evaluation Form (PEF)1 
 
 
 
Venue     Date    Mission code  
 
 
 
Title of TA Activity  
 
 
 
1. What is your general opinion about this TA activity?  
2. Is this your first WTO TA activity? (Y/N)  
3. Your knowledge of the subject matter was: 
a. Prior to the activity  
b. After the activity  
4. To what extent were the following specific objectives achieved?  (Description of the specific 
objectives)  
5. Do you find the TA activity professionally useful or relevant?  
6. Your overall opinion of the TA activity in  the following areas : 
a. Quality of lectures and presentations  
b. Use of audio -visual aids  
c. Practical exercises, case studies  
d. Interaction with TA provider(s)  
e. Quality of training materials (handouts, CDs)  
f. References to national/regional experience  
g. Duration of the TA activity  
7. The logistical arrangements for the TA activity were  
8. Comments  
 
 
1 All the questions are mandatory except for the comment box (Q8). The rating questions go from 1to  5, 
1 being the lowest score and 5 the highest.  WT/COMTD/W/255  
 
- 25 - 
 
  ANNEX II: COUNTRY -BASED TA EFFECTS AND I MPACT ASSESSMENT  
1  BACKGROUND AND PURPO SE 
1.1.  As part of its mandate to promote RBM across WTO  TA activities, TAME undertakes regular 
reviews to assess the effectiveness of TA provided to Members and to identify good 
practice, innovative approaches and areas for further improvement.   These reviews are 
carried out using a variety of instruments (e.g. a ctivity -based on -site evaluations, 
activity/programme -based ex -post evaluations, country -based effect and impact assessments, etc.) 
that measure different levels of training effectiveness according to the Kirkpatrick's Training 
Evaluation Model.  This mode l identifies four levels:  
a. Reaction:  this level measures how trainees reacted to the training;  
b. Learning:  this level measures what the trainees have learnt from the activity;  
c. Behaviour:  this level evaluates how far the trainees have modified their behaviour as a 
result of the training received;  
d. Results:  finally, this level tries to assess the outcome of the training for the teams or 
organisations in which the trainees operate.  
1.2.  Country -based effects and impact assessment of TA activities are used to assess the  
Behaviour and Results levels:  
a. To assess the change in past TA participants' Behaviour  at work as a result of trainings 
provided by the WTO, the evaluator will seek to gather information through self -
assessment and 360° feedback.  
b. To assess the Results of the trainings provided by the WTO for the teams or organisations 
in which former trainees operate, the evaluator will collect evidence -based data about 
factors such as efficiency gains in the workplace, specific achievements, or introduction of 
new practice s or systems in the workplace to create a positive impact on desired results.  
1.3.  In addition, where possible TAME seeks to gather information on any possible impact that WTO 
TA has collectively contributed to at the national level and to collect good practice s implemented by 
TA beneficiaries to promote the sustainability of TA results in order to disseminate them to other TA 
beneficiaries during other field missions or through TA annual performance reports.  
1.4.  Furthermore, through its exchange with former TA part icipants, TAME seeks to gather 
suggestions on how to improve TA activities either content - or process -wise.  Meetings with past 
participants, their supervisors and line ministries, also provide an opportunity for TAME to collect 
input on TA needs for the f uture. Information gathered in this regard is shared with the relevant 
Secretariat staff.  
1.5.  For an efficient use of resources TAME seeks, as much as possible, to couple country -based 
effects and impact assessment with onsite or ex -post evaluations. However, a country -based 
assessment can be implemented independently should a TA monitoring & evaluation need arise.  
2  IMPLEMENTATION OF TH E EVALUATION  
Upon selection of a country for a TA effect and impact assessment:  
• TAME analyses the history of participation of t he country in WTO TA activities over the past 
5 years. The most frequently attended subjects and the main beneficiary national institutions 
are identified.  
• TAME contacts country's Permanent Mission to the WTO and/or focal point in capital to 
organize the f ield visit. An explanatory note of the purpose and methodology of the 
assessment is shared with the authorities.  WT/COMTD/W/255  
 
- 26 - 
 
  During the field mission:  
• TAME holds focus group discussion meetings with multiple national authorities1 and/or 
individual meetings to collect feedback on WTO TA received and on outstanding needs. 
Guiding questions for the interviews are presented in Annex A.  
• Interviews/focus group discussions target former trainees, their supervisors, head of 
national agenc ies that benefited for past trainings, and any other relevant national 
authorities identified by TAME as a useful informants (e.g. agencies that have not benefited 
from WTO TA in their specific area of work can be interviewed by TAME if the area appears 
to be a priority for the country, to understand the background to non -participation).  
• TAME collects information on outstanding TA needs and any supporting data or information 
to provide evidence of the relevance of TA provided, its impact and sustainability (e.g. 
information on national policies, data on gain of efficiency or internal curricula developed 
based on WTO trainings, …).  
After the activity:  
• TAME analyses the data gathered during the evaluation and when necessary triangulates the 
findings with othe r data (review of BTORs of activities, or participation in WTO bodies, 
submission of notifications, etc.) and/or interviews with relevant staff members.  
3  CONTENT OF THE CONCL USION REPORT  
3.1.  There is no standard outline for the country assessment report.  The s tructure of the report 
and its thoroughness will depend on the extent of past participation of the country in WTO TA; the 
variety and number of national interlocutors met during the field mission; and the scope/richness of 
information provided by national authorities.  However, throughout its various sections, it will seek 
to address the five OECD/DAC criteria featured in the evaluation framework for TA: relevance, 
effectiveness, efficiency, impact and sustainability.2  Effectiveness and efficiency are addr essed at 
an aggregated level and not on an activity -by-activity basis.  
3.2.  A typical report could address:  
a. The background and purpose of the country -based TA effects and impact assessment;  
b. A brief summary of the history of participation of national authorities  in TA activities;  
c. TAME's analysis of the relevance of WTO TA to country's policy context and needs 
(alignment with national priorities, harmonization and synergies with other TA received 
from other providers);  
d. Indications gathered on the effectiveness of WTO TA and any suggestions for improvement 
of TA procedures;  
e. Evidence of TA results for beneficiaries at individual level and impact of their enhanced 
capacity for their host institutions and national trade -related technical capacity as a whole;  
f. Evidence o f contribution of WTO TA to higher order objectives (impact on trading 
capacity);  
g. Outstanding TA needs (in terms of subjects and type of activities).  
 
 
1 If necessary, TAME may use visuals (ppt slides) to share the trends of country's past p articipation in 
WTO TA and stimulate the discussion.  
2 See: www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm   WT/COMTD/W/255  
 
- 27 - 
 
  Annex A  
Below are examples of guiding questions that the evaluator will seek to address when meeting with 
national informants.  These questions are provided for reference only, for a scenario of an interview 
with a supervisor of former TA participants .  They  will be adapted to the role of the 
institution/individual interviewed and will focus on any other element of interest arising from TAME's 
preliminary desk analysis of the country's context and history of past participation in WTO TA.  They 
are likely to evolve according to the flow of the discussion.  
 
• Have you noticed from WTO TA any effects and impacts on:  
- gaining advantages for your country from the multilateral trading system [e.g. use 
of dispute settlement system, TPR, improved trading environment, foreign direct 
investment, transfer of technology, increased trade, etc.],  
- concrete improvements on the im plementation of the WTO Agreements [e.g. 
increased number notifications, organization of national focal points for TBT and SPS 
Agreement, other],  
- an increased and more active participation in trade negotiations [increased 
leadership and participation in the negotiations through elaboration of the country 
position, presentation of proposals and active participation in the meetings directly 
in the WTO or through a regional grouping].  
 
• Have you noticed any effects and impacts on the staff that have participa ted in 
WTO TA programme of activities?  
[e.g. Knowledge of WTO Agreements; quality of analytical work; initiative regarding his 
work; more independence at work; confidence in contributing to meetings; more active in 
writing papers or proposals; suggestions for better implementation of WTO Agreements;…]  
 
• To what extent your division / service / department has benefited from the 
knowledge and skills acquired by your staff through WTO TA?  
[e.g. Sharing of responsibilities; increased leadership on certain subjec ts; suggestions; 
contribution to the preparation of the country position for multi - or bi -lateral trade 
negotiations; other]  
 
• Were the knowledge and skills acquired by your staff relevant for meeting the 
needs of your division / service / department?  Are these knowledge and skills 
regularly applied?  [Provide examples]  
• What is your assessment of the overall effectiveness and efficiency of WTO TA? Are 
there any suggestions on how WTO TA could be improved?   
[e.g. Procedures of TA (availability of information  on TA offer, application and selection 
procedures, etc.), duration of the activities; more advanced courses;  more e -training 
courses;  less quantity of activities but better prepared to target the right needs and 
government staff;  less regional and more  national activities or vice -versa;  multiyear 
approach; use of national resources in delivery of activities, blended -learning approaches, 
more activities targeting non -government officials (academia, trade -support institutions, 
private sector associations , etc.…]  
 
• According to our data, X staff members from your team participated in X activities 
over the past 5 years, were some of these activities more useful than others in 
terms of effects and impact?  
 
• Does your institution receive TRTA from other providers? How do you qualify the 
synergy between WTO's TA and that received from other providers? What is the 
value -added of WTO's TA compared to other providers? Any suggestions to 
enhance synergies and harmonization of TRTA received by your institution?  
 
• What are the outstanding needs of your institution?WT/COMTD/W/255  
 
- 28 - 
 
  ANNEX III: EX -POST E VALUATIONS  
1  BACKGROUND AND PURPO SE 
1.1.  As part of its mandate to promote RBM across WTO  TA activities, TAME undertakes regular 
reviews to assess the effectiveness of TA provided to Members and to identify good 
practice, innovative approaches and areas for further improvement.   These reviews are 
carried out using a variety of instruments (e.g. activity -based on -site evaluations, 
activity/programme -based ex -post evaluations , country -based effect and impact assessments, etc.) 
that measure different levels of training effectiveness according to the Kirkpatrick's Training 
Evaluation Model.  This model identifies four levels:  
a. Reaction:  this level measures how trainees reacted to  the training;  
b. Learning:  this level measures what the trainees have learnt from the activity;  
c. Behaviour:  this level evaluates how far the trainees have modified their behaviour as a 
result of the training received;  
d. Results:  finally, this level tries to ass ess the outcome of the training for the teams or 
organisations in which the trainees operate.  
1.2.  Ex-post evaluations of a particular TA activity (course) or a programme are used to assess the 
Behaviour and Results levels:  
a. To assess the change in past particip ants' Behaviour  at work as a result of their 
participation in the activity/programme being evaluated. The evaluator will seek to gather 
information through self -assessment and 360° feedback.  
b. To assess the Results of the activity/programme for the teams or organisations in which 
former trainees operate, the evaluator will collect evidence -based data about factors such 
as efficiency gains in the workplace, specific achievements, or introduction of new 
practices or systems in the workplace to create a positive  impact on desired results.  
1.3.  In addition, ex -post evaluations also provide an opportunity to collect good practices 
implemented at national level to promote the sustainability of the results produced by the 
activity/programme being evaluated (e.g. scheme fo r activity's participant to cascade the training 
to other colleagues, etc.).  
1.4.  Finally, ex -post evaluations allow the evaluator to gather suggestions on how to improve TA 
activity/ programme being evaluated either substantially or process -wise. 
1.5.  By collecting  from past participants (entire population or a representative sample thereof) 
evidence o n Behaviour change and Results  generated by the activity/  programme evaluated and 
analysing the impact patterns, ex -post evaluations provide an answer to whether the 
activity/programme achieved its objectives and has contributed to the desired impact.  
2  IMPLEMENTATION OF TH E EVALUATION  
2.1.  Selection of an activity/programme to be evaluated:  Ex-post evaluations are normally 
implemented for time -limited activities/programmes.  Unlike final evaluations performed  upon 
activity /programme completion to assess whether  they achieved their intended objectives, ex -post 
evaluation s are  conducted some  years later.  Depending on the time expected for outcomes to 
materialize, this lag can be of one, three or five years after the end -date.  For the WTO TA, it is 
unreasonable to implement time  and resources intensive ex -post evaluations for a single edition of 
a TA activity (most of which are of a short duration) .  Instead , ex-post evaluations cover  several 
editions of the activity selected (e.g. ex -post evaluation of the Introduction Course for LDCs over 10 
years).  Therefore, activities/programme s are selec ted based on their recurrence, features of special 
interest, turnover in topics to be covered, etc.  WT/COMTD/W/255  
 
- 29 - 
 
  2.2.  Desk -based scoping work and engagement of activity/programme's implementing 
team(s) : TAME carries out a preliminary review of information related to the act ivity selected 
(mainly through information available in TAMS or some initial discussions with activity/programme 
implementing team).  Based on this scoping work, ToRs are prepared and discussed with the team(s) 
involved.   ToRs provide the background to the  evaluation, the scope of the work, the methodology 
suggested, the timeline and when relevant the expected contribution from the implementing 
team(s).  The methodology can differ depending on the activity/programme evaluated.   Some ex -
post evaluations may involve onsite observation by the evaluator of one edition of the activity, 
country visits, face -to-face interviews of stakeholders, focus group discussions, etc.   The 
stakeholders may differ from an activity to another (donor interviews, external partners , other 
institutions for comparative analysis, etc.).  
2.3.  Develop the evaluation tools:  once the ToRs  have been finalized, tools (online 
questionnaires, interview questions, data collection sheets, etc.) will be elaborated and tested 
whenever possible. TAME will also collect all the documents required for triangulation. These are 
usually requested from th e implementing team (s).  
2.4.  Roll-out the evaluation: This consists in the implementation of the evaluation plan including 
field visits, when relevant.  
2.5.  Analysis phase: TAME analyses the data gathered during the evaluation and when necessary 
triangulates the fi ndings with other historical data (review of BTORs of activities, or participation in 
WTO bodies, submission of notifications, etc.) and/or interviews with relevant WTO staff members.  
3  CONTENT OF THE CONCL USION REPORT  
3.1.  There is no standard outline for the ex -post evaluation report.   The structure of the report and 
its thoroughness will depend on the extent of size of the audience of the activity/programme and 
the scope/richness of information collected.  However, throughout its various sections, it will seek 
to address the five OECD/DAC criteria featured in the evaluation framework for  TA: relevance, 
effectiveness, efficiency, impact and sustainability.1 
3.2.  A typical report could address:  
a. The background and purpose of the evaluation;  
b. A brief summary of the histor y of the activity/programme concerned including any major 
evolution during the review period;  
c. TAME's analysis of the relevance of the activity/programme to beneficiaries' needs 
(alignment with national priorities, harmonization and synergies with other TA received 
from other providers) and to other stakeholders involved;  
d. Indications gathered on the effectiveness of the activity/programme, good practices 
identified and any suggestions for improvement;  
e. Any elements regarding efficiency where possible (includi ng management costs, 
comparison with alternatives, etc.);  
f. Evidence of results for beneficiaries at individual level and impact of their enhanced 
capacity for their host institutions and national trade -related technical capacity as a whole;  
g. Evidence of cont ribution of the activity/programme to higher order objectives (impact on 
trading capacity);  
 
1 See: www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm  WT/COMTD/W/255  
 
- 30 - 
 
  h. Evidence of long -term sustainability and good practices implemented by beneficiaries in 
this regard as well as any features of the activity/programme that contribute(ed) to 
sustainability of the outco mes. 
 
 
__________