RESTRICTEDWORLD TRADE
ORGANIZATIONWT/COMTD/W/58
24 February 1999
(99-0726)
Committee on Trade and Development
EVALUATING WTO'S TECHNICAL COOPERATION
Note by the Secretariat
I. INTRODUCTION
1. At the informal meeting of the Committee on Trade and Development (23 November 1998)
and at its formal meeting (7 December 1998), Members agreed that the approach to evaluation ofWTO technical cooperation should be simple, objective, transparent, and practical.  It can be
qualitative, quantitative, or a combination of both.  It can also comprise internal and or external
audits. In order to develop evaluation modalities, it was agreed to hold informal consultations with
members of the Committee, which had expressed an interest in the subject, the WTO Secretariat and
with the participation of ITC.  These consultations were held on 17 December 1998. The Secretariathad prepared an informal note containing concrete modalities on how to obtain information on
technical cooperation activities, which could provide the basis for and input into a broader evaluation
of WTO technical cooperation.
2. In light of the discussions held at that meeting and comments/suggestions made by Members
and the representative of ITC to improve the proposals, the Secretariat has now revised and refinedthe two questionnaires. The revised versions (see attachments) are submitted to the Committee for its
consideration. These two questionnaires would be used simultaneously for each WTO activity and are
designed to provide information regarding the activ ity held in beneficiary countries, including the
possible impact of the assistance provided and the expected follow-up. Such assessment would
provide an important source of information in the feed-back process and thus be beneficial for the
organization of further activities. Also, this information would be critical in assessing the
effectiveness of various modes of delivery in attaining the main objectives of technical assistance, asdefined in the guidelines for WTO Technical Cooperation, adopted in October 1996. The current
proposals, if adopted, would also generate a feed-back mechanism that allows the lessons of the past
and present to be integrated into future technical cooperation and assistance activities.
3. The present note does not pretend to provide a solution to all issues that have come up in
discussing evaluation modalities, but should be seen as a first, but critical, step in the evaluationprocess, i.e. data collection. Issues were raised that would need to be considered further in light of the
data obtained including the establishment of an appropriate methodology for evaluation, e.g.
quantitative versus qualitative assessments, internal and external evaluation, the criteria for
evaluation, such as the impact on recipients, cost efficiency, degree of success in enhancing
institutional and human trade policy capacity in developing countries.
II. BASIC OBJECTIVES OF TECHNICAL  ASSISTANCE
4. The guidelines for WTO Technical Cooperation (WT/COMTD/8) adopted by the Committee on
Trade and Development on 15 October 1996 identified inter alia , two key objectives:
(i) Assist in the full integration of benefici aries into the multilateral trading system and
contribute to the expansion of their trade;WT/COMTD/W/58
Page 2
(ii) Strengthen and enhance institutional and human capacities in the public sector for an
appropriate participation in the multilatera l trading system; whenever possible and in
consultation with the government concer ned, capacity building activities could include
representatives of the private sector.
5. In order to meet these objectives, the WTO Secretariat has several instruments at its disposal to
deliver technical cooperation, which are outlined in the Implementation Modalities (WT/COMTD/W/29
Rev.1) of 17 October 1997.  While some instruments may seem more appropriate for attaining one or
another objective, all technical cooperation inst ruments are geared towards achieving both goals.
6. The Committee adopted language on the Monito ring, Evaluation and Follow-up of technical
cooperation, as reflected in para. 5 of the Implementation Modalities:
(i) "To ensure the effectiveness of technical cooperation activities administered by the
Secretariat and the optimum use of resources, the Committee on Trade and
Development shall undertake annual and if necessary more frequent reviews, on the
basis of reports provided by the WTO Secret ariat.  In this context, the Committee on
Trade and Development shall consider, at the appropriate time (say every three years),
the possibility of commissioning an exte rnal evaluation of the WTO Secretariat's
technical cooperation activities, based on the Guidelines for Technical Cooperation.
Such an evaluation should follow inte rnationally recognized methodologies.
(ii) Appropriate follow-up activities may be es tablished with a view to ensuring the
effectiveness of technical cooperation and training activities.  These can include
training of trainers programmes, the use of internet and interactive technology (CD-ROM) for self-training, follow up courses, seminars, workshops (national/regional),
making available local technical expertise and advice."
III. SOME FEATURES OF WTO TECHNICAL COOPERATION
7. While the WTO is not a technical cooperation organization per se , the increased complexity
of the multilateral trading system, combined with  WTO's enlarged membership (134 members), have
meant that technical cooperation now occupies an increasingly important place in WTO's overall
activities, involving most Divisions and staff members, who provide expertise in their area of
competence. In 1998, the WTO carried out and/or participated in over 300 technical cooperation
activities, which were budgeted at some CHF 5mio, over 80 per cent of which were financed through
extra-budgetary resources. This compares with some 79 activities carried out in 1995, with a budgetof some CHF 700.000. The average national technical cooperation activity lasts between 3-5 days,
and costs less than CHF 20.000, while a regional seminar averages 10 working days and costs around
CHF 150.000
8. WTO technical cooperation contains several features that make it different from technical
cooperation carried out by other international organizations, which often have a 'built-in' evaluationmechanism, mostly based on UNDP criteria. It would seem, however, that these criteria are not
directly applicable for WTO's technical coopera tion programmes, as the UNDP criteria are geared
towards 'projects', often self-contained and involving large sums of money, e.g. over CHF 1mio each.
Most projects carried out by other agencies (ITC, UNCTAD, UNDP) are of longer duration than
WTO's activities. In addition, these often lead to more tangible, measurable and quantifiable results.This is not necessarily the case in WTO technical cooperation, with many activities geared towards
human and institutional capacity building. The effects of the assistance could at best be expected to be
measurable over time, but even then, analytical problems would remain on how to measure the impact
and how to establish a direct relation between WTO's input and the outcome.WT/COMTD/W/58
Page 3
IV. DATA GATHERING THROUGH QUESTIONNAIRES
9. For the purpose of systematic information a nd data collection on WTO technical cooperation
activities, two evaluation forms (questionnaires, see attachment) have been prepared in consultation
with all WTO Divisions that are directly or indirectly involved in providing technical assistance and
in coordination with ITC. They would be used si multaneously by all WTO officials involved in the
delivery of technical assistance.  Each questionnaire serves a specific purpose:
10. The first questionnaire, entitled 'country  request and evaluation form' , to be filled out in
part by the Secretariat and in part by the beneficiary country, essentially captures all technical
information regarding the event to be held in th e beneficiary country. One important element in the
form will be to define the objective(s) of the activity. This needs to be done by the beneficiary country
before the activity is carried out.  The WTO Secret ariat can assist, if required to ensure that the
objective can realistically be met by its modes of delivery. It is against a clearly defined objective thatthe attainment or non-attainment of the objective can be measured.
11. Furthermore, at the conclusion of the event (on-site), the representatives of beneficiary
country would be requested to assess the activity and identify a follow-up, if so required. The WTO
official(s) would also be requested to provide an assessment on the activities. The officials would be
expected to provide information on the attendance, the level of participation, whether the target groupwas 'right', the degree of technicality, the skills acquired, whether the programme was completed,
expected follow-up, or any other aspect that it considers critical for attaining the objective of the
mission.
12. As it would appear that for certain activities it takes time to bear fruit, e.g. notification
requirements, introducing and/or adapting legislation, it is proposed to contact the beneficiary countryagain after three to six months. It will be asked again to assess the impact of the activity having taken
place; to provide information on a follow-up to the activity and any other relevant information. This
may include explanations on whether and why it has difficulties in complying with the notification
requirements, implementing agreements and/or following-up on the technical assistance that was
provided, if such was required/expected.
13. Through this questionnaire, a dialogue (interaction) would be established between the
beneficiary country and the WTO Secretariat in a way that hitherto did not exist.
14. The second questionnaire, entitled 'Participant Evaluation / Feed-back Form'  is intended
to improve the quality of WTO's technical cooperation through feed-back by the participants. It is
self-explanatory. It would be handed out to participants in all technical cooperation activities, for
which the WTO has the lead. If properly filled out by participants, it would provide information on
the objective of the participants in attending the WTO activity, whether expectations were met and onvarious aspects of the activity, including the substance, areas covered, presentations, documentation,
logistics, etc. Similar forms are used by ITC. The feed-back of missions would provide a main source
of information on how to further improve the delivery of technical cooperation.
15. These two questionnaires can be refined, expanded and modified in due course, with
experience gained.  To assess the effectiveness of technical assistance, the Committee might thenwish to analyze available information in different ways:
 (i) by topic covered, which would reveal detailed information on WTO areas (subjects)
covered by the relevant WTO Divisions and how the information was assimilated by the
beneficiary countries;
 (ii) by country, which would show the efforts undertaken by the WTO Secretariat with regard
to a particular beneficiary country and the country's response;WT/COMTD/W/58
Page 4
 (iii) by region or sub-region, which would thus cover technical cooperation activities for a given
group of countries,
 (iv) by mode of delivery, as defined in the guidelines and reflected in the manual on technical
cooperation, which provide information on the efficiency of a particular mode of deliveryin achieving a certain objective.
16. Moreover, the information could be analyzed in different ways and provide insights from two
angles, i.e. from the point of view of the WTO Secretariat, as a service provider, and from the
receiving end, i.e. the beneficiary country. Thus, the global picture could be rather complete.
17. As to the evaluation of training through WTO's regular trade policy courses, evaluation is
conducted on an on-going basis by the WTO's Training  Division. This process may be further refined.
V. EVALUATION OF "JOINT" PROGRAMMES AND ACTIVITIES CARRIED OUT
WITH OTHER ORGANIZATIONS
18. The WTO is also involved in several programmes  carried out jointly with other organizations,
such as the Integrated Framework for Trade-Related Technical Assistance (follow-up to the HighLevel Meeting) involving six major international organizations and the Joint ITC/UNCTAD/WTO
Integrated Technical Assistance Programme in Selected Least Developed and Other African Countries
(JITAP). In these programmes, WTO's contribution to the programme is well-defined, and is
sometimes carried out as 'stand-alone' activity, but it nevertheless remains part of a global programme.
19. A proper and full assessment of the project would require more than an assessment of the
WTO component, as the evaluation of WTO's cont ribution in these programmes would thus go in
hand with an overall evaluation as stipulated in these joint programmes and based on the criteria
developed for that purpose.
20. In several cases, the WTO is called upon to provide its expertise in activities and programmes
organized by or jointly held with other organizations.  This can include the World Bank, the WorldCustoms Organization (WCO), the World Intellectu al Property Rights Organization (WIPO), the
International Standards Organization (ISO). In addition, the WTO frequently attends conferences and
workshops, national or regional seminars, organized under the responsibility of other bodies and
where the WTO is called upon to make an input. While all of these activities are often very valuable
in disseminating specific or general information, or in training officials on a specific subject matter
directly related to the WTO Agreements, the impact of WTO's participation cannot be evaluated in
isolation.
21. It would thus seem necessary, for the purpose of evaluating technical cooperation activities, to
distinguish between activities organized by the WTO, with direct responsibility for the activity, and
those where the WTO participates. The evaluation should ideally be confined to activities for which
the WTO has direct responsibility.
VI. EVALUATION OF ACTIVITIES MANDATED UNDER WTO'S TRUST FUNDS
22. As most of WTO's technical cooperation activities are directly financed from member's
contributions to the WTO's  Trust Fund, the Secretariat is often directly accountable to the donor for
the use of the funds.  Moreover, some Members have explicitly specified in their Memorandum of
Understanding (or the exchange of letters accomp anying the donation) with the Secretariat the
elements they wish the (semi-)annual report to co ntain, which can include an evaluation of how the
funds were used and their expected impact on the beneficiary. The Secretariat will continue to meetWT/COMTD/W/58
Page 5
these reporting obligations, where required and at the same time, through the 'global' evaluation
process that is now proposed.
VII. THE EVALUATION PROCESS IN THE WTO
23. The process of evaluation of WTO's technical cooperation activities would be monitored and
centralised in the Technical Cooperation Divi sion, where all information gathered would
systematically be kept, organized and analyzed to facilitate WTO's internal evaluation. Coordination
with the relevant WTO Divisions would be ensured on a continuous basis, as well as any follow-up,
where such is required to obtain the specific information from beneficiary countries.
24. It is proposed to start the process by 1 July, so that a first evaluation report can be produced
by early Spring of the year 2000.WT/COMTD/W/58
Page 6
WORLD TRADE ORGANIZATION
ORGANISATION MONDIALE DU COMMERCE
ORGANIZACIÓN MUNDIAL DEL COMERCIO
15 February 1999Country Request and
Evaluation Form
MISSION CODE COMTD Reference WTO Focal Point 
Country Focal Point: lead
ministry/agency
Activity dates Duration (days)
Mode of delivery Initial request/
reference
Activities to be
undertaken/
Subject(s)covered
Participating
Countries
WTO/other
lead agency
Specific
objective(s)of themission
Estimated cost of the activityFunding
Division(s) Nb of staffWT/COMTD/W/58
Page 7
Country Request and
Evaluation Form
15 February 1999__________________________________________________________________________________________
Activities undertaken:
assessment by WTO
Immediate
assessment bybeneficiary (lead
agency)
Assessment of
mission after 3-6months by
beneficiary (lead
agency)
Nature of follow-up
expected/requested
ParticipantWT/COMTD/W/58
Page 8
WORLD TRADE ORGANIZATION
ORGANISATION MONDIALE DU COMMERCE
ORGANIZACIÓN MUNDIAL DEL COMERCIO
Centre William Rappard  Rue de Lausanne 154  Case postale  CH – 1211  Genève 21
Téléphone:  (+41 22) 739 51 11  Fax:  (+41 22)  731 4206
Internet:  http://www.wto.orgEvaluation /Feed-Back
Form
Filling out this questionnaire should not take more than 5 minutes. It
would help us to further improve and adapt our forthcoming technicalcooperation.  Thank you in ad vance for your collaboration.
1.Title of event: __________________________________________________
2. Mode of delivery:
National seminar
Technical mission
Regional seminar Conference/Symposium
Workshop Short training course
3. Country/location where event took place: __________________________
4. Date(s):_____________________
5. Please indicate your area of activity:
 Government  Business
 Other:_______________________________________WT/COMTD/W/58
Page 9
6(a) What were your main objectives for participating in this event?
6(b) Where they achieved? [Yes = largely; No = not entirely]                Yes      No
 Obtain general information a bout the multilateral trading system
 Obtain specific information a bout one/several WTO Agreements
--which ones: _____________________________________________
           _____________________________________________
 Address implementation i ssues of WTO Agreements
 Obtain information on 'new' policy issues and 'built-in' agenda
 Use the information to train other persons
 Improve working efficiency in specific areas/
 Other_________________________________________________
   _________________________________________________
7. How would you rate the ev ent you have just attended?
Content (substance)  Excellent  Good  Satisfactory  Poor
Structure (topics)  Excellent  Good  Satisfactory  Poor
Presentations (style)  Excellent  Good  Satisfactory  Poor
Doc. Provided  Excellent  Good  Satisfactory  Poor
Logistics (travel arrang.,accomm.,
meeting room)  Excellent  Good  Satisfactory  Poor
Duration  Sufficient  Insufficient
Time per topic  Sufficient  Insufficient
Other____________  Excellent  Good  Satisfactory  PoorWT/COMTD/W/58
Page 10
Please give a short explanation:
______________________________________________________________________________________________________________________________________________________________________________________________________
___________________________________________________________________________
8. Which part(s) of the programme and/or  topic(s) did you find most useful?
a.________________________ b. _______________________________
c.________________________ d. _______________________________Please give a short explanation__________________________________________________________________________________________________________________________________________
9. Which areas or aspects, in your view, were not adequately addressed?
a._________________________ b. ______________________________
c. ________________________ d. ______________________________What was lacking?
_____________________________________________________________________
_____________________________________________________________________
10. Which documentation or handouts were the most interesting for your work?
a. ________________________ b. ______________________________
c. ________________________ d.. _____________________________Please explain why?__________________________________________________________________________________________________________________________________________WT/COMTD/W/58
Page 11
11. How will you use the informatio n provided during these sessions?
 Disseminate information to:  Colleagues
 Employees
 Business people
 Other
 Provide input into refinement of my  country's policy position on WTO issues.
________________________________________________________________________________________________________________________________
 Improve own working performance. How?
________________________________________________________________
________________________________________________________________
 Other ________________________________________________________
12. Does this event require a follow-up        Yes   No 
If yes, please explain:
_____________________________________________________________________________________________________________________________________________________________________________________________
__________