WORLD TRADE
ORGANIZATIONWT/COMTD/W/111
28 March 2003
(03-1792)
Committee on Trade and Development
TECHNICAL COOPERATION AUDIT
REPORT FOR 2002
Note by the Secretariat
BACKGROUND
1. This is the second annual report of the Technical Cooperation Audit (TCA) on evaluation
activities in WTO.  In accordance with established practice, it is submitted to the Committee on Trade
and Development in the first quarter of the year.  The report covers technical assistance activities thattook place during the preceding year, in this case 1 January to 31 December 2002.  More frequent
qualitative reporting on a more limited number of evaluations of individual activities, such as those
carried out by TCA, would not provide sufficient critical mass to be meaningful.
2. In the period under review the Technical Cooperation Division, the Training Institute and the
Operational Divisions continued to function as the main providers of technical assistance.  The annualreport does not constitute an evaluation of the entire Technical Assistance (TA) programme of WTO
implemented in 2002.  For example, TCA, constrained by the current level of resources, continues to
rely on the Training Institute's own evaluation of the long-term trade policy courses which it runs.
Neither does TCA evaluate large-scale projects carried out in collaboration with other agencies, e.g.
Integrated Framework, JITAP, which are evaluated by external evaluators.
3. The new system for monitoring and evaluating (M/E) TA projects, introduced in April 2002
on an experimental basis for two years, became operational and started generating an increasingamount of qualitative and quantitative information, which are all used in this report.
4. The findings, conclusions, recommendations, and lessons learned included in the annual
report are based on on-site observation of selected projects by TCA, the review of a large number of
back-to-office reports (BTOR) prepared by the providers of technical assistance activity, as well as
interviews with TA participants and delegates on-site or at headquarters.  In 2002 TCA continued toreceive financial support for evaluation work from the UK Department for International Development
(DFID).
5. The annual report is meant to inform Members, and to enable senior management and
providers of technical assistance to take into account the conclusions, lessons learned, and
recommendations in future policy and operational decisions.
METHODOLOGY
6. Under the approved methodology, evaluation is  a shared responsibility between the TA
providers and TCA.  Because of the large number of individual activities and limited resources for
evaluation, the most frequently applied form of evaluation is self-evaluation by the TA providers,
which engages their responsibility for achieving the desired results, effects and impact.  These internalWT/COMTD/W/111
Page 2
evaluations benefit from the in-depth knowledge of the TA provider of his/her project and facilitate
immediate action for further improvements.  Furthermore, recourse to self-evaluation permits much
broader coverage of TA projects than would otherwise be possible and the accumulated findings and
conclusions, recommendations and lessons learned derived from such evaluations help to enhance
quality from within, at no additional cost.  The results of the self-evaluation of a TA project areincluded in BTORs.  TCA represents, through its limited activities, a modest additional, internally
independent, evaluation capacity.
7. For practical and methodological reasons only those TA projects for which WTO assumes
responsibility for the full project cycle, i.e. preparation, design, implementation and evaluation, are
subject to evaluation.  TA activities in which WTO is invited by other agencies to deliver a
component, or which take the form of a regional symposium or conference with hundreds of
participants, do not easily lend themselves to evaluation.  In such cases it would be extremely difficultto identify any effects and impact which are exclusively attributable to the WTO contribution.  Hence
not all the TA activities included in the annual TA plan, or carried out on an ad hoc basis, are subject
to evaluation.
8. Established procedures make it a requirement that TCA gets a copy of every BTOR prepared
by TA providers.  These reports constitute a valuable input.  In 2002, the number of reports receivedin TCA was lower than the total of TA activities for which evaluation was due.  There are a number of
reasons for this:  the new monitoring and evaluation procedures were introduced in April 2002 and its
reporting tools were not immediately known to and a pplied by all the TA providers;  evaluation as a
management function integrated into the delivery of TA activities may not yet be generally accepted
and exercised by the TA providers;  evaluation culture is only just beginning to be accepted as anintegral part of the Secretariat's work.  It is encouraging, however, that a great number of reports is
received in TCA.  This is a sign that monitoring, evaluation and reporting have started generating
results.  The reports reviewed by TCA provided  a solid basis for drawing conclusions and for
suggesting recommendations to improve TA.
9. The structure of this report reflects the normal project cycle, i.e. the planning and formulation
of technical assistance, implementation, performance and evaluation.  The report focuses on the core
elements of evaluation:  efficiency
1, effectiveness2, relevance3, and sustainability.4  T h e
recommendations and lessons learned can be applied in the next phase of planning, organising and
implementing technical assistance.  Concerning the lessons learned, it should be noted that
organizational learning is an on-going but slow process;  therefore it is not surprising that some of the
lessons identified are similar to those derived also from the last year's evaluation exercise.  In any
case, it is worth repeating lessons learned so that the message has more chance of getting through.The application of lessons learned is primarily the responsibility of TA providers and their
organizational units.  Within its own means, TCA continues to disseminate, and encourage the
application of lessons learned.
TECHNICAL ASSISTANCE AS A MEANS OF ACTION IN WTO
10. The Doha Development Agenda (DDA) clearly reflects WTO Members' support for technical
assistance as an instrument for the integration of developing and least-developed countries into the
multilateral trading system.  The DDA has been the guiding force in the organization and delivery of
WTO technical assistance in 2002, and will continue to have an important impact in the current and
subsequent years.  Members have manifested a growing interest in and support for each phase of the
                                                     
1 Efficiency: did the results justify the costs?
2 Effectiveness: has the evaluated TA achieved its objectives?
3 Relevance: does the TA delivered correspond to the needs of its recipients/beneficiaries?
4 Sustainability: can the results of the TA be sustained without further external assistance?WT/COMTD/W/111
Page 3
project cycle.  All planned projects were specifically linked to one or several issues in the DDA.  The
evaluations of the individual TA activities were required to highlight those paragraphs in the DDA to
which the TA was meant to relate.
11. The TA plan for 2002 was finalized and adopted in March.  Subsequently, its implementation
gradually accelerated.  According to information that is being made available elsewhere to Members,481 TA activities were carried out in 2002, representing the highest volume of technical assistance
delivered in any year.
12. With regard to the planning process, it has to be recognized that it is difficult to strike a
balance between needs and the limited financial and staff resources to meet them.  This point has been
made often enough in presenting the plan.  Furthermore, the time and effort put into the prioritisation
of needs, the identification of beneficiaries, the lo cation, the counterparts, dates, etc. both at country
and at regional levels, should also be considered;  they represent a significant cost.  There is a largeelement of uncertainty involved, as shown by the fact that the approved TA plan had listed 228 "to be
confirmed" (TBC) activities either in terms of dates or venue, and a number of activities for which the
precision of timing had to remain within a range of  three months.  At the time of the preparation and
approval of the TA Plan, neither the Secretariat nor the Members were in a position to be more
precise, a clear shortcoming of the planning process.  Hence some of the projects planned in thecourse of 2002 had to be abandoned, postponed, reformulated and new needs came up that also had to
be addressed.  Members have received a report on implementation separately.   The proportion of the
ad hoc  activities is quite high (27 per cent or 130 of the 
[476] activities implemented).  All the ad hoc
TA activities were registered in the database, but analysis is made difficult because many of them are
not subject to BTORs for the reasons explained above.  The evidence does suggest, however, that
such activities are the result of a number of factors, not all of which should systematically result in
their condemnation. For example, agencies with which WTO cooperates may have a different
planning time frame, and their conferences and workshops, which require a WTO input, may be
organized in such a way that they could not be foreseen in WTO Plan.  The WTO would probably be
criticized, and justifiably, if it did not respond positively in such cases.  Non-participation in suchevents would have weakened the position of WTO by not capitalizing on its comparative advantage,
and leaving the field to others.  Clearly on the negative side, however, is the existence of activities
which cannot really qualify as technical assistance at all, and need to be excised.
13. The implementation of the TA plan and meeting ad hoc  needs put great pressure on all staff
involved in the provision of technical assistance and training.  Particularly demanding were the
planning, organization and delivery of the additional three-month trade policy courses in Geneva, and
those in the African region, Nairobi and Casablanca.  Almost all units of the Secretariat were involvedin one or other aspects of  technical assistance.  The budget estimates each year give an indication of
the proportion of time spent on this.   Some minor organizational and coordination difficulties can be
attributed to short deadlines and stretched resources.  But all this has to be seen in the light of the
delivery of more technical assistance than ever in the history of WTO.  What would certainly be
helpful in this regard is better coordination between the operational divisions, TCD and the TrainingInstitute.  The establishment by the Secretariat of a set of guidelines regarding the priority-setting and
the selection of TA activities for inclusion in the TA Plan would also improve transparency and
accountability.
TECHNICAL ASSISTANCE COORDINATION
14. The Technical Assistance Management Committee (TAMC)  has further developed its role
as a forum for coordinating TA efforts and discussing relevant policy and practical issues.  It had ten
meetings in 2002, as opposed to six in 2001, the year when it was established.  The meetings have
benefited from the participation of TA focal points from the operational divisions.  TCA believes that
the TAMC can become a very useful forum for sharing experience among TA providers in TCD, inWT/COMTD/W/111
Page 4
the Training Institute and in the operational divisions, both for planning and for implementation.  It
also provides a means of propagating and reinforcing lessons learned.
15. Not all issues related to TA, however, lend themselves to discussion in full sessions of the
TAMC.  Some issues of a policy or strategic nature might be better dealt with by the core units that
are directly involved in these issues, before they are considered by the full membership.
TECHNICAL COOPERATION AUDIT (TCA)
16. A new system for the design, monitoring, evaluation and reporting of technical assistance was
introduced during the period under review.  At the beginning of April, the system was introduced on
an experimental basis for two years.  In developing and implementing the system, TCA received
continuous support from the senior management of the Secretariat.  An evaluation briefing kit was
developed and distributed to help TA providers in the preparation, implementation and evaluation of
their projects.  The required reporting formats were simplified and made more evaluative.  TCA madeits advisory services available to TA providers for each phase of the project cycle.  In order to make
the evaluations of the hundreds of TA activities a useful source of information, a database was
developed, which provides for storing and retrieving relevant information on TA, and facilitates the
analysis of all BTORs.  TCA has fed back its observations and views also to TA providers of
particular projects.  Based on the analysis of the feedback received from the users of the M/E system,TCA will soon review some of its forms and procedures used to make them more adapted to the
specificities of WTO TA.
17. TCA carried out on-site evaluations of 15 TA activities (See details in Annex I).  Most of
these activities were regional.  When TCA has an opportunity to concentrate in the same way on
national activities, it will then be possible to make an analysis of the relative strengths and weaknessesof the two types of activity.  In addition to its mandatory activities, TCA was requested by the
Director-General to conduct an evaluation of the first regional trade policy courses delivered in
Casablanca and Nairobi.  The relevant report was submitted to the Committee in October 2002.
5
THE PERFORMANCE OF TECHNICAL ASSISTANCE
The Big Picture
18. TCA reviewed a random sample of evaluation reports, and its findings are based on that
review.  In the surveyed reports, 108 staff members worked as resource persons.  The most common
capacity-building activities were delivered in the form of seminars and workshops.  Somewhat more
national than regional and sub-regional activities were included.  Regarding sectoral coverage,
Services appeared to be the most "popular" subject (29.52%) followed by SPS/TBT (18%);  Dispute
Settlement (13.3%) and Agricultural issues (11.4%).  In 75 per cent of the cases, the duration of the
TA was between 2 to 3 days.  The projects benefited 7265 persons (5985 men and 1280 women).Fifty-two per cent of the beneficiaries are employed in ministries of trade, commerce or finance and
21 per cent work in foreign ministries.  The rest are employed in other ministries, e.g. agriculture, or
institutions, e.g. customs, etc.  The representation of the other economic sectors remains insignificant,
except for some national activities, despite general recognition that the involvement of the private
sector and civil society representatives is desirable.
19. The overall quality of the assistance delivered has remained very high.  The resource persons
in general demonstrated an impressive command of their subjects, and a thorough professionalism in
delivery.  Teaching skills of TA providers have certainly been enhanced through participation in the
training-of-trainers courses organized in 2002 by the Training Institute.  The TA activities have
                                                     
5 Regional Trade Policy Courses in Africa (WT/COMTD/W/107 – 31 October 2002)WT/COMTD/W/111
Page 5
reflected more conscious efforts in improving methodology, presentations and also more attention has
been paid to applying participative methods, alth ough a common comment in feedback is that still
more can be done.
20. The results typically generated by TA projects provide participants with a good understanding
of, and practical exposure to, rights and obligations under the different WTO Agreements, improvetheir knowledge, provide them with up-to-date information on the negotiations, and in the case of
regional and sub-regional technical assistance, help them to network with trade experts from other
countries.  In only a very few instances did beneficiaries question the utility of the activity in which
they participated.  On the contrary, they were convinced on the whole that their participation had
significantly enhanced capacity, both for them personally and for their governments.
21. In TCA's view, however, the very short duration of TA activities and the often great number
of participants (sometimes 40 to 100 persons) allow more for dissemination of information,sensitization or awareness creation rather than real skill development and capacity-building.  It is the
eternal question of quantity vs quality.  On the one hand, thousands of staff of ministries of Trade,
Commerce, Finance, Agriculture, etc., have acquired information and knowledge that can effectively
be put into use in the current trade negotiations.  On the other hand, the short duration of activities and
the great number of participants in some individual TA activities is more suited to a policy the mainpurpose of which is to reach the maximum number of persons possible, without due account being
taken of the depth of the training.  It is possible to conceive of capacity-building through a series of
short training activities oriented towards the same target groups, but in that case they have to be seen
as building blocks, and designed as a systematic and coherent capacity-building programme.  From
that angle the three-month trade policy courses (TPCs), both in Geneva and in the regions, as well aslarge-scale TA programmes such as JITAP, have greater potential and an explicit objective of
bringing about substantial increases in human and institutional capacities.  The activities of the TA
Plan in 2002 were not planned, designed or implemented as part of systematically and coherently
developed multi-year technical cooperation project or programme for a technical sector, a country or a
region or sub-region.  The plan lives and dies from one year to the next.  Therefore, the cumulativebenefits of the current TAs for the beneficiaries are more the result of coincidence than systematically
thought out efforts. Sending a questionnaire to Members each year may be a necessary way of
proceeding with needs-identification, but experience shows that the responses are far from complete,
and only partially helpful in determining an appropriate plan for TA. Furthermore, if the planning
process continues to respond to annual and quite often short-term needs, there is a danger that TA
resources will be spread too thin and the outcome not sustainable.
22. It is important that the Secretariat should continue to help Members formulate their TA
demands. In the course of committee meetings and consultations, which are regularly held in TCD
and operational Divisions with donors and potential beneficiary countries or groups, the Secretariat
has an opportunity to shape the programme of activities.  The analyses contained in TPR reports can
be helpful in assisting the Members and Secretariat to identify suitable issues for TA.  Inevitably,
there will be situations where, e.g. decisions taken by committees that TA of a given kind should beprovided, or an unforeseen request of a donor, a beneficiary country or a cooperating agency will have
to be addressed.  TCA's experience suggests that Members, once they have got beyond the generally
held view that they should benefit from far more TA  than they actually get, are on the whole receptive
to discussing with the Secretariat priority areas and other capacity-building issues based on the
Secretariat's advice.  Operational divisions have a particular role to play in this regard because of theirmore in-depth knowledge of each country's situat ion with respect to each Agreement. Furthermore,
the technical areas, such as agriculture, services , competition policy, dispute settlement, investment,
rules, market access, etc lend themselves to the development of technical assistance projects of a
larger size and longer duration in the form of c ountry projects or programmes. The latter may cover
several technical issues defined through a combination of needs assessments, diagnostic studies or theWT/COMTD/W/111
Page 6
above-described consultation process. More generally, if "demand" is to be accurately assessed and
usefully channelled, canvassing of requirements, as defined by Members, is particularly important.
DESIGN OF TA ACTIVITIES
23. No specific strategy appears to govern individual TA activities in terms of countries/regions
or in terms of sectors.  Ideally, all TA activities should be designed with a view to addressing a
capacity deficit identified in concert between the Secretariat and the recipient country(ies).  The
change that has to be brought about to meet the needs and solve the problems should be set out clearly
as an objective, should be supported by indicators of achievement, and from this the outputs to be
produced and the activities carried out should logically follow.  Indeed the objectives should reflect
what the participants are expected to do or change as a result of the TA; a TA objective is not what the
TA provider will do in delivering the project.  A good design makes the project transparent, helpsaccountability and provides a solid guidance in implementation and evaluation.
24. Although the current evaluation procedures provide guidance in the design of TA activities,
only 28 per cent of the evaluations reviewed revealed objectives which by and large correspond to the
above definition.  Sixty-five per cent of the reports examined contained project activities set out as an
objective.  For instance: " To explain the structure of GATS..." "Present detailed information..."; or
"Inform the government..." .  This is what the TA provider does, rather than the expression of an end-
of-project situation showing how the beneficiaries of the TA will have enhanced their capacity.
Awareness-creation, which is the aim of a number of WTO activities, is important in some issues,
because without it, nothing else will follow.  Hence there is nothing wrong if some WTO projects also
aim at that.  It is preferable, however, to be specific about the actions that should flow from theimproved awareness.  The difficulty, really, lies in evaluating the extent of awareness and to prove
that it can be attributed to the specific WTO training.  Some projects did not set out any objective.
This reflects what appears to be a fairly common approach, namely: "Just do it...".  Another weakness
in the design of the TAs is that the majority of TA providers have not made any attempt to set out
indicators of achievement in the design phase that could be used at the end of the activity in verifyingthe effects and impact of the project.
THE FOUR PILLARS OF EVALUATION
25. The following basic evaluation components have been looked into in this report: efficiency,
effectiveness, relevance and sustainability. These concerns are interlinked. Leaving out any one or
several of them will weaken the outcome and the validity of the evaluation. For example, an
evaluation that looks into a project only if its costs justify the outputs will procure cost-effective
findings, but will not provide any information on the quality of the outputs (presentations, lectures,discussions, documentation, new ideas, approaches, etc), or on the use of these outputs by the officials
working in ministries related to WTO. And by the same token, an evaluation that examines
exclusively if the objective has been achieved, will not provide information on the costs needed to
achieve the objective. Therefore, a minimum expectation in any WTO TA activity is, "to do the right
thing right". In other words, a TA activity should be both efficient and effective. Furthermore, a TA
activity may be effective and efficient, but if it does not meet the needs of the recipient organisation or
the results are not sustainable, there will be no outcome (relevance and sustainability, respectively).
Efficiency
26. Of the evaluations reviewed, 73 per cent provided evidence of having been implemented in an
efficient manner (33 per cent very efficient, 40 per cent satisfactory). This involved the activeparticipation of people with a high motivation to learn; the application of participatory approaches;
the adaptation of the examples and exercises to the local conditions; holding workshops back-to-back,
with corresponding savings in travel costs;   coop eration with other agencies or host organisations thatWT/COMTD/W/111
Page 7
provide logistical support at no additional cost to  WTO; the distribution of course documentation on
CD-ROMs, instead of mailing kilos of documents to the project site; and the advance despatch of the
TA programme to the participants.
27. From the point of view of capacity-building the long-term training programmes, e.g. the
regional trade policy courses, are more efficient.  Nevertheless, from the point of view of thebeneficiary countries, there is a question of short-term efficiency related to the employer having to
release a staff member for such a long duration.  In the long-term however, the official thus trained
may return the investment through more effective work.
28. Such signs of efficiency as there are can be traced back to the planning process, but also
appear at the implementation stage.  Individual TA providers carry out the preparation of their
training materials and aids with minimum interaction with colleagues from other divisions.  It can
happen that three or four versions of the same presentation are developed in an independent manner.TCA's observations show that, for example, the level of Powerpoint skills varies, and so does the
capacity to use computers and projectors, and other hi-tech aids.  TCA believes that efficiency in
delivering TA could be further increased if there were standardized training packages (presentations,
documents, briefing papers, CD's, etc.) developed for the substantive issues for different groups of
targeted beneficiaries, including tests and cases which are taken from the country or the region of theparticipants. The current individual and discrete preparation efforts could be channelled into an
organised "software" development process ensuring that WTO relays the same message anywhere in
the world.  And TCA repeats the comment made in respect of 2001:  hi-tech is not synonymous with
most effective, and carries with it the risk of break-down, incompatibility, etc.
29. Some efficiency considerations can be laid at the door of the beneficiaries.  For example,
participants expressed frustration in some instances that they had a full workload during the course,
but were expected to take part in "business-as-usual" overlapping with the course.  This is not a
formula for success.  Activities in which the participants do not speak the language of the course and
local interpretation is not adequate or available have hardly any chance of achieving objectives.  The
organization should review its policy towards providing interpretation when it is clearly needed.
30. Logistical, organisational and administrative backup services are crucial to successful
implementation of TA. The review of the evaluation reports and TCA's own observations have
revealed that the overwhelming majority of TA projects (77 per cent) have benefited from efficient
backup services.  Twenty-three per cent of them, however, reported difficulties for the resource
persons in striking a balance between administrative obligations and substantive tasks (presentations,running discussions, chairing the activity etc).
31. The support of an administrative assistant can be critical to the success of a regional TA
project, especially those where there is only one resource person from WTO.  This is particularly the
case where the local counterpart organization is not in a position to provide such services.  Inevitableproblems in connection with hotel accommodation, per diems, travel, reproduction of documents on
the spot, communication with headquarters, etc. are more efficiently dealt with by an administrative
officer so that the TA provider does not have to be away from the classroom in order to deal with
them.  Absence of such back-up detracts from the quality of the training, and casts the WTO in a bad
light.
32. A number of small things might also help improve efficiency.  These include:
(a) giving participants the opportunity to introduce themselves and indicate what they
expect from the TA;
(b) providing name tags;
(c) circulating a list of participants at the outset of the TAWT/COMTD/W/111
Page 8
They all go towards encouraging interaction and networking among participants from
different countries (they are particularly relevant for regional activities, but not without interest for
others, as well).
33. In order to enhance efficiency even more, it would be desirable to follow-up the training-of-
trainers courses for WTO staff members, which were run by the Training Institute, with similar
training in project design, monitoring and evaluation, all of which are components in project cycle
management.
Effectiveness
34. It is legitimate to expect that all TA activities achieve the objectives(s) that they set out.  It is
a challenge to prove that the objectives were actually achieved.  But in order for this to happen, twothings must be present:  clearly stated objectives and collection and verification of data on indicators
of achievement.  These latter need to be set out prior to the TA taking place, which in practice rarely
happens.  Only six per cent of the BTORs reviewed included reference to pre-set indicators.  Other
indicators used were: active participation (36%); feedback from participants (38%); feedback from
participants' employers (9.5%).  In almost 20 per cent of the reports there were no such indicators.  Ithas been found that all too often no specific objectives were stated at the beginning of the TA
activities, a situation that was already a matter for concern in 2001.
35. From the analysis that is possible, it emerges that in 70 per cent of cases there was full
attainment of objectives and in 18 per cent it was partial.  This is a laudable result.  Participants' stated
satisfaction with WTO capacity-building activities is important but there are other more objectivemeans, e.g. tests, case studies, individual presentations, etc. to verify if their knowledge and skills
have really improved as a result of the TA concerned.  Only a few resource persons used tests to
verify whether there was improved knowledge or strengthened capacities of the participants at the end
of the TA.  In some courses, the quality of papers written by the participants was found to be a useful
tool to measure project effects, e.g. regional trade policy courses, or short trade policy coursesorganized in the Joint Vienna Institute in Vienna.  This practice should be encouraged and used on a
much wider basis, although it is recognized that there are limitations, especially in respect of seminars
of two-three days.
36. The usual explanation for not doing more is lack of time both prior to and during the seminar
or workshop.  Another reason is the great number of participants, which would mean taking too much
time to measure achievements.  Further work obviously needs to be done to sensitize TA providers to
the importance of defining reachable objectives and indicators of achievement, in order to measure theperformance of TA activities.  For example, end-of-workshop verbal evaluation sessions are not yet a
standard feature of all TA activities.  TCA will have to do further work to refine what is meant by
indicators in the current guidelines and formats, and offer services to help the providers to improve
the design and evaluation of their activities.
Relevance
37. To what extent has WTO TA been relevant to the needs of the Members, the recipient
organizations and the participants?  WTO TA projects are closely linked to the Doha Development
Agenda, and so by definition must be considered relevant, at least at the political level.  In the same
vein as was reported in 2001, however, it has to  be acknowledged that not even the best and
continuous WTO TA for capacity-building can on its own compensate for other deficits:  marketable
products, social and economic conditions of development, etc., which are needed for effective
participation in the multilateral trading system, and for deriving some of the benefits of globalization.
As far as the WTO's TA remit is concerned, and with regard to the expectations of Members and
participants, it can be said that there is a very good correlation.  The only potential concern is asWT/COMTD/W/111
Page 9
mentioned above, whether all aspects have been covered to the required depth, and whether the
varying circumstances of each country at a regional activity have been reflected.  That is largely
defined by what the participants themselves think, and on the whole the indications are that they find
the substance of the activities directly relevant to their work. There may be quibbles about emphasis,
but these do not seriously undermine the value of the activity itself. Most participants indicate theirwillingness to apply the newly acquired knowledge and skills in the current trade negotiations, as well
as pass them onto their colleagues.  It should be reiterated, however, that finding out the participants'
individual expectations prior to the TA and having them reflected in the programme or giving them an
opportunity to highlight their expectations at the beginning of the TA, helps to improve its relevance.
38. Since the issue is one on which there has been some divergence of views, it is perhaps worth
mentioning that those ad hoc  activities which can be qualified as really being technical assistance
would appear to be no less relevant than the planned ones.  On the contrary, they often correspond toclearly identified and current needs that a Member has.
Sustainability
39. This is no doubt the most difficult aspect of the performance of TA to measure.  There is a
very clear link to the outcome and long-term impact of a given activity, which depend to a very large
extent on external factors, over which WTO does not have control.  The Secretariat may set outcertain conditions that are considered necessary for results to be sustained, but it is the beneficiaries
that determine who will participate in WTO TA and what happens after the activity.  The current
evaluation system does not provide evidence of the sustainability of the results since it stops where
the TA ends.  Indeed, sustainability can only be verified through ex-post evaluations taking place one-
two years after the completion of the TA.  Such evaluations may generate valuable information, butcannot be carried out without additional resources.  In  any case, there is little point in measuring the
sustainability of individual activities, which are too short and specialized to have much long-term
impact;  that is not even their aim normally.  Assessing whether the collective TA activities in which a
country has participated have had a lasting impact is  more interesting.  So is assessing the impact of
the TPCs, where there are some indications that they are successful in that regard.
40. All that is by way of introduction.  What can be said more concretely about this question is
that resource persons should, already at the design stage, think about and provide for those
components which are known from experience to have  a chance of being sustainable, e.g. reference
centres, suitably drafted legislation, databases, re porting procedures, documentation, and networking
arrangements.  The application of participatory methods in the activity also facilitates sustainability.
41. It has been found that in almost all TA projects there are sustainable components.  The only
exceptions are activities concerned exclusively with projects of information dissemination.  However,if the right participants are not selected, if the me ssage from the TA provider(s) does not pass, if the
participants, with their improved knowledge and skills, leave their jobs soon after the TA is completed
or do not get the necessary support and opportunity to apply their knowledge, the results of the TA
will soon fade away. Participants should have backgrounds relevant to the activity. Training people in
WTO matters whose work has in fact nothing to do with WTO, or technical skills to people who do
not need them, is not a good use of their time or WTO's resources. In light of the costs involved, the
distances travelled, and the impact of the resource person's absence on work in Geneva, there is a need
to reflect on how participants might be better "targeted".  It would seem reasonable that at least for
regional projects, the WTO could seek to be advised who it is intended to send, since there is a
substantial financial investment involved. All these examples show that the sustainability of theresults is at the very least a shared responsibility, and lies to a large extent in the hands of the
recipients.WT/COMTD/W/111
Page 10
42. There is some scepticism that the participants will, as they claim, be able to use the newly
acquired knowledge and improved capacity in their jobs on a day-to-day basis, and share that
knowledge and capacity with colleagues who have no t attended the activity. This can be influenced by
a number of factors, such as institutional background, and by the knowledge-base which the
participants bring with them to the TA activity.  The lower the knowledge-base of the participant, the
less likely that the acquired knowledge and capacity will have been effectively mobilized in the trade
negotiations. Furthermore, the more uneven is the level of experience and the knowledge and skill
composition of the group of the participants, the higher the danger that the TA is not generating the
same benefits for all of them. To remedy this situation, and in the absence of procedures for ensuring
the homogeneity of the participants, the importance of well-organised hand-out material cannot be
overemphasized. That material, either on CD-ROMS or in hard-copy, should serve as a permanent
reminder and allow participants to refresh their knowledge and skills as required, and consolidatewhat has been learned at the activity. Quick and reliable access to databases and internet sites also
facilitates sustainability.
43. Networking is also worth mentioning. Assuming that participants do remain in the trade
business, the connections that they can establish at  TA activities, be it at the national level or the
regional level, will also contribute positively to sust ainability. There is an additional advantage to this
in terms of bringing together representatives of government, the private sector and civil society, and in
some instances, representatives of different branches of the same government who would not
normally come into contact with each other.  It is generally recognised that the more interaction is
fostered at the national level, the greater the likelihood that the results of TA will be sustained.
44. Finally a most important aspect of evaluating sustainability is the evidence of national
commitment to maintain and further develop the re sults of WTO TA.  However, this is not often
available and Members may wish to consider ways in which to foster that commitment.
REPORTING AND FOLLOW-UP
45. The evaluation function is enabling WTO to meet its substantive accountability requirements.
Evaluations provide the Members and the Secretariat with useful operational and strategicmanagement information to address failure policies and strategies.  A basic analytical tool to transmit
such information is the BTOR.  TCA has noted with satisfaction that the overwhelming majority of
BTORs follow the established format. However, the evaluative substance can be further improved.
Follow-up actions are rarely formulated with precision. TCA has now accumulated enough experience
with the application of the established formats, e.g. Participant's Evaluation Form (PEF) and BTOR
procedures to be in a position to review and make appropriate adjustments.  The evaluation function
would be strengthened by an improved evaluation feedback mechanism, such as BTORs that put moreemphasis on lessons learned.
46. In the view of TCA, the concept of a Plan represents both a potential and a limitation.  It
reflects priority areas of concern and constitutes a kind of indicative framework for implementing
capacity-building activities.  The closer the impl ementation is to the plan, the greater is its
transparency;  Members, both beneficiaries and donors, can more easily follow how the Secretariat isresponding to their requirements.  However, even the best implemented plan does not provide a
guarantee that the impact of the activities will be what was hoped for, namely enhanced capacity of
trade negotiators to participate in the WTO system.  There are many other factors which will have an
influence on this, not least how effectively the Member receiving technical assistance can put into use
what was imparted and learned through the TA.  It is suggested that the Technical CooperationDivision (TCD) annual report be made evaluative by inclusion of information on effects and impact of
TA activities.WT/COMTD/W/111
Page 11
LESSONS LEARNED
47. Some conclusions have already been drawn in earlier sections of the report.  This section
attempts to draw some more general conclusions from the evaluations carried out in 2002 and put
forward some ideas about how to achieve further improvements.  While it is ultimately for Members
to judge, it can be said that the Secretariat has fulfilled the commitment  to provide technical
assistance and capacity-building, both in terms of volume and in terms of quality.  Technical
assistance and capacity-building activities were considered as a priority, and a real effort was made to
ensure that they were effective, well-targeted, an d coordinated with other agencies to get the best
results.
48. WTO TA activities could more effectively contribute to building lasting capacities  if they
were planned and designed on the basis of a thorough assessment of the Members' needs and
problems. Regional and sub-regional activities should primarily aim at developing networking amongthe participants and facilitating an exchange of views regarding negotiating positions, and national
policies and practices. National activities should focus on systematic training in the form of both
generating new skills and knowledge, and institutional development with a view to creating a
supportive environment within the Members.  This process could be facilitated if WTO expertise were
available on an ongoing basis from within the regions, for example from staff actually posted awayfrom headquarters.  This is an approach used to very good effect by other international organizations
engaged in development and capacity-building act ivities.  It is usually also cost-effective.
49. The emphasis in the prevailing approach to TA in the WTO is on quantity.  This is perhaps
not surprising given the demand-driven notion.  But w ithin this there is a need to focus also on the
quality  of the capacity-building.  Squeezing complex i ssues into 2-3 days when they need five, not
providing an administrative assistant for regional seminars, reinventing presentations with each
resource person who deals with a given subject, a nd omitting to make reference to local/regional
issues, all detract from quality.  On the other hand, the presentation skills training undergone by
members of the Secretariat in 2002 are clearly bearing fruit in terms of quality.  There are no grounds,
however, for complacency.
50. While it was perhaps inevitable that the implementation of the DDA would oblige the
Secretariat to identify needs quickly through a great number of TA activities in 2002, thought could
be given to the possibility of developing some new approaches  to capacity-building, which go
beyond responding to the immediate needs of the DDA.  Consideration might be given to the
developing of a few large-scale, multi-year technical assistance projects and programmes, similar to
JITAP.  Such TA programmes, organized around technical sectors, countries or regions, would
provide the backbone of WTO's technical cooperation.  The Committee on Trade and Developmentwould then perhaps have less need to be invo lved in managing the detail surrounding hundreds of
small-scale activities, and be able to concentrate more on providing guidance as to the broad strategic
options to be followed.  Needless to say, the details of such large-scale technical cooperation projects
and programmes should be developed with the full participation of the beneficiary country(ies) and
donors.  This is a way of getting more quality mileage out of TA.
51. The monitoring and evaluation  procedures introduced last year have started generating
valuable data on the effects and impact of TA.  Further improvements can be achieved if the design of
TA projects follows the suggested guidelines, with clear, realistic and achievable objectives and
related indicators of achievement.  This means that not only will high quality TA be delivered, but it
will be more results and outcome oriented.WT/COMTD/W/111
Page 12
52. It is important that efforts to improve the efficiency  of TA activities continue unabated.  The
development of standard training packages, and improved logistical and administrative backup
services are among the measures that can contribute to this.  But above all, enhanced cooperation and
coordination between TCD, the operational divisions and the Training Institute can help ensure the
most effective use of the human resources and avoid possible duplications or overlaps.  The TAMChas played a useful role as a forum for exchanging views on different aspects of TA.WT/COMTD/W/111
Page 13
ANNEX I
MISSIONS CARRIED OUT BY TCA IN 2002
Harare Reference Centre, 14-18 January
St. Lucia TBT Regional Seminar, 18-21 FebruaryTallinn SPS Sub-Regional Seminar, 5-7 MarchDoha TRIPs Regional Seminar, 28-30 AprilVienna Inter-agency Working Group on Evaluation, 19-20 JunePretoria DSU Regional Seminar, 24-27 JuneWindhoek Trade and Environment Regional Seminar, 8-11 JulyCasablanca Regional Trade Policy Course, 24-27 SeptemberNairobi Regional Trade Policy Course, 25 September – 4 OctoberNairobi Regional Trade Policy Course, 30 September – 4 OctoberVienna Applied Economics Seminar, 14-18 October
Mauritius Competition Policy Regional Seminar, 13-15 November
St. Lucia IDB Regional Seminar, 24 November – 1 DecemberSuva Toolkit for Negotiators Regional Seminar, 10-13 DecemberKuala Lumpur Toolkit for Negotiators Regional Seminar, 17-20 December
__________