Summary of Terminal Evaluation
1. Outline of the ProjectCountry : Islamic Republic of PakistanIssue/Sector :Division in Charge :Period of Cooperation：
Feb. 2009 –Feb. 2012

Project title : Technical Cooperation for Establishment of Environmental Monitoring
System in the Islamic Republic of Pakistan
Cooperation Scheme : Technical Cooperation Project
Total cost:
450 million yen (Dec 2011)
Partner Country’s Implementing Organization：
Pakistan Environmental Protection Agency
Punjab Environmental Protection Agency
Sindh Environmental Protection Agency
KP Environmental Protection Agency
Balochistan Environmental Protection Agency
Supporting Organization in Japan：
Related Cooperation：

1-1 Background of the Project
Air and water pollution was on the rise in Pakistan due to automobile emissions as well as wastewater discharged fromdomestic and industrial sources. According to the Pollution Conditions Survey conducted by JICA in 2000, concentrations ofpollutants exceeded Japanese or WHO environmental standards by 20-90%. A concern was growing over the emission ofparticulate matter in the atmosphere, wastewater seepage into aquifers and adverse health effects on the population. However, thecountry did not established fully functional environmental monitoring network and trained personnel fit for this circumstances.
Given this situation, the establishment of an environmental monitoring system was urgently required to conduct appropriate
environmental administration. With the aim of establishing the basis of a permanent nationwide environmental monitoring system
in Pakistan, under the "Establishment of Environmental Monitoring System" grant aid project of 2005 (hereinafter referred to as the
EMS grant aid project), the Japanese government built a central environmental analysis laboratory (Islamabad), and completed theimplementation of air and water quality monitoring systems and the analysis equipment in April 2007. This Technical cooperation
project started in February 2009 aiming at enhancing the technical capacity of EPAs in air & water quality monitoring in Pakistanwith the utilization of facilities and equipment’s provided under the Grant Aid project. Before the termination of the project, JICA
dispatched the terminal evaluation mission in order to assess accomplishment of the project, withdraw lessons learned and provide
recommendations.
1-2 Project Overview(1) Overall Goal: Environmental monitoring systems are place at the Federal and Provincial EPAs.
(2) Project Purpose: The federal and Provincial EPA’s capacity of environmental monitoring on air and water is enhanced.
(3) Output :Output 1
Pak-EPA and Provincial EPAs are capable of formulating Environmental monitoring plans.
Output 2
Pak-EPA and Provincial EPAs are capable of measuring the major parameters of National Environmental Quality
Standards (NEQS) based on uniform methodologies of sampling measurements and analysis.
Output 3
Laboratory management system is improved and Quality Assurance /Quality Control (QA/QC) system is established
in Pak-EPA and Provincial EPAs.
Output 4
Pak-EPA and Provincial EPAs are capable of interpreting and evaluating monitoring data based on the internationallyrecognized environmental standards/ NEQS.
Output 5
Based on the Environmental Monitoring Information System, Pak-EPA and Provincial EPAs are capable of compilingmonitoring data and disseminating to the public.
1-3 Inputs＜Japanese side＞
Japanese experts dispatched
13
Provision of equipment App. JY15,890 thousand
Acceptance of trainees in Japan 12
Operational cost
App. JY18,380 thousand＜Pakistan side＞
Counterparts
19
Provision of facilities
Office space
Operational cost
App. JY41,230 thousand (incl. salary for counterparts)
2. Evaluation Team
1. Mr. Hideo Noda (Leader), Director, Environmental Management Division 1, Global Environment Department,
Members of
JICA
Evaluation
2. Mr. Shun Nesaki (Cooperation Planning / Minutes Discussion), Assistant Director, Environmental
Team

Management Division 1, Global Environment Department, JICA
3. Mr. Hideyuki Kubo (Evaluation Analysis) Social Development Department, Global Link Management Inc.
4. Mr. Muhammad Ashraf, Joint Secretary, Ministry of Disaster Management
5. Mr. Raja Aurangzeb Khan, PhD, Chief, Planning & Development Division, Planning Commission
6. Mr. Waqar Hussain Abbasi, Deputy Secretary, Economic Affairs Division, Ministry of Economic Affairs andStatistics
29 Nov. 2011 – 15 Dec. 2011
Type of Evaluation: Terminal evaluation

Period of
Evaluation
3. Results of Evaluation
3-1 Summary of Project Accomplishment
Output 1:
All EPAs developed environmental monitoring plans for pilot sites of both water and air quality monitoring. Persons in chargeof the plans are now confident on their capacity of formulating monitoring plans by themselves. A guideline for environmental
monitoring plans is also formulated by Pak-EPA.
Output 2:
In all EPAs, there is at least one technical expert for each of water and air monitoring who can collect samples and analyze themby referring to NEQS. SOPs were developed on 32 items for water monitoring, 8 items for environmental air monitoring and 15
items for effluent gas. A maintenance manual is also developed for 14 equipment. Quality control records for environmental air
monitoring and log books for main equipment are also maintained.
Output 3:
Laboratory management systems have been improved in a sense that responsible persons are identified, management manuals
are developed and QA/QC activity plans are formulated.
Output 4:
All EPAs already conducted the interpretation and evaluation of monitoring data by referring to NEQS. There are a few
technical personnel who are credibly capable of interpreting and evaluating monitoring data by themselves. The result of thecapacity assessment before and after training courses on the interpretation and evaluation of water monitoring data indicates that the
rating of staff’s capacity is increased from 2.3 to 3.9 in average (0 – 5 rating).
Output 5:
At least one technical personnel for each of water and air monitoring in all EPAs is capable of compiling data. Data are alreadydisclosed to the public on demand basis although the arrangement of regular publication is not realized due to the constraint onequipment and connectivity except KP-EPA.
Project Purpose:
Project purpose is achieved. The result of the proficiency test for the participants that has been conducted at the occasion of aseries of ambient and emission air monitoring trainings demonstrates that their proficiency rate has increased from 18% (at theirfirst participation) to 40% (at their latest participation) for ambient air monitoring (n=23) and from 9% (at their first participation)
to 34% (at their latest participation) for emission air monitoring (n=20). All the senior officials of each EPA interviewed during theterminal evaluation articulated that capacity of their technical staff has been largely improved and they expressed their appreciation
to the Japanese Expert Team for this achievement. Also, all the trained technicians interviewed during the terminal evaluation
expressed their confidence in environmental monitoring work as most of them did not have knowledge and skills on the monitoring
when they were initially involved in the project.
Overall Goal:
The achievement of overall goal is subject to the availability of securing of operational budget and technical personnel by each
EPA after the project is closed. Currently, Punjab-EPA and KP-EPA are most likely to achieve as they already confirm a new PC-1
while other EPAs are in the process of securing budget and technical positions so that they also have possibility to achieve.
Institutional settings for environmental monitoring are being developed in all EPAs (i.e. planning, sampling, analyzing, compilingand reporting) while their progress differs among EPAs. Budget for technical personnel is already confirmed after the terminationof the project in Punjab and Balochistan EPAs. The maintenance cost for equipment is also confirmed in Punjab EPA.
3-2 Implementation Process(1) Progress of Activities
This technical cooperation project is designed to utilize the equipment provided by the EMS grant aid project. When the project
started in February 2009, however, it was found out that some equipment was not properly installed by the EMS grant aid projectand some went out of order due to frequent power cuts, voltage fluctuations and power surge. Hence, the project was required to
repair them first before planned project activities were carried out since the equipment was necessary for the project activities. The
first year was mostly spent for the repairing work. Aside from this issue, planned activities have been implemented in due course.
(2) Communication and management
As formal communication, monitoring and decision-making bodies, JCC and PSC were established. Even though important
issues were discussed and their solutions were suggested at JCC and PSC and they were recorded and signed in minutes ofmeetings, some of them were not carried out after the meetings in some EPA, including the case of the extension of EMS staff
contract and of holding of bi-monthly meetings.
On the issue of communication among concerned project personnel including senior officials of EPA, laboratory staff andJapanese experts, there are not much difficulty in exchanging views and sharing ideas except the following cases: (a)communication between Japanese experts and EPA staff of KP and Balochistan because the former is prohibited to visit the latter
due to security concern, and (b) communication among EPA staff in the context of sharing technical information as they are notproperly disseminated. However, less difficulty in communication does not lead to smooth project implementation since there arevarious impeding factors as described in (3) below, which are often beyond the capacity of concerned personnel to address.
(3) Factors that affected the implementation processa) Unstable budget allocation by the federal government
Rescue work from the large scale earthquake that occurred in October 2005 as well as actions for the War on Terrorism forced
the federal government to divert existing PC-1 budgets of various Ministries to these activities. In this project context, actual
expenditures for the fiscal year 2008/09 and 2009/10 were less than 10% of the initial budget stipulated in the PC-1. This budget
condition further deteriorated for the fiscal year of 2010/11 due to the severe flood that hit the country in August 2010. Emergency
relief work was financed by diverting existing PC-1 budgets as the case before. Hence, securing the budget for project activities has
become further difficult. Since the procurement of consumable materials such as reagent and materials for equipment is the duty of
the Pakistan side, reduction of budget allocation for the project has directly caused the operational difficulties of the project.
b) Devolution of Ministry of Environment
The government of Pakistan made decision that Ministry of Environment shall cease to exist with effect from 1st July 2011
following the Eighteenth Amendment of the Constitution that stipulated the devolution under Article 270AA. Since the projectbudget of PC-1 was authorized by the government through Ministry of Environment, the provincial portion of the PC-1 was also
devolved to provincial governments. This required respective EPAs to formulate a new PC-1 and submit to their provincial
government to secure the project budget while Pak-EPA continues to receive the budget of the federal portion.
c) Shortage and delay of counterpart assignment
Initially, the EMS grant aid project was supposed to recruit 120 new staff at the federal level and assign them to provincial
EPAs for the implementation of the EMS grant aid project and the project. The actual recruitment, however, was 28 staff as ofMarch 2007 (the planned number was dropped to 100 staff at this stage). This is because (a) new recruitment was periodicallybanned by the federal government, (b) institutional difficulty for provincial governments to employ centrally recruited staff.
In addition, each EPA attempted to employ their own permanent staff for environment monitoring activities who were theninvolved in the project. However, the total number was not increased up to the planned level and the insufficient number ofassigned counterparts naturally caused the limitation of the project implementation.
d) Persistence of load-shedding
Due to load-shedding that frequently occurs in the country, many of sophisticated equipment provided by the EMS grant aid
project have been subject to the risk of damages that are caused by voltage fluctuations and power surge. With the instability of themaintenance budget for the equipment, timely repair has not been (and will not be) feasible and monitoring activities are (and willbe) hampered although some EPAs procured Uninterruptible Power Supply (UPS) and Automatic Voltage Regulator to minimize
damages against equipment.
e) Restriction of access to project sites by Japanese experts
The visit of Japanese experts to both KP and Balochistan has been prohibited due to security reasons though it was notprohibited at the time of the EMS grant aid project in 2005. This condition has seriously constrained communication between EPA
personnel and Japanese experts and also opportunities of capacity building for the personnel in two EPAs.
3-3 Evaluation Results(1) Relevance
Relevance of the project is basically high. The government of Pakistan really needs to take bold actions to address air and waterpollution problems and environmental monitoring should be the initial step of such actions. At the policy and legal level, thegovernment of Pakistan has developed various policy documents and legal framework that involve the issue on environmental
pollution and monitoring including Pakistan Environmental Protection Act (PEPA) of 1997, National Environmental Action Plan
(NEAP) of 2001 and National Environmental Policy of 2005. In the context of Japanese ODA policy, one of priority areas for
Pakistan is on the environment, as a cross-cutting issue.
In terms of the project design, however, it was overstrained that all the five EPAs were the target of direct technical transfer andoutputs included a broader scope of environmental monitoring including management planning. In addition to it, the original designof the implementation structure that was developed by the EMS grant aid project involved the issue of feasibility. As indicated in
3-2 (3) c) above, it was designed in the originally approved PC-1 that 120 project personnel were to be newly recruited by thefederal government. The cost of these personnel as well as operational expenses was expected to be borne by Pak-EPA for the firsttwo years and by provincial EPAs after two years, which required the provincial government to double their annual budget. In
general, the assumption of doubling the organizational budget can be considered as impractical under the ordinary situation, eventhough concerned government agencies including financial authority confirmed to provide. Furthermore, it is not easy forprovincial EPAs to simply accept the transfer of staff recruited by Ministry of Environment since they have their own staff
recruitment systems that are rather independent under the federal system of the government. Although such design was developed

by the grant aid (not by the technical cooperation project), it was not appropriate that the R/D of this project simply followed theoriginal design.
(2) Effectiveness
Effectiveness of the project is high based on PDM (ver.2). Knowledge and skills of technical staff in each EPA are significantly
improved (i.e. project purpose is basically achieved) and five outputs effectively contributed to the enhancement of organizational
capacity of EPAs. It should be noted, however, that the enhancement of EPA’s capacity is only in the relative term (while theysatisfy indicators) and the level of staff’s knowledge and skills should be further enhanced in order to effectively conduct
environmental monitoring by themselves. One weakness is on the publication of results to the public. Although EPAs often provide
the analytical results of monitoring data to Environmental Tribunal, systematic provision of data to the public on the regular basis is
not realized at the time of the terminal evaluation except KP-EPA where data dissemination is arranged through electronic andprinted media every day.
For Pak-EPA, Punjab-EPA and Sindh-EPA, their technical staff have benefited to have direct opportunities of technicalcoaching by Japanese experts at their own workplace in Islamabad, Lahore and Karachi. This condition has enabled technicians ofboth for water and air monitoring to be skillful on various aspects including operation and maintenance of equipment, laboratory
analysis, data processing/interpretation and even quality control in laboratory management.
Although there was disadvantage for KP-EPA and Balochistan-EPA that Japanese experts could not visit their laboratory forcoaching, the improvement of knowledge and skills on operation of equipment, laboratory analysis and data analysis/interpretationis remarkable in a sense that at least one technical staff for each of water and air monitoring is now able to operate AAS, GC, Air
Monitoring Station and High Volume Sampler and to check BOD/COD and the pollution level of emission sources, as the case ofother three EPAs.
(3) Efficiency
Efficiency of the project is high. Throughout the project implementation period, various unexpected troubles occurred that werebeyond the control of the project. Despite such difficulties, the project could have managed to produce outputs and achieve project
purpose at the satisfactory level.
While inputs were planned rationally by both Japanese and Pakistan sides, actual utilization of the inputs was affected by
unexpected causes such as budget diversion to rescue operation on flood in 2010, rehabilitation from earthquake damages anddevolution of Ministry of Environment in 2011 that led to the devolution of the Umbrella PC-1. This is, indeed, equivalent to thenullification of pre-condition that is “Financial and human resources are allocated each EPA to implement the project during the
project period.” Due to the devolution and slow progress of new PC-1 processes, the allocation of maintenance cost for equipment
and salary to EMS staff stopped after July 2011.
In addition, frequent load-shedding has hampered laboratory work and often caused damages on equipment. Since major
monitoring equipment is sophisticated, repairing cost is expensive in terms of spare parts (imported one) and repairing fee. Thiscondition of high-cost operation could not be improved due to the characteristics of equipment and persistence of load-shedding
which is beyond the project’s control.
(4) Impact
In general, all EPAs are now able to formulate monitoring plans, collect samples and analyze them, compile data and report toconcerned agencies. Hence, it can be recognized that environmental monitoring systems are to some extent institutionalized, eventhough they are not sufficient, and these systems will function (i.e. the overall goal will be achieved) if financial arrangement ismade and current technical capacity is maintained in each EPA. Considering the situation, impact of the project can be assessed ashigh in Punjab-EPA and KP-EPA, and will be positive for other EPAs.
In Punjab-EPA, the budget for the next three years is already confirmed and also for the case of KP-EPA so that it is mostly
likely that the overall goal is achieved. For other EPAs, financial arrangement is not confirmed yet at the time of the terminal
evaluation although there are prospects that they can secure budget for monitoring personnel and activities and will thus achieve theoverall goal. While the capacity of technical staff is enhanced, it is still unclear whether the EMS staff, the main target personnel
who participated in capacity building activities, are employed by provincial EPAs after the termination of the project. Unless theexisting capacity of technical personnel is maintained, it is very unlikely that the overall goal will be achieved. In addition to thecontribution to the achievement of the overall goal, the project has largely facilitated the provision of more precise and reliable
environmental data to Environmental Tribunals when pollution claims are raised in respective provinces. Before the project started,the capacity of EPAs was limited to the acquisition of a handful of environmental monitoring data but they are now capable ofproviding full set of NEQS parameters to the Tribunals if required. Furthermore, it led to the formulation of Pakistan Clean Air
Program and contributed to the development of Drinking Water Quality Standard.
There is no negative impact that has been caused by the project. It needs to pay attention to, however, the issue on the disposalof heavy metals and organic solvent in the future. This issue should be addressed property under the QC/QA system when EPAs
face the requirement of disposal of these materials.
(5) Sustainability
Sustainability of the project also differs among EPAs. It is very high for Punjab-EPA where around Rs.82 million PC-1 wasalready approved for the next three years and that technical positions and operational costs are secured. It is high for KP-EPA
where daily data dissemination is in place and related PC-1 projects were already approved. It is also high for Balochistan-EPA
where technical personnel are permanent although budget for monitoring activities are not secured yet. For other three EPAs,
sustainability is still unclear at the time of the terminal evaluation since the status on budget and technical positions for monitoring
activities are not confirmed yet.

At the national policy level, environmental management and monitoring policy has been consistent during the last two decades
and its transformation will not be expected in the near future. However, there is a priority issue among policies of different sectors.
Although environmental monitoring gained high priority within the environment sector, this would not be the case in general.
Hence, policy consistency does not necessarily lead to consistent budget allocation to environmental monitoring activities.
At the organizational level, the main project counterparts who enhanced their knowledge and skills are those recruited as aresult of the EMS grant aid project. Since they are time-bound employees, there is no guarantee that they can remain at respective
EPAs. This is particularly the case for the staff dispatched to provincial EPAs because they were recruited at the federal level andnot integrated within the provincial human resource management system.
From the financial viewpoint, EPAs are still in the process of securing budget for environmental monitoring except the case of
Punjab-EPA (where a new PC-1 is already confirmed) and KP-EPA (where several related PC-1 are confirmed, and securing staff
positions of Air and Water Monitoring are secured through non-development budget), and partly the case of Balochistan-EPA(where technical staff are permanent). In addition, maintenance of equipment is questionable due to the nature of high maintenancecost for equipment. In Pakistan, spare parts are required to be imported and repairing services can be provided only by handful ofmonopolized local agents (thus service fee is quite high). Furthermore, the risk of trouble occurrences is constantly high asload-shedding continues to take place, which is obviously beyond EPAs’ control. Since financial provision to maintenance islimited, it is likely that some equipment may not be repaired once it is damaged.
Technical sustainability is basically high as technical staff of EPAs demonstrated significant improvement in their knowledge
and skills for environmental monitoring and they are now capable of implementing many of activities without external assistance indue course. This achievement, however, faces a risk. Although their capacity has been enhanced to a great extent, they need further
improvement for effective implementation of monitoring activities. What needs to be done for that purpose is to let themcontinuously implement monitoring work. However, due to multiple factors including financial constraint and equipment trouble,
continuous operation of monitoring work might become difficult to realize. If this happens, their technical capacity would
deteriorate as time goes by.
3-4

Conclusion
The terminal evaluation team concludes that the project can be closed as scheduled as it has achieved the project purpose in asense that each EPA is now able to draft environmental monitoring plans, collect samples from monitoring sites, analyze them byusing equipment, compile the data and prepare a report. While this is the significant achievement, it can be recognized that the levelof progress is diverse among EPAs as well as among technical personnel and there are some technical areas that can be improvedfor some personnel, including the capacity of data analysis and interpretation, and also some areas in which they are encouraged tofurther enhance their skills such as quality control of laboratory operations.
One critical concern is sustainability of the achievement of the project. So far Punjab-EPA has confirmed the budget under a
new PC-1 and KP-EPA confirmed the budget under several PC-1 and non-development budget over the next several years, whileother EPAs are still in the process of securing the operational budget for environmental monitoring. Without operational budget,
opportunities of using knowledge and skills of technical staff and further enhancement of their capacity are not provided. Hence,
there is a risk that the capacity that has been accumulated through the implementation of the project may not be sustained or may
not be further enhanced. It should be noted that the ultimate goal to secure sustainability is to provide all the cost from
non-development fund of each province as well as federal government. However, the regularization of the EMS staff from PC-1
budget to non-development budget may take some time. In such a case, continuous adoption of PC-1 is an alternative scenario ofsustainability.
3-5 Recommendations(1) Securing operational budget and regularization of the technical staff
The EPAs (except Punjab-EPA and KP-EPA) are requested to go through the process for the confirmation of the budget thatshould cover maintenance cost of equipment, salary for technical staff and other operational costs for water and air monitoring. At
the same time, the EPAs (except Balochistan where monitoring staff are already permanent) are also requested to regularize theexisting temporal technical staff (i.e. the EMS staff) through the due process of each province in order to utilize human resource
capacity that has been developed by the project. This is essential for the overall institutional strengthening of each EPA and theircapacity to act as technically sound government organizations to monitor and suggest collective measures for air and water quality
improvement and their respective areas of jurisdiction.
(2) Follow-up monitoring
As expressed by EPA senior officials, EPAs are expected to continue the periodical follow-up monitoring meetings on the
progress of securing the budget as well as regularization of the EMS staff for environmental monitoring in order for the technicalcapacity of the EPAs to be sustained and further enhanced.
(3) Assessment of current capacity of each EPA for further improvement in terms of organizational capacity development
Japanese expert team is requested to conduct a capacity assessment of each EPA in order to identify areas of technical capacity
for further improvement. The result of the assessment is expected to provide clear picture for senior officials of each EPA about thestrength and weakness of their existing capacity and the way forward for further enhancement of their environmental monitoring
capacity. In addition, if there are some personnel who hold particular strength that is scarce in other EPAs, they can function as
resource persons and provide their knowledge and skills for the capacity development of technical staff.
(4) Extension of the existing PC-1
Pak-EPA is requested to consider the extension of existing PC-1 that was terminated at the end of November 2011 in order toprovide continuous financial arrangement for the operational and EMS staff cost. During the extension period, that is considered as“transition period,” other EPAs except Punjab-EPA and KP-EPA are requested to confirm new PC-1 for accommodating the cost.
(5) Data sharing among EPAs and dissemination of the information
EPAs are recommended to coordinate themselves to develop a system in which monitoring data are shared among all of them.
At the moment, the data of air monitoring is sent to Pak-EPA so that they can understand the situation of the environment in all the
major cities. As stated on Output 5 of the project, information / data should be disclosed to the public on the regular basis.
(6) Sharing of technical information within each EPA
Respective EPAs are requested to develop a system to share technical information among their staff in order to encourage themto actively share the information they have.
3-6 Lessons Learned(1) Needs of careful implementation structure in case of umbrella PC-1
Although umbrella PC-1 is a better option in some cases (e.g. when a new program is introduced at the nation-wide scale), thesystem of umbrella PC-1 is found to be difficult to manage from the operational viewpoint. First, major decision-making on the
project design and implementation was made at the central level including budget allocation and staff assignment. This caused
various problems in terms of the implementation and sustainability of the project; namely, (a) upon the devolution of Ministry ofEnvironment and umbrella PC-1, provincial EPA suddenly lost operational budget and (b) the technical staff employed under the
EMS grant aid project cannot be automatically regularized by provincial EPAs because human resource management systems ofprovincial government are independent from the federal government. Second, the situation of environmental monitoring capacity isdiverse from province to province. Centralized project coordination does not really meet the needs of respective EPAs that havedifferent level of requirement. In such a case, careful thought is required in developing implementation structure. For example, itwould be more accommodative if provincial PC-1 is elaborated in line with umbrella PC-1 because local needs are duly reflected in
the provincial scheme while new concept and approach can be introduced through the federal scheme.
(2) Needs of identification of available agents including a third country’s agents for spare parts and consumable goods that are not
locally available
Insufficient spare parts and consumable goods for the equipment affected the smooth implementation of the project. Since someof the spare parts and consumable goods are not locally available, EPAs faced difficulties in operating monitoring activities
properly when they exhausted consumable goods and equipment was damaged. It is better at the initial stage that locallyunavailable spare parts and consumable goods are listed and also a list of agents including third country’s agents who can procure
spare parts and consumable goods is prepared in order to ensure equipment to be maintained and repaired in case of damages. This
experience will help them to cope with the procurement of these materials in a proper manner.