Official
Use OnlyIndonesia
Ex-post Monitoring of Completed ODA Loan Project“The Bepedal Regional Monitoring Capacity Development Project”
External Evaluator: Nobuyuki Kobayashi(OPMAC Corporation)
Field Survey: August 2009
1. Project Description

Map of the project area
1.1

Equipment procured by this project

Project Objective

The objective of this project is to improve the capacity for collection and analysis ofenvironmental data(water/air pollution, etc.), which forms the basis for the development ofmonitoring programs, by developing the capabilities of regional laboratories (research
institutes), thereby contributing to improvements in Indonesia's environment throughimproving the efficiency of environmental administration.
1.2 Outline of the Loan Agreement
Approved Amount /2935 million yen / 2743 million yen

Disbursed Amount
Loan Agreement Signing Date/

November 1994 / December 2001

Final Disbursement Date
Ex-post Evaluation

FY 2003

Executing Agency

Environmental Impact Management Agency（BAPEDAL）（at the time of appraisal）

Main Contract

Itochu Corporation (Japan)

Main Consultant

Nihon Suido Consultants (Japan)

1

1.3

Background of Ex-post Monitoring

As rapid population and economic growth accelerated environmental degradation inIndonesia, the establishment of a nation-wide network for environmental monitoring
became an urgent task.

At the time of appraisal (1994) however, there were few

laboratories which owned the appropriate equipment for testing besides in Jakarta and, thus,
environmental data was neither sufficient in quantity nor quality. In order to cope with thisproblem, this project provided regional laboratories with the equipment for environmental
monitoring. The ex-post evaluation confirmed that the equipment was utilized to someextent but it also revealed that there had been no clear program for the utilization of theregional laboratories assisted by this project following the resignation of President Suharto
in 1998. The laboratories were merely measuring and providing data in response to client
requests. Decentralization altered the institutional arrangements for the operation andmaintenance of this project. The ownership of the procured equipment was to be transferred
to provincial governments while the location of the equipment was to be changed toprovincial environmental departments. Because of this, at the time of the ex-post evaluation,
careful monitoring was required as to how decentralization would affect the sustainability ofthe project effects. Therefore, this project was selected for ex-post monitoring and reviewed
under each criterion with the findings from the field survey and other research activities
with a final conclusion being drawn.
2. Monitoring Results
2.1

Effectiveness (Impact)

There was no significant change in effectiveness from the ex-post evaluation (2003) tothe ex-post monitoring (2009). Among the laboratories with equipment procured by thisproject, the number of testing samples has stayed at the same level as at the ex-post
evaluation and the number of laboratories accredited ISO17025 remains the same. 1 As
equipment little used for testing tends not to be repaired after machine trouble, their use has
become less frequent than at the time of the ex-post evaluation. Although the target
laboratories play a similar role in the environmental monitoring, there are several new cases(the contribution to the reduction of pollutant, the enhancement of environmental
monitoring

capacityin Kota/Kabupaten governments,

and

the

enforcement

environmental regulations) which contribute to realizing the desired project effects.
2.1.1 Quantitative Effects
1

ISO17025 is for testing or calibration organizations and accredits a certain type of material testing or that of
the calibration of meters.

2of(1)Usage of the procured equipment
Out of the laboratories with equipment procured by this project, 10 laboratories werevisited during the field survey when interviews about the current status of equipment usage
took place. For comparison with past data, the laboratories which were inspected in the
ex-post evaluation were revisited. These were:North Sumatra province: BLH (provincial government)
West Jawa province: BBPK (Ministry of Industry: MOI), BPLK (provincial government),
BPMKL (provincial government)
East Jawa province: Baristand (MOI), BBTKL (Ministry of Health: MOH), BLH(provincial government)
South Kalimantan province: Baristand (MOI), BLK (provincial government), Dinas PU(provincial government)*Ministries and local governments with which the laboratories are affiliated are inparentheses
Amongst the equipment for water quality monitoring, UV-VIS and Atomic Absorption
Spectrophotometers (AAS) are widely used for water testing by private companies and local
government and their use is very frequent (see Table 1 and Table 2). On the other hand, theuses of Gas Chromatography (GC) and Total Organic Carbon Meters (TOC) are lessfrequent than it was at the time of the ex-post evaluation. For most of the target laboratories,
the main purpose of the equipment is to meet testing requests from clients. The small
number of clients with testing needs results in the infrequent use of equipment. There arevery few clients which require testing using TOC. Except for residual pesticide, testing with
GC is also very limited.
Table 1

Use of procured equipment at the time of ex-post evaluation(for the previous 6 months)

EquipmemtN

Used

Not used

Unknown

UV-VIS

18

13

0

5

AAS

18

11

0

7

GC

15

11

1

3

TOC

18

11

2

5

Air pollution observation vehicle

10

1

4

5

Source: Data collected for the ex-post evaluation on
The Bepedal Regional Monitoring Capacity Development Project3

Table 2 Use of procured equipment at the time of the ex-post monitoring(Used/Not Used for the previous 6 months) 2
Equipment
UV-VISN
10

Used

Not used

AAS

10

9

1

GC

7

2

5

TOC

10

4

6

Portable multi gas analyzer

4

3

1

Air pollution observation vehicle

4

1

3

10

0

Source：Interviews at the 10 laboratories visited during the field survey
Photo 2
Photo 1

Photo 3

UV-VIS

Atomic Absorption

Spectrophotometer (AAS)GC

Photo 42

Total Organic Carbon Meter(TOC)

The ex-post evaluation report did not mention the use of portable multi gas analyzers. In consideration of its
importance in air quality testing, the use of the equipment was assessed in this ex-post monitoring.

4

Among the equipment for air quality monitoring, the portable multi gas analyzers areused for the testing of smoke-stacks at factories and that of ambient air (see Table 1 andTable 2). The laboratories under MOI have clients who require the testing of smoke-stacks.
The laboratories which indicated that they do not use the portable multi gas analyzer quoted
malfunction as the reason of nonuse. As for Air pollution observation vehicles, out of four
laboratories visited during the field survey, three laboratories had experienced problems at
the time of procurement (the controlling PC could not be connected to meters, etc.). In thecase that the vehicles were used, equipment was removed from the vehicle and used inside
the laboratory.
Photo 5

Portable multi gas analyzer

Photo 6

Air pollution observation vehicle(2)Number of Monitoring Samples
In order to obtain information on the usage of equipment, a questionnaire survey with thelaboratories was carried out for the ex-post monitoring. Using the latest information,questionnaires were sent out to 39 laboratories in 14 provinces which own or received the
procured equipment. Questionnaires were returned from 24 laboratories, 20 laboratories ofwhich still owned the equipment at the time of the ex-post monitoring. 10 laboratories
answered on the numbers for water testing, 6 laboratories on stack testing, and 4
laboratories on ambient air testing.
The change in the number of testing samples in comparison with data at the time of the
ex-post evaluation is shown below (see Table 3). A majority of the laboratories replied that
there was no change in the numbers for testing of water, stacks, or ambient air.

5

Table 3

Change in testing samples(comparison with the time of the ex-post evaluation 2003)
Increase

No Change

Decrease

Total

Water

3

5

2

10

Stacks2

4

0

6

Ambient Air

1

3

0

4

Source: Questionnaire with target laboratories(3) ISO 17025 accredited laboratories
According to the database of the Indonesian national standards organization Badan
Standardisasi Nasional (BSN), among those with the procured equipment, 10 laboratories
had ISO17025 in 2008. At the time of the ex-post evaluation, out of those with theequipment, 10 laboratories had ISO17025. After April 2009, MOE Regulation No.6 of2009 requires accredited environmental laboratories (which are allowed to test data to besubmitted to national and regional laboratories) to have ISO17025.

Because of this new

regulation, all of the 10 laboratories visited during the field survey had obtained or wereapplying for ISO17025. As the preparation of Standard Operational Procedures (SOP) is
required for ISO17025 accreditation, most of the visited laboratories had SOP.
2.1.2 Qualitative Effects(1)Factors affecting the use of equipment
Focus group discussions (FGD) were carried out at BBTKL 3 , which belongs to MOH, inorder to obtain views on the factors that affect the effective use of equipment. Two FGD
sessions were conducted: one for administration staff and another for testing staff. In the
FGD, the most common reason given for the effective use of equipment among theadministrative staff was that the equipment satisfied the needs of BBTKL (see Box 1).
Similarly, testing staff were of the opinion that the procured equipment complemented theexisting equipment and had a high utility, in addition to this, the improvement in testingskills contributed to the better use of equipment. Those who use the equipment perceived
that not only capability in the use and maintenance of the equipment but also the assessmentof equipment needs plays a vital role in the effective use of the equipment.

3

As BBTKL scored highly in the Special Assistance for Project Sustainability for the Bapedal Regional
Monitoring Capacity Development Project funded by the Japan Bank for International Cooperation, and it waschosen as a laboratory which uses the equipment effectively.

6

Box 1: Focus Group Discussion (No.1)
Date: August 11, 2009
Location: BBTKL (affiliated with Ministry of Health), Surabaya, East Jawa province
Discussion Theme: “Why does BBKTL effectively utilize the equipment?”
Participants: BBTKL testing staff (11 participants) and administrative staff (8 participants)
After discussion, participants were asked to cast votes (up to three votes per person) on thereasons for why they agreed about effective equipment use. The results were as follows:

Testing staff
Operators’ competence issatisfactory due to training
New equipment complements
existing equipment
Routine maintenanceImprovement of the accuracy ofanalysis results
Fast and simple methods foranalysis

# of
votes

Administrative Staff
Equipment fulfilled the needs ofthe laboratory
Laboratory is a accredited
environmental laboratory

1085State of the art equipment

# of
votes
10
8
6

3
3

More testing parameters

2

Improvement of analysis
methods2

2.1.3 Impact(1)Contribution to regional environmental monitoring systems
The target laboratories play a similar role in environmental monitoring as they did at thetime of the ex-post evaluation. The laboratories test and provide samples upon request fromclients (provincial governments, Kota/Kabupaten governments, and private companies).
However, there are several new cases contributing to the realization of project effects otherthan testing for clients.
1) Efforts to reduce pollutant
FGD were carried out at BBPK, which belongs to MOI, in order to obtain views on theeffectiveness of equipment use (see Box 2). Unlike other laboratories, BBPK also place anemphasis in independent research and the consulting services to companies. BBPK does not
only test samples from clients but also contributes to the reduction of pollutants at clientcompanies via research and consulting. The procured equipment is utilized for the researchand consulting activities of BBPK.

7

Box 2: Focus Group Discussion (No.2)
Date: August 7, 2009
Location: BBPK (affiliated with the Ministry of Industry), Bandung, West Jawa province
Discussion Theme：“How does BBPK contribute to the protection of the environment?”
Participants：BBPK research staff (8 participants) and testing staff (10 participants)
Figures next to opinions are the number of participants’ votes. After discussion, participants
were asked to cast votes (up to three votes per person) on the reasons for why they agreed
about effective equipment use.
Research staff# ofTesting staff

# of

votes

votes

More efficiency in waste
processing (less pollutants)

6

More recycling

6

Training for companies

4

Monitoring of waste

2

Design of the processing system
for liquid waste
Improvement
of
score
in
environmental assessment
Dissemination of research results

Conducting analysis of various
kinds of industrial waste
Consultation with companies

15

Conducting
research
environmental protection
Training for companies

5

for

6

4

2
2
2

2) Review of environmental data from Kabupaten /Kota governments and training
Under the environmental monitoring scheme in Indonesia, Kota/Kabupaten governments
inspect river water quality in their assigned areas. Meanwhile, provincial governments put
together data from the Kota/Kabupaten governments and collect data in their assigned river
sections. Among the laboratories visited during the field survey, the procured equipment
was transferred to the laboratories under the provincial environmental departments in North
Sumatra and East Jawa provinces. According to interviews at the laboratories, the provincial
governments checked water quality data from the Kota/Kabupaten governments with thedata being tested using the procured equipment. In North Sumatra province, theKota/Kabupaten governments purchased new equipment with the support of the special
national account for the environment (DAK) but the number of staff who can use this new
equipment is limited. The laboratory under the provincial department of the environment
provides training to the Kota/Kabupaten governments using the equipment procured by this8

project.
3) Contribution to compliance with environmental regulations
For the 10 laboratories visited during the field survey, the department of environment inEast Jawa province routinely conducts forced inspections jointly with the police. Samples
taken from forced inspections are tested in laboratories near Surabaya, some of which aretarget laboratories. Based on results of tests, the closure of factories and the punishment ofrelevant persons may be executed. The procured equipment contributes to compliance withenvironmental regulations.
(2)Impact on the environment
Negative effects from the target laboratories on the surrounding environment were notobserved during the field survey.

In accordance with MOE Regulation No.6 of 2009,

which has been effective since April 2009, an accredited environmental laboratory is
required to obtain ISO17025. As the accreditation of ISO17025 demands the appropriate
treatment of waste water, the laboratories visited are now dealing with this requirement.
2.2

Sustainability

After the ex-post evaluation, many of the target laboratories changed their affiliation from
the Ministry of Public Works (MOPW) and MOH to provincial governments. While therelocation of equipment has been carried out in a handful of provinces, in other provinces
equipment is still located at the laboratories where it was originally installed and theselaboratories still have the right of use. As the Ministry of Environment (MOE) still has
ownership of the equipment, some laboratories have inappropriate audit results that show
allocation of O&M budget to equipment which is not included in their accounting. As
technical aspects, the Environmental Impact Management Laboratories (formerly theEnvironmental Management Center, EMC) were providing training for the target
laboratories at the time of the ex-post evaluation. At the time of the ex-post monitoring, theEnvironmental Impact Management Laboratories provided training to environmental
departments. The target laboratories have training in various other institutions.
2.2.1Operation and Maintenance Agency
2.2.1.1 Structural Aspects of Operation and Maintenance
Along with the progress of decentralization, many of the target laboratories changed theiraffiliation from MOPW and MOH to provincial governments. Out of 20 laboratories where
the questionnaire survey confirmed that equipment still remains, the laboratories under MOI
continue to be under the same ministry. However the laboratories under MOPW and some9

of the laboratories under MOH are now under provincial governments (See Table 4).
Table 4Affiliation of laboratories

Name of Laboratory (at the time of appraisal)

Public Works Laboratory, MOPW (PU）At the time of

At the time of

Ex-post Evaluation

Ex-post Monitoring

MOPW,

Provincial govt.

Provincial govt.
Industrial Research and Development Laboratory

MOI

MOI

MOI

MOI

MOH,

Provincial govt.

(BPPI）
Cellulose Research and Development Center, MOI(BBS)Public Health Laboratory, MOH（BLK）

Provincial govt.
MOH

Environmental Health Laboratory（BTKL）

MOH

MOPW: Ministry of Public Works, MOI: Ministry of Industry, MOH: Ministry of Health
Source: Questionnaire survey with the laboratories、ex-post evaluation on the Bepedal Regional
Monitoring Capacity Development ProjectAt the time the of ex-post evaluation, there was a policy to transfer the ownership ofequipment to provincial governments and the location and the right of use equipment toprovincial environmental departments. In North Sumatra and South Sumatra provinces, theequipment was relocated. At the time of the ex-post monitoring, MOE still had ownershipand the relocation of equipment was confirmed in Lumpung (see Table 5). The laboratories
where equipment was originally installed still have the right of use.

Even in cases where

the location and right of use of equipment were changed, the ownership was not transferred.
The transfer of ownership requires approval from the Ministry of Finance but this process isdelayed.
At the laboratories where equipment is installed, interviewees explained the reason for
non-relocation as follows: relocation would be the interruption in service to clients,
provincial environmental departments do not have to own a laboratory as testing is possible
at a number of laboratories. In West Jawa province, the provincial government decided not
to move equipment from the laboratories where it was initially installed.

10Table 5 Structure of O&M
At the time of ex-post evaluation

At the time of ex-post monitoring（2003）（2009）

Ownership ofTo be transferred from MOE to

The policy to transfer ownershipequipment

provincial governments (theis valid but MOE still has

policy was informed provincial

ownership due to a delay in thegovernments in December 2002)

procedure.

To be move to provincial

Equipment was moved inenvironmental departments where

Lampung province as well as inrelocation is possible. The

North Sumatra and Southequipment has been relocated in

Sumatra provinces. In otherNorth Sumatra and Southstates, transfer is not confirmed.

Installation site

Sumatra provinces.
Right ofThe laboratories where equipment

equipment use

is installed have the right of use.

Ditto

Source: Interview at MOE, Questionnaire survey with the target laboratories

2.2.1.2 Technical Aspects of Operation and Maintenance
The Environmental Impact Management Facilities under MOE provided training toprovincial departments of environment and regional laboratories, including the target
laboratories, up to 2006 (after the ex-post evaluation). Out of 20 laboratories where thequestionnaire survey confirmed that equipment still remained, 7 laboratories replied thatthey participated in training at EMC. At the time of the ex-post monitoring, theEnvironmental Impact Management Facilities still conducted training for environmental
departments in local governments. The target laboratories had training in various otherinstitutions. Among the laboratories visited, many had the training for equipment use at theIndonesian Institute of Sciences (LIPI) and training for laboratory management at private
consulting firms which provide services to assist in ISO-accreditation.
Out of 20 laboratories where the questionnaire survey confirmed that equipment still
remained, 15 laboratories replied that spare parts for equipment are still available but are
difficult to obtain. Interviews at the laboratories visited revealed that the laboratories
requested repairs via suppliers who carry measuring instruments and that response was not
smooth except in Jakarta, Surabaya, and the outskirts of these cities.

It takes several

months to one year to obtain spare parts, occasionally out of Indonesia, even if a repair
service is available.

112.2.1.3 Financial Aspects of Operation and Maintenance
A majority of the 10 laboratories visited during the field visit are allocated an operational
budget from ministries and provinces. 4 In these laboratories, service fees from clients must
contribute to the revenue of ministries and provinces and the appropriation of revenue to
O&M is not allowed. The laboratories which currently use the procured equipment takeexpenses for the O&M of the equipment. As MOE has the ownership of equipment, somelaboratories have inappropriate audit results allocating O&M budget to equipment which is
not included in their accounting books.
The asset life of the procured equipment is approximately 5 years. The equipment can be
retired after this period is over. The retirement of equipment needs to follow procedures set
by the Ministry of Finance. In general, however, the retirement of equipment is very rare
because of these cumbersome procedures.
2.2.2 Current Status of Operation and MaintenanceFor the 10 laboratories visited during the field survey, the main purpose of the equipment
is to meet clients’ requests for testing. Equipment little used for testing, such as GC and
TOC, are more likely to remain unrepaired after machine troubles.
3．Conclusion, Lessons Learned and Recommendations
3.1 Conclusion
In view of the number of testing samples and the number of laboratories accredited
ISO17025, it can be concluded that there has been no significant change in effectiveness.
For the effective use of equipment, not only the capacity of laboratories but also assessmentof needs is an important factor. The role of the target laboratories remains to test samples
upon the request of clients as it was at the time of the ex-post evaluation. As forsustainability, the policy was to transfer the ownership of the procured equipment from
MOE to provincial governments. However, MOE still retains ownership due to the slow
progress of transfer. Furthermore, some laboratories have inappropriate audit results that
allocate O&M budget to equipment which they do not own.
3.2

Lessons Learned(1)Needs assessment by the executing agency
The main purpose of equipment use is to meet requests for testing from clients. As
4Only the Dinas PU laboratory in South Kalimantan is on a self-supporting basis.

12

equipment little in demand for testing is likely to remain unrepaired, the needs assessmentof equipment affects sustainability. At the time of the ex-post monitoring, testing demandfor UV-VIS, AAS, and portable multi gas analyzers is strong. On the other hand, that for
TOC and GC is weak. The type of equipment is determined by the affiliation of laboratories
as the number of target laboratories is too high for them to be assessed one by one. However,laboratories under the same ministry have different types of clientele and face various
testing demands. Equipment little in demand for testing was procured as a result of this. In
cases where dispersed project sites do not allow needs assessment, it is desirable that
advantage is taken of finance schemes, such as sector loans, in which the executing agency
conducts needs assessment and flexibly changes the scope of project, on condition that theimplementation capacity of the executing agency can be assured.
3.3

Recommendations(1)Ownership and right of use of the equipment (for MOE)
Since the ownership and the right of use do not matched for the procured equipment,
allocating O&M budget to the equipment can be misappropriated in the strict sense. Thissituation may prevent appropriate O&M. The transfer of ownership is unlikely to becompleted in the near future and, therefore, it is recommended that budget allocation isallowed for the O&M of equipment with proof of the right of use.
(2)Retirement of obsolete equipment (for MOE)
After a decade since procurement, aging, wear and tear, and the unavailability 5 of spareparts has disabled some of the equipment. Equipment which has passed its asset life andwhich is difficult to be repaired is still on the accounting books and is not yet retired.

In

order to accelerate the renewal of equipment, it is recommended that this equipment isdisposed of, following proper accounting procedures.

5

Laboratories pointed out some reasons, one of which is that suppliers’ inadequate response requires the
procurement outside of Indonesia.

13Comparison of Original and Actual Scope
Item

Original

ActualProvision of water quality monitoring

39 laboratories in 14 provinces

As planned

26 laboratories in 14 provinces

As planned

15 laboratories in 15 provinces

As planned

Foreign：92 M/M
Indonesian：134 M/M

Foreign：110 M/M
Indonesian：190 M/M

Consultant selection

June 1995

November 1995

Tender / Contract

December 1996

November 1999

Procurement / Installation

November 1997

October 2000

Maintenance phase

December 1998

November 2001

Foreign currency

2,935 million yen

2,349 million yen

Local currency

401 million yen

394 million yen（8,035million rupiah）(17,440 million rupiah)

Total

3,336 million yen

2,743 million yen

ODA loan portion

2,935 million yen

2743 million yen

Exchange rate

1IDR = 0.05 yen

1IDR = 0.226 yen（as of November 1994）（Average between September

Project Output:
equipment
Provision of air quality/noisemonitoring equipment
Provision of mobile laboratories
Consulting services
Project Period:

Project Cost:

1996 and June 2001）

14