# 6. Comprehensive Performance Evaluation {#comprehensive-performance-evaluation .unnumbered}

## Introduction

Based on individual filter monitoring requirements in the IESWTR, some
systems may be required to arrange for a Comprehensive Performance
Evaluation (CPE). Specifically, systems must conduct a CPE if any
individual filter has a measured turbidity level of greater than 2.0 NTU
in two consecutive measurements taken 15 minutes apart in two
consecutive months. The system must report the filter number, the
turbidity measurement, and the date(s) on which the exceedance occurred.
The system shall contact the State or a third party approved by the
State to conduct a CPE.

A CPE is the evaluation phase of the Composite Correction Program (CCP).
The CCP, including detailed CPE procedures and qualifications for CPE
providers, is described in a separate handbook (USEPA, 1998). This
chapter's goal is to present a fundamental discussion of CPE concepts
and provide a general understanding of what a plant should expect when a
CPE is completed. Detailed CPE procedures are not included in this
guidance manual. Detailed CPE procedures should be obtained from the CCP
Handbook (available by calling the EPA Safe Drinking Water Hotline at
1-800-426-4791).

## 6.2 Background On The CPE {#background-on-the-cpe .unnumbered}

The CCP is a systematic, comprehensive procedure that identifies and
corrects the unique combination of factors, in the areas of design,
operation, maintenance and administration, that limit the performance of
a filtration plant. It was developed to improve performance at
filtration plants using existing facilities thereby minimizing
construction alternatives. The capable plant model, presented in Figure
6-1, shows conceptually how the CCP considers the various aspects of the
operation, design, maintenance, and administration of a filtration
plant. A plant is considered capable when it has treatment processes of
sufficient size with adequate mechanical equipment to meet current water
demand, adequate administrative support including funding and policies,
and a maintenance program that keeps key equipment operational. Once
these components are in place, proper operations capabilities are
required for the plant to achieve its performance goals, whether for
regulatory compliance or treatment optimization.

At the core of the CCP is the assumption that if a filtration plant
cannot achieve specific performance, there is a unique combination of
interrelated factors with respect to the design, maintenance,
administration and/or operations of the filtration plant that are
limiting its performance. The purpose of the CPE is to identify these
factors and prioritize them with respect to their relative importance in
preventing compliance and/or optimized performance. Once the factors are
identified and prioritized they can be corrected so that performance can
improve and compliance can be achieved. During a CPE, the historic
performance of the plant is assessed with respect to pathogen removal
and inactivation. The design, administration, and maintenance of the
plant are completely reviewed to determine if they properly support a
capable plant. If they are not supporting a capable plant, the root
causes are identified as to how they are contributing to the performance
problem. Operational practices are also reviewed to assess if operators
have the necessary skills to achieve required performance and compliance
when provided with a capable plant.

[]{#__RefHeading___Toc448537623 .anchor}Figure 6-1. Capable Plant Model

In using the CPE/CCP it is important to understand that the approach has
applications in addition to achieving regulatory compliance and should
be applied as appropriate for meeting desired performance needs. All of
the CPE procedures are designed to focus a plant toward meeting the
compliance requirements and performance goals described in Table 6-1.

[]{#__RefHeading___Toc448537846 .anchor}Table 6-1. CPE Treatment
Performance Goals

  ----------------------- ------------------------ ----------------------
                          IESWTR Compliance        CCP Optimized
                          Requirements             Performance Goals

  *Minimum Data           Continuous individual    Daily raw water
  Monitoring and/or       filter turbidity         turbidity.
  Reporting Requirements* monitoring with values   
                          recorded at 15 minute    
                          intervals (conventional  
                          and direct filtration    
                          systems).                

                          Representative           4-hour settled water
                          filtered/finished water  turbidity from each
                          effluent turbidity every sedimentation basin.
                          4 hours.                 

                                                   On-line continuous
                                                   turbidity from each
                                                   filter.

  *Individual             Not applicable.          Settled water
  Sedimentation Basin                              turbidity less than 1
  Performance Criteria*                            NTU 95 percent of the
                                                   time when raw water
                                                   turbidity is less than
                                                   or equal to 10 NTU.

                                                   Settled water
                                                   turbidity less 2 NTU
                                                   95 percent of the time
                                                   when raw water
                                                   turbidity is less than
                                                   or equal to 20 NTU.

  *Individual Filter      Maximum filtered water   Filtered water is less
  Performance Criteria*   turbidity of 1 NTU in    than 0.1 NTU 95
                          two consecutive          percent of the time
                          measurements taken 15    (excluding 15 minute
                          minutes apart            period following
                          (conventional and direct backwashes) based on
                          filtration systems).     maximum values
                                                   recorded during 4-hour
                                                   increments.

                          Maximum filtered water   Maximum filtered
                          turbidity 4 hours        turbidity measurement
                          following backwash of    of 0.5 NTU.
                          less than 0.5 NTU in two 
                          consecutive measurements 
                          taken 15 minutes apart   
                          (conventional and direct 
                          filtration systems).     

                                                   Maximum filtered water
                                                   turbidity following
                                                   backwash of less than
                                                   0.3 NTU.

                                                   Maximum backwash
                                                   recovery period of 15
                                                   minutes (e.g., return
                                                   to less than 0.1 NTU).

                                                   Maximum filtered water
                                                   measurement of less
                                                   than 10 total
                                                   particles per
                                                   millileter (\>3m) of
                                                   particle counts are
                                                   available.

  *Combined Filtered      Representative           
  Water Performance       filtered/finished water  
  Criteria*               turbidity less than 0.3  
                          NTU 95 percent of the    
                          time based on 4-hour     
                          measurements             
                          (conventional and direct 
                          filtration systems).     

                          Maximum                  
                          filtered/finished water  
                          turbidity of 1 NTU based 
                          on 4-hour measurements   
                          (conventional and direct 
                          filtration systems).     

  *Disinfection           CT values to achieve     CT values to achieve
  Performance Criteria*   required log             required log
                          inactivation of          inactivation of
                          *Giardia* and viruses.   *Giardia* and viruses.
  ----------------------- ------------------------ ----------------------

## Components of a CPE

A CPE consists of the following three components:

-   Performance assessment (evaluates historical plant performance);

-   Major unit process evaluation (for assessing the physical plant
    capabilities); and

-   Factors limiting performance.

The following subsections discuss each of these components; detailed
procedures are provided in the CCP Handbook.

### Performance Assessment

The performance assessment component of the CPE determines the status of
a facility relative to achieving compliance requirements and performance
goals and verifies the extent of any performance problems at the plant.
This information also provides the CPE evaluators with some initial
insights on possible causes of performance problems. These insights are
then used to focus other activities during the CPE to assess the design,
operation, maintenance and administration of the plant. Historical
turbidity data from plant records is used, supplemented by data
collected during the CPE.

To achieve desired performance levels (compliance or optimized), a water
treatment plant should demonstrate that it can take a raw water source
of variable quality and produce a consistent, high quality
finished-water. Further, the perform­ance of each unit process should
demonstrate its capability to act as a barrier to the passage of
par­ticles at all times. The performance assessment determines if major
unit treatment proc­esses con­sistently perform at optimum levels to
provide maximum multiple barrier protection. If perform­ance is not
optimized, the assessment also provides valuable insights into possible
causes of the performance problems and serves as the basis for other CPE
findings.

During the performance assessment, historical turbidity data for the
raw, settled, and finished water is collected from the plant records and
trends are charted as shown in Figure 6-2. From this example data the
CPE evaluator can see that the plant treats a raw water that varies
moderately throughout the year. The settled and finished water
performance indicates that this plant has a performance problem since
turbidity levels produced for treatment processes are significantly
above compliance requirements and performance goals described in Table
6-1.

Figure 6-2 also shows how the CPE evaluator can use the performance
assessment to gain some insights into the causes of the poor
performance. In reviewing this data it is apparent that a spike in raw
water turbidity on March 9^th^ carried through the plant resulting in
finished water turbidities close to 1 NTU. These pass through variations
and spikes provide some insight into the root cause of these performance
problems that the CPE evaluators will use to direct the subsequent
portions of the CPE. Typically, these types of performance problems are
related to the process control skills of the plant staff, but other
design and/or administrative issues or raw water events may also make a
significant contribution to the problem. During their review of the
design, operation and administration of the plant, the CPE evaluators
will use these insights to focus the discussions they have with the
plant staff. Information on the possible causes of this spike will be
investigated until the evaluators are sure they understand the root
cause.

Additional data is collected during the CPE to confirm the historical
performance data, further assess the performance of individual treatment
processes, and confirm insights on possible causes of poor performance.
Typically additional data is collected through special studies including
the following:

-   Verification of filtered turbidity results by independently
    comparing a system's measurements with measurements from a
    continuous turbidimeter brought by the CPE evaluators. If the plant
    is not already individually measuring turbidity from each filter,
    the CPE team can select the filter which the operators believe has
    the most problems and collect individual filter data on that filter.

-   Filter inspections for media depth and media condition.

-   Filter media expansion during backwash.

-   Verification of chemical dosages to be sure plant staff are actually
    adding the amount of chemicals they are intending to add.

-   Verification of the benchtop turbidimeter in the plant laboratory
    with a unit brought by the CPE evaluators.

Additional data on the performance of individual sedimentation basins
may also be collected depending on the needs of the CPE evaluators.
Continuous monitoring of individual filters during the CPE allows for an
in-depth assessment of the filter performance during critical periods of
startup, backwash, and/or changes in plant flow rates. Figure 6-3 shows
the performance of a filter during a CPE immediately after start-up
following a backwash. Backwash spikes of this magnitude also indicate a
possible problem with the plant's process control procedures.

##  {#section .unnumbered}

##  {#section-1 .unnumbered}

##  {#section-2 .unnumbered}

##  {#section-3 .unnumbered}

##  {#section-4 .unnumbered}

##  {#section-5 .unnumbered}

##  {#section-6 .unnumbered}

##  {#section-7 .unnumbered}

##  {#section-8 .unnumbered}

##  {#section-9 .unnumbered}

##  {#section-10 .unnumbered}

[]{#__RefHeading___Toc448537624 .anchor}Figure 6-2. An Example of
Performance Assessment Using Historical Data

![](media/image7.wmf)

[]{#__RefHeading___Toc448537625 .anchor}Figure 6-3. An Example of
Individual Filter Data Collected During CPE

### Major Unit Process Evaluation

After the performance assessment, the CPE begins to focus on the causes
of the identified performance problems. The major unit process
evaluation determines if the various key existing treatment processes in
the plant, if properly operated, are of sufficient size to meet the
performance goals at the plant's current peak instantaneous operating
flows. If the evaluation indicates that the major unit processes are of
adequate size, then the opportunity for the existing facility to achieve
compliance by addressing operational, maintenance or administrative
limitations is available. If, on the other hand, the evaluation shows
that major unit processes are too small, then construction of new or
additional processes may be required to obtain compliance or optimize
performance.

The major unit process evaluation only considers if the existing
treatment processes are of adequate size to treat current peak
instantaneous operating flows and to meet the desired performance
levels. The intent is to assess whether existing facilities, in terms of
concrete and steel, are adequate. This evaluation does not review the
adequacy or condition of existing mechanical equipment. The evaluation
assumes that if the concrete and steel are not of adequate size then
major construction may be warranted, and the pursuit of purely
operational approaches to achieve performance may not be prudent. The
condition of the mechanical equipment around the treatment processes is
an important issue, but in this part of the CPE it is assumed that the
poten­tial exists to repair and/or replace this equipment without the
disruption of the plant inherent to a major construction project. These
types of issues are addressed in the factors limiting performance
component of the CPE. It is also presumed in the major unit process
evaluation that the necessary process control procedures are in place
and practiced to meet performance goals. By assuming that the equipment
limita­tions can be addressed and that operational prac­tices are optimum,
the evaluator can project the performance potential or capability of a
unit proc­ess to achieve performance goals.

> ![](media/image8.wmf)Flocculation criteria: Hydraulic detention time =
> 30 minutes; total volume = 202,500 gal; single stage, tapered
> flocculation
>
> Sedimentation criteria: Surface loading rate = 0.7 gpm/ft2; total
> surface area = 13,440 ft2; swd=15 ft
>
> Filtration criteria; Surface loading rate = 4 gpm/ft2; 6 filters in
> service; 30 inches mixed media
>
> Disinfection criteria; Total Giardia inactivation = 3 log, 0.5 log
> required by disinfection; available volume = 900,000 gallons @
>
> depth = 10 ft; pH = 7.5; temp = 0.5 C; chlorine residual = 1.5 mg/L;
> T10/T = 0.7

[]{#__RefHeading___Toc448537626 .anchor}Figure 6-4. Example Performance
Potential Graph

During the CPE, a performance potential graph similar to that shown in
Figure 6-4 is developed. The four treatment processes included in this
major unit process evaluation are flocculation, sedimentation,
filtration and disinfection. The CPE evaluators determine the peak
instantaneous operating flow that the plant has seen over the last year
and collect data on the sizes of the various basins. To prepare the
performance potential graph, the CPE evaluators should select loadings
for each process that they consider adequate for the plant to achieve
the performance goals. The assumptions and loadings used in this example
are shown at the bottom of the graph. Based on these loadings a
projected capacity is calculated and shown as a bar on the performance
potential graph. Bars above the dashed line in Figure 6-4 represent unit
processes that have the capacity to treat the peak instantaneous flow.
Bars below the dashed line indicate processes where major or minor
changes may be necessary.

### Factors Limiting Performance

The last and most significant component of a CPE is the identifica­tion
of factors that limit the filtration plant's per­formance. All
information collected during the CPE is reviewed and the root causes of
any performance problems are identified and prioritized. This step is
critical in defining the future activities that the plant will need to
focus on to achieve the compliance or optimized performance goals. To
assist in factor identification, a list of 50 different factors and
definitions that could potentially limit water treatment plant
performance is provided in the CCP Handbook. These factors are divided
into the four broad categories of administration, design, operation, and
maintenance. This list and definitions are based on the results of more
than 70 water treatment plant CPEs. Definitions are provided for the
convenience of the user as a reference to promote consistency in the use
of factors from plant to plant and to assist others in interpreting the
CPE results.

While the definitions for the administrative, operation and maintenance
factors adequately explain when these factors are identified, the plant
staff may find several of the design factors confusing when reviewing
the CPE findings. Design factors are included for each of the treatment
processes in the major unit process evaluation. If any of the treatment
processes in the major unit process evaluation were classified as
marginal or inadequate, they would be identified in the CPE findings as
a factor limiting the plant's performance. Treatment processes that were
identified as adequate in the major unit process evaluation can also be
identified as a factor when there are equipment related problems that
are limiting performance. This would occur when key equipment (e.g.,
filter rate-of-flow control valves) needs to be repaired and/or replaced
before desired performance can be achieved.

A CPE is intended to be a performance-based evaluation and therefore
factors should be identified only if they impact performance. A proper
CPE does not contain factors that are primarily observations that a
utility does not meet a particular "industry standard" (e.g., utility
does not have a documented preventive maintenance program or does not
practice good housekeeping) unless a clear link is made between the
practice and the identified performance problem.

The major challenge in identifying a plant's unique list of factors is
making sure that the root causes are identified. This is difficult
because the actual problems in a plant are often masked. This concept is
illustrated in the following example:

#### Example {#example .unnumbered}

A review of plant records revealed that a conventional water treatment
plant was periodically producing finished water with a turbidity greater
than 0.5 NTU. The utility, assuming that the plant was operating beyond
its capability, was beginning to make plans to expand both the
sedimentation and filtration unit processes. Field evaluations conducted
as part of a CPE revealed that settled water and finished water
turbidities averaged about 5 NTU and 0.6 NTU, respectively. Filtered
water turbidities peaked at 1.2 NTU for short periods following a filter
backwash.

Conceivably, the plant's sedimentation and filtration facilities were
inadequately sized. The major unit process evaluation, however, showed
that these processes were capable to handle the plant's current peak
flows.

A review of the plant's operation procedures revealed that the poor
performance was caused by the operator adding coagulants at excessive
dosages, leading to formation of a pin floc that was difficult to settle
and filter. The operators did not have an adequate process control
program or equipment to allow them to identify and set the proper
chemical doses. Additionally, the plant was being operated at its peak
capacity for only 8 hours each day, further aggravating the washout of
solids from the sedimentation basins.

The CPE evaluators assessed that by implementing proper process control
of the plant (e.g., jar testing for coagulant control, calibration and
proper adjustment of chemical feed) and operating the plant at a lower
flow rate for a longer time period would allow the plant to continuously
achieve the desired performance.

When the operator and administration were questioned about the reasons
that the plant was not operated for longer periods of time, it was
identified that it was an administrative decision to limit the plant
staffing to one person. This limitation made additional daily operating
time as well as weekend coverage difficult.

It was concluded that three major factors contributed to the poor
performance of the plant:

1.  Application of Concepts and Testing to Process Control: Inadequate
    operator knowledge existed to determine proper coagulant doses and
    to set chemical feed pumps to apply the correct chemical dose.

2.  Administrative Policies: A restrictive administrative policy existed
    that prohibited hiring an additional operator to allow increased
    plant operating time at a reduced plant flow rate.

3.  Process Control Testing: The utility had inadequate test equipment
    and an inade­quate sampling program to provide process control
    information.

In this example, pursuing the perceived limitation regarding the need
for additional sedimentation and filtration capacity would have led to
improper corrective actions. Completing a plant expansion without
correction of the operation and administrative factors probably would
not have solved the performance problems. The limitations in process
control would have remained even with a new plant. Administrative
policies that led to insufficient staffing of the old plant could have
remained with a new plant. The CPE, however, indicated that addressing
the identified operational and administrative factors would allow the
plant to achieve the desired performance on a continuous basis without
major expenditures for construction. The funds that initially were
directed towards construction could then be directed towards other
factors that truly are limiting the plant's performance.

This example illustrates that a comprehensive analysis of a performance
problem is essential to identify the actual performance limiting
factors. The CPE emphasis of assessing factors in the broad categories
of administration, design, operation, and maintenance helps to ensure
the identification of root causes of performance limitations.

##  Activities During a CPE

When a plant is required or decides to have a CPE conducted, there are
several activities that they should expect to occur. In general, if all
of the following activities do not occur, the plant should question
whether the evaluators are following the procedures in the CCP Handbook.

A CPE involves numerous activities conducted within a structured
framework. A schematic of CPE activities is shown in Figure 6-5. Initial
activities are conducted prior to on-site efforts and involve notifying
appropriate plant personnel to ensure that they, as well as other
neces­sary resources, will be available during the CPE. The kick-off
meeting, conducted on site, allows the evaluators to describe
forthcoming activities, to coordinate schedules, and to assess
availability of the materials that will be required.

Following the kick-­off meeting, a plant tour is conducted by the
superintendent or process control supervisor. During the tour, the
evaluators ask questions regarding the plant and observe areas that may
require additional attention during data collection activities. For
example, an evaluator might make a mental note to investigate more
thoroughly the flow splitting arrangement prior to flocculation basins
if one basin appeared to receive more flow than the other units (e.g.,
flooding).

Following the plant tour, data collection activities begin. Depending on
team size, the evaluat­ors split into groups to facilitate simultaneous
collec­tion of the administrative, design, operations, maintenance, and
performance data. Appropriate forms are provided in Appendix F of the
CCP Handbook to facilitate the data collection activities. After data
are collected, the performance assess­ment and the major unit process
evaluation are conducted. It is noted that often the utility can provide
the per­formance data prior to the site visit. In this case the
performance graphs can be completed prior to the on-site activities.
However, it is important to verify the sources of the samples and
quality of the data during field efforts.

Field evaluations are also conducted to continue to gather additional
information regarding actual plant performance and confirm potential
factors. This activity may typically include a special study focusing on
an individual filter or filters. Once all of this information is
collected, a series of inter­views are completed with the plant staff and
administrators. Initiating these activities prior to the interviews
provides the evaluators with an understanding of current plant
performance and plant unit process capability, which allows inter­view
questions to be more focused on potential factors.

After all information is collected, the evaluation team meets at a
location isolated from the utility personnel to review findings. At this
meeting, fac­tors limiting performance of the plant are identified and
prioritized. The prioritized list of factors, performance data, field
evaluation results, and major unit process evaluation data are then
compiled and copied for use as handouts during the exit meeting.

An exit meeting is held with appropriate operations and administration
personnel where all evaluation findings are pre­sented and the plant
staff are given the opportunity to ask questions. **The evaluation team
answers clarifying questions during the exit meeting but does not make
recommendations or offer solutions to the factors identified.** A CPE
report is then generated off-site by the CPE providers which formalizes
the information presented in the exit meeting**. It is intended that all
of the CPE findings**

<table>
<colgroup>
<col style="width: 1%" />
<col style="width: 14%" />
<col style="width: 5%" />
<col style="width: 13%" />
<col style="width: 2%" />
<col style="width: 3%" />
<col style="width: 13%" />
<col style="width: 3%" />
<col style="width: 2%" />
<col style="width: 0%" />
<col style="width: 12%" />
<col style="width: 0%" />
<col style="width: 0%" />
<col style="width: 1%" />
<col style="width: 3%" />
<col style="width: 0%" />
<col style="width: 9%" />
<col style="width: 3%" />
<col style="width: 0%" />
<col style="width: 8%" />
</colgroup>
<tbody>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"><h6 id="location" class="unnumbered">Location</h6></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="6">Initial Activities</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"><p id="off-site"
class="heading unnumbered">Off-Site</p></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="6">Kick-Off Meeting</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="6">Plant Tour</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="4"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="4"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td colspan="12">Data Collection Activities</td>
<td colspan="3"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td>Administrative Data</td>
<td></td>
<td>Design Data</td>
<td></td>
<td colspan="3">Operations Data</td>
<td></td>
<td colspan="3">Maintenance Data</td>
<td colspan="3"></td>
<td colspan="3">Performance Data</td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td>Conduct Performance Assessment</td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3">Evaluate Major Unit Processes</td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"><p id="on-site"
class="heading unnumbered">On-Site</p></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3">Field Evaluations</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3">Conduct Interviews</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3">Identify and Prioritize Factors</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3">Assess Applicability of CTA</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3">Exit Meeting</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3">CPE Report</td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"><p id="off-site-1"
class="heading unnumbered">Off-Site</p></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
</tbody>
</table>

[]{#__RefHeading___Toc448537627 .anchor}Figure 6-5. Activities During a
CPE

**are presented in the exit meeting and it is critical that the report
not present any additional findings**. The CPE provider should not save
any controversial findings for the report.

A CPE is typically conducted over a three to five-day period by a team
consisting of at least two personnel. A team approach is necessary to
allow a facility to be evaluated in a reasonable time frame, and for
evaluation personnel to jointly develop findings on topics requiring
profes­sional judgment. Professional judgment is critical when evaluating
subjective information obtained during the on-site CPE activities. For
example, assessing administrative versus opera­tional performance
lim­iting factors often involves the evaluators' interpretation of
inter­view results. The synergistic effect of two people making this
determination is a key part of the CPE process.

Because of the wide range of areas that are evalu­ated during a CPE, the
evaluation team needs to have a broad range of available skills. This
broad skills range is another reason to use a team approach in
conducting CPEs. Specifically, per­sons should have capability in the
areas shown in Table 6-2.

[]{#__RefHeading___Toc448537847 .anchor}Table 6-2. Evaluation Team
Capabilities

+-------------------------------+---------------------------------------+
| **Technical                   | ######## Leadership Sk                |
| Skills/Knowledge**            | ills {#leadership-skills .unnumbered} |
+-------------------------------+---------------------------------------+
| -   Water treatment plant     | -   Communication (presenting,        |
|     design                    |     listening, interviewing)          |
+-------------------------------+---------------------------------------+
| -   Water treatment           | -   Organization (scheduling,         |
|     operations and process    |     prioritizing)                     |
|     control                   |                                       |
+-------------------------------+---------------------------------------+
| -   Regulatory requirements   | -   Motivation (involving people,     |
|                               |     recognizing staff abilities)      |
+-------------------------------+---------------------------------------+
| -   Maintenance               | -   Decisiveness (completing CPE      |
|                               |     within time frame allowed)        |
+-------------------------------+---------------------------------------+
| -   Utility management        | -   Interpretation (assessing         |
|     (rates, budgeting,        |     multiple inputs, making           |
|     planning)                 |     judgments)                        |
+-------------------------------+---------------------------------------+

Regulatory agency personnel with experience in evaluating water
treatment facilities, consult­ing engineers who routinely work with plant
evalua­tion, design and start-up, and utility person­nel with design and
operations experience represent the types of personnel with appropriate
back­grounds to conduct CPEs. Other combinations of personnel can be used
if they meet the minimum experi­ence requirements outlined above.
Although teams composed of utility man­agement and opera­tions personnel
associated with the CPE facility can be established, it is often
difficult for an inter­nal team to objectively assess administrative and
operational factors. The strength of the CPE is best represented by an
objective third party review.

## CPE Quality Control

It is important for CPE providers and recipients of CPEs to be aware of
appropriate CCP concepts and expectations of the process. The providers
should maintain the integrity of the program and the recipients should
make sure they receive the full benefit of the CPE. This is accomplished
by fol­lowing the protocols described in the CCP Handbook. However, to
assure effective and consistent CPE results, quality control
considerations have been developed. Table 6-3 presents a checklist for
CPE providers and recipients to assess the adequacy of a CPE relative to
the guidance provided in the CCP Handbook. The following discusses some
of the key areas of concern in more detail.

[]{#__RefHeading___Toc448537848 .anchor}Table 6-3. Quality Control
Checklist for Completed CPEs

+-----------------------------------------------------------------------+
| **Checklist**                                                         |
+-----------------------------------------------------------------------+
| -   Findings demonstrate emphasis on achieve­ment of compliance and/or |
|     optimized performance goals (i.e., per­formance emphasis is        |
|     evident in the discus­sion of why priori­tized factors were         |
|     identi­fied).                                                      |
+-----------------------------------------------------------------------+
| -   Lack of bias associated with the provider's back­ground in the     |
|     factors identified (e.g., all design fac­tors identified by a      |
|     provider with a design back­ground or lack of operations or        |
|     administrative fac­tors identified by the utility personnel        |
|     conducting a CPE).                                                |
+-----------------------------------------------------------------------+
| -   Emphasis in the CPE results to maximize the use of existing       |
|     facility capability.                                              |
+-----------------------------------------------------------------------+
| -   All components of the CPE completed and docu­mented in a report    |
|     (i.e., performance assessment, major unit process evaluation,     |
|     identification and pri­oritization of factors, and assessment of   |
|     CTA appli­cation).                                                 |
+-----------------------------------------------------------------------+
| -   Fewer than 15 factors limiting performance identified (i.e.,      |
|     excessive factors indicates lack of focus for the utility).       |
+-----------------------------------------------------------------------+
| -   Specific recommendations are not presented in the CPE report, but |
|     rather, clear examples that support the identification of the     |
|     factors are summarized.                                           |
+-----------------------------------------------------------------------+
| -   Identified limitations of operations staff or lack of             |
|     site-specific guidelines instead of a need for a third            |
|     party-prepared operation and mainte­nance manual.                  |
+-----------------------------------------------------------------------+
| -   Findings address administrative, design, opera­tion and            |
|     maintenance factors (i.e., results demonstrate provider's         |
|     willingness to iden­tify/present all perti­nent factors).           |
+-----------------------------------------------------------------------+

A challenging area for the CPE provider is to main­tain the focus of the
evaluation on perform­ance and public health protection. Often, a
provider will tend to identify limitations in a multi­tude of areas which
may not be related to the per­formance criteria. Typical areas may
include poor plant housekeeping practices, lack of preventive
maintenance, or lack of an operation and mainte­nance manual. Limitations
in these areas are easily observed and do not challenge the capability
of the operations staff. While they dem­onstrate a thoroughness by the
provider to identify all issues, their identification may cause the
utility to focus resources on these areas while ignoring areas more
critical to achievement of performance goals. The evaluator should be
aware that a utility may take the CPE results and only address those
factors that are considered relatively easy to correct without
consideration of priority or the inter-relatedness of the factors.

Another significant challenge in conducting an effective CPE is the
tendency for providers to iden­tify limitations that are
non-controversial rather than real factors that may challenge the plant
person­nel\'s roles and responsibilities. For example, it is often easy
to identify a design limita­tion, since the utility could not be expected
to achieve desired performance with inadequate facilities. It is much
more difficult to identify "lack of administrative support" or an
operator's "inability to apply process control con­cepts\" as the causes
of poor performance. This may be especially a problem when the CPE
findings tend to criticize the administrators that have hired the CPE
providers. Failing to appropriately identify these difficult factors is
a disservice to all parties involved. **A common result of this
situation is the utility addressing a design limi­tation without
addressing existing administra­tive or operational issues.** Ultimately,
these administrative and operational issues remain and impact the
utility's ability to achieve desired per­formance. Understand­ing this
concept allows the CPE provider to pres­ent the true factors, even though
they may not be well received at the exit meeting. CPE recipients should
be suspicious when a plant has a performance problem and no operations
or administrative factors are identified.

A final consideration when implementing a CPE, is to understand the
importance that specific recommendations involv­ing plant modifications
or day-to-day operational practices should not be made by the CPE
provider or accepted without question by the recipient. For example,
direc­tion on changing coagulants or chemical dosages is not appropriate
during the conduct of a CPE. These types of changes should be evaluated
to determine if they are truly appropriate for the specific plant. A
coagulant that worked for the CPE provider at one plant may not work for
the plant being evaluated; causing unnecessary costs and/or poor
performance. There is a strong bias for providers to give specific
recommendations and for recipients to want spe­cific checklists to
implement. CPE providers should focus their observations during the
evalua­tion on two key areas:

1.  Identification of factors limiting the facility from achieving
    desired performance goals (compliance or optimized); and

```{=html}
<!-- -->
```
4.  Providing specific exam­ples to support these factors.

Recipients should, also, not request specific guidance from the
providers and, if this guidance is provided, they should make sure that
the information provided is truly appropriate to their plant.

## Next Steps

The results of the CPE provide systems and States with a thorough
evaluation of processes at a treatment plant. CPE results identify
factors which may be limiting performance and subsequently causing
compliance problems. The CPE affords systems the opportunity to achieve
improvements largely through administrative and operational changes.
Most systems can implement any necessary changes through a
self-improvement program, but if assistance is necessary facilities
should work closely with EPA, the States, and technical assistance
programs geared towards improving treatment plant performance.

The second phase of the CCP, the Comprehensive Technical Assistance
(CTA), may be used to improve performance in a more formal and
structured setting. During the CTA phase, the system, with assistance
from the State, identifies and systematically addresses plant-specific
factors. The CTA is a combination of utilizing CPE results as a basis
for follow-up, implementing process control priority-setting techniques,
and maintaining long-term involvement to systematically train staff and
administrators.

## References

1\. USEPA. 1998. *Handbook: Optimizing Water Treatment Plant Performance
Using the Composite Correction Program*. EPA/625/6-91/027.

6\. Comprehensive Performance Evaluation
[1](#comprehensive-performance-evaluation)

6.1 Introduction [1](#introduction)

6.2 Background On The CPE [1](#background-on-the-cpe)

6.3 Components of a CPE [4](#components-of-a-cpe)

6.3.1 Performance Assessment [4](#performance-assessment)

6.3.2 Major Unit Process Evaluation [7](#major-unit-process-evaluation)

6.3.3 Factors Limiting Performance [8](#factors-limiting-performance)

6.4 Activities During a CPE [10](#activities-during-a-cpe)

6.5 CPE Quality Control [13](#cpe-quality-control)

6.6 Next Steps [15](#next-steps)

6.7 References [16](#references)

Figure 6-1. Capable Plant Model [2](#__RefHeading___Toc448537623)

Figure 6-2. An Example of Performance Assessment Using Historical Data
[6](#__RefHeading___Toc448537624)

Figure 6-3. An Example of Individual Filter Data Collected During CPE
[7](#__RefHeading___Toc448537625)

Figure 6-4. Example Performance Potential Graph
[8](#__RefHeading___Toc448537626)

Figure 6-5. Activities During a CPE [12](#__RefHeading___Toc448537627)

Table 6-1. CPE Treatment Performance Goals
[3](#__RefHeading___Toc448537846)

Table 6-2. Evaluation Team Capabilities
[13](#__RefHeading___Toc448537847)

Table 6-3. Quality Control Checklist for Completed CPEs
[14](#__RefHeading___Toc448537848)