Abt Associates Inc., June 18, 2008 Adjusting the Phase 4 Facility Weights for the Cost Analysis 1

Environment and Resources

55 Wheeler Street, Cambridge, MA  Tel: 617 492-7100  www.abtassociates.com

Date June 18, 2008 

To Erik Helm and Josh Hall, U.S. EPA 

From Matt LaPenta, Tatiana Melnikova, and Michael Fisher, Abt Associates Inc. 

Subject Adjusting the Phase 4 Facility Weights for the Cost Analysis 

This memorandum proposes revised weights to be used for estimating the cost and economic impacts of 

with the 316(b) Phase II Remand Rule (P2RR weights). Abt Associates’ May 7, 2008 memorandum stated 

that we would like the cost analysis weights to provide accurate estimates – based on the full set of 543 

DQ/STQ facilities and using the previous Phase II STQ weights – for the following control variables: 

1. Generating capacity 

2. Number of facilities 

3. Design intake capacity. 

and according to the following classification variables: 

1. NERC region 

2. Capacity type (coal steam, combined cycle, etc.) 

3. Ownership (investor-owned, nonutility, etc.). 

These classification variables become the candidates for use in either a raking process or poststratification analysis for development of adjusted weights in accordance with a given control variable. 

1 Developing the Revised Weights 

We generated the proposed weights (P2RR weights) using a standard post-stratification approach: 

1. The set of strata are defined 

2. The weights are calculated so that the weighted estimates are equal to the control totals for each 

stratum (where the control totals are calculated using the original combined DQ/STQ sample 

weights (P2F-STQ)). 

We limited the weights development effort to facilities without an in-place cooling water recirculating 

system. Although facilities with a recirculating system may incur certain administrative costs under a 

regulatory option, they will not incur technology costs. As a result, the administrative costs to these 

facilities should be able to estimated and applied at the level of the individual DQ or STQ facility, and the 

weights that would be used with these recirculating system in-place facilities would be the DQ/STQ 

sample weights used in the Phase II Final Regulation analysis. 

memorandum 
Abt Associates Inc., June 18, 2008 Adjusting the Phase 4 Facility Weights for the Cost Analysis 2

In developing the revised weights for the cost and economic impact analysis, we faced several questions: 

3⁄4 Whether a single set of weights would provide sufficiently accurate estimates for each of the 

control variables 

3⁄4 Whether all of the classification concepts could be reasonably accounted for in developing the 

weights 

3⁄4 Whether to include information from the original Phase II Final DQ weights in the revised 

weights 

We discuss each of these questions below. 

Single set of weights would provide sufficiently accurate estimates for each of the control 

variables 

In developing the revised weights, we initially sought to develop a single set of weights that would be 

applicable for all three of the control variables listed above and according to the three classification 

variables. However, as we expected might occur, designing a single set of weights to meet adequately all 

of the above requirements proved not possible. 

As a possible approach to developing the “single” set of weights, we first developed weights for each of 

the three control variables and then looked to combine the weights in a way that would work “well 

enough” for all three control variables, at once. Each of the weight sets developed according to an 

individual control variable (e.g., generating capacity) performs well for that particular control variable but 

suffers deficiencies in the quality of estimates for the other control variables (e.g., design intake capacity). 

Our efforts to combine the individual weight sets (e.g., by averaging or other combination algorithm) to 

achieve a single set that performs well for all three of the control variables did not succeed. Specifically, 

we were able to develop combination weights that provide good estimates for generating capacity and 

number of facilities according to the NERC region classification variable; however, the population-level 

estimates for design intake capacity were substantially below the STQ weights-based estimates for that 

variable. Given the importance of design intake capacity in estimating some elements of compliance cost, 

we were concerned that the combination weights might not perform sufficiently well in the cost and 

impact analysis. 

Faced with this finding, we consulted again on the weights development possibilities with the Abt 

Associates statistics specialists, who recommended that multiple weight sets, developed according to the 

individual control variables, would be the best practical approach – if we believe it is important that the 

estimates for each of the control variables be “accurate.” These separate weights would be used to 

estimate specifically the values for the given control variable on which the weight set was developed, or 

for other variables that are linked closely in concept to a specific control variable. The obvious downside 

of the multiple weights approach is greater complication of the analysis. 

Whether all of the classification concepts could be reasonably accounted for in developing the 

weights 

During this process, we also observed that we would face difficulty in defining weights to account for 

more than one of the classification variables. Specifically, when we set up the multiple criteria cells, a 

number of the cells contained no DQ observations but did contain STQ observations that would need to 

be “represented” in the weighting framework. We first worked to combine cells in a rational way that 

would avoid the missing DQ observations, but eventually decided that we would need to compromise on 

the extent to which the revised weights would account for the proposed classification variables. 
Abt Associates Inc., June 18, 2008 Adjusting the Phase 4 Facility Weights for the Cost Analysis 3

Specifically, based on the classification preferences we outlined in our May 7, 2008 memorandum, we 

decided to focus on the NERC region classification as the concept for stratification. 

Whether to include information from the original Phase II Final DQ weights in the revised weights 

In developing the adjusted cost weights, we began by scaling the original DQ weights so that the new 

weighted estimate for a given control variable equals the population estimates (using the P2F-STQ 

weights) within each stratum. This approach retained the differential weighting information underlying 

the stratification framework for the original DQ weights. However, in some instances, this method 

produced anomalous weights – i.e., sample weights less than one – for one or more of the DQ 

observations in a stratum. Given this observation, we reviewed the stratification framework underlying 

the original DQ weights as documented by Battelle in its memorandum of March 26, 20081

 to assess the 

importance of retaining the original stratification information in developing new weights. According to 

the Battelle memorandum, traditional steam electric utilities were stratified by flow volume, SBA size, 

system type, and waterbody type. Steam electric nonutility power producers were stratified into three 

groups: (1) nonutility industrial; (2) nonutility nonindustrial; and (3) nonutility small. The first two 

nonutility strata were further stratified on the basis of flow volume. From this review, we judged that 

development of new weights without starting from the original weights would not result in substantial 

loss of information that would be important in the cost and economic impact analysis. Our judgments 

with respect to each stratification concept are as follows: 

3⁄4 Flow volume. Flow volume is an important concept in estimating compliance cost. However, we 

are explicitly capturing this concept in developing a set of weights that is based on design intake 

flow as a control variable. Moreover, from the Battelle memorandum, the definition of “flow 

volume” is not clear and may differ from design intake flow, which is the flow-related variable 

used in estimating some categories of compliance cost. Thus, the new weights should improve in 

capturing this cost-related information in the analysis. 

3⁄4 SBA size. The SBA size determination of facilities is a moving concept because of the change in 

ownership and entity size status of parent entities over time. Thus, the entity size information in 

the original DQ weights may now be incorrect for a number of facilities. In addition, we are 

dealing this concept explicitly and more appropriately in the firm impact analysis by developing 

firm-level weights that account for the known entity size classification of entities owning only 

STQ facilities (and thus are not captured in the DQ facility-based analysis)and entities that own at 

least one DQ facility (and thus are captured in the DQ facility-based analysis). Hence, we judged 

loss of this information in the building new DQ facility weights as not material. 

3⁄4 System type. Cooling water system type in-place is an important factor in determining the level 

and character of technology required for compliance with a regulatory option, and thus would 

appear important as a stratification concept. However, as we understand the original stratification, 

this variable was broken into only three classes: (1) recirculating system, (2) once-through 

system, and (3) “other.” We looked for, but couldn’t find, a clear definition for “other,” so we are 

not certain what information is capture by that classification. The proposed weights scheme will 

capture the important distinction between recirculating system in-place and once-through system 

in-place, because (1) we developed revised weights only for facilities without a recirculating 

system in-place and (2) we propose to use the Phase II Final DQ/STQ sample weights with DQ 

and STQ facilities that have a recirculating system in-place. 

 

1

 EPA Contract No. EP-C-05-030, Work Assignment No. 2-14: 316b Survey: Survey Weights Related to Costing and 

Benefits, memorandum to Marla Smith dated March 26, 2008. 
Abt Associates Inc., June 18, 2008 Adjusting the Phase 4 Facility Weights for the Cost Analysis 4

3⁄4 Waterbody type. Waterbody type can be a factor in determining compliance requirements, 

depending on the specifications for a given regulatory option. At this time, we are not accounting 

for waterbody type in the new weights, so this concept is the only previous stratification concept 

that we believe is of potential consequence in the analysis that is not being accounted for in the 

new weights. 

3⁄4 Business classification – utility vs. non-utility. Like the entity-size classification, this concept is 

subject to change over time. In addition, we view this concept of relatively low importance in 

assessing the cost and economic impact of the Remand Rule regulatory options. 

On balance, although we lose some information in developing the revised weights without consideration 

of the previous DQ weights, we view the revised weights as substantial improvements over the previous 

DQ weights, as demonstrated in the following section. 

2 Review of Estimates Based on P2RR Weights 

The following tables compare the population estimates, by NERC region, using the revised weights – the 

P2RR weights, by control variable – with the estimates based on the Phase II Final DQ weights (the P2FDQ weights). Each table reports the estimates for the specific control variable on which the particular 

P2RR weight set is based. In each table, the P2RR-based estimates are identical to the population-level 

estimates based on the combination of DQ and STQ facilities with the P2F-STQ weights, and 

substantially improve over the estimates based on the P2F-DQ weights. 

Table 1 presents the estimated number of facilities using the DQ-Only weights (P2F-DQ) and Facility 

Count-Based P2RR weights. The P2RR weights are assigned so that the weighted estimate of the number 

of facilities is equal to the control total number of facilities by NERC region. 

Table 1: Number of Facilities by NERC Region: P2F-DQ and Facility CountBased P2RR estimates (also equal to Facility Count Control Total) 

Number of Facilities Using 

NERC Region P2F-DQ Facility Count-Based P2RR 

ECAR 93 83 

ERCOT 44 36 

FRCC 24 20 

HICC 5 3 

MAAC 51 39 

MAIN 43 45 

MRO 40 41 

NPCC 58 60 

SERC 95 87 

SPP 15 19 

WECC 22 28 

Total 489 462 
Abt Associates Inc., June 18, 2008 Adjusting the Phase 4 Facility Weights for the Cost Analysis 5

Table 2 presents the estimated steam generating capacity using the P2F-DQ and Capacity-Based P2RR 

weights, where the weights are assigned so that the weighted estimate of the steam generating capacity is 

equal to the control total for steam generating capacity. 

Table 2: Steam Generating Capacity by NERC Region: P2F-DQ, and 

Capacity-Based P2RR Estimates (also equal to Generating Capacity 

Control Total) 

Steam Generating Capacity Using 

NERC Region P2F-DQ Capacity-Based P2RR 

ECAR 69,160 59,343 

ERCOT 39,577 31,293 

FRCC 29,755 19,973 

HICC 1,123 1,086 

MAAC 32,682 27,305 

MAIN 33,835 31,127 

MRO 19,353 16,060 

NPCC 40,569 42,042 

SERC 111,136 100,328 

SPP 9,961 9,874 

WECC 10,651 24,931 

Total 397,802 363,361 

Table 3 presents the estimated total design flow using the P2F-DQ and DIF-Based P2RR weights, where 

the weights are assigned so that the weighted estimate of the design flow is equal to the control total for 

design flow. 

Table 3: Design Flow by NERC Region: P2F-STQ, P2F-DQ, and DIF-Based 

P2RR Estimates (also equal to Design Flow Control Total) 

Design Flow Using 

NERC Region P2F-DQ DIF-Based P2RR 

ECAR 47,170,546,533 46,380,822,156 

ERCOT 39,208,628,064 28,588,275,746 

FRCC 22,678,886,534 18,203,990,663 

HICC 1,803,183,030 1,866,563,147 

MAAC 30,868,106,045 32,457,176,162 

MAIN 25,624,917,080 25,832,144,652 

MRO 14,114,904,604 15,293,644,526 

NPCC 25,316,081,421 43,728,623,136 

SERC 77,677,563,759 79,874,546,396 

SPP 5,350,950,001 9,161,351,687 

WECC 8,846,605,973 21,048,734,339 

Total 298,660,373,045 322,435,872,610