The New York Power Authority (NYPA) wanted to develop a 10-year schedule for refueling its Indian Point 3 Nuclear Power Plant (IP3) that balanced fish protection, which occurs when IP3 is shut down for refueling, and the costs of buying and loading fuel. We developed a decision analysis model to compare alternative strategies for refueling. It explicitly considered key uncertainties associated with future operation: how well IP3 operates, how long it takes to refuel, and when New York State is likely to deregulate the electric utility industry. The NYPA decision makers used the model to reinforce their choice of a refueling strategy. They were not surprised that more fish protection occurred with strategies that restricted the starting date for refueling to the third week in May, rather than allowing the starting date to float throughout the period from May through August. However, the decision makers were surprised that the more restrictive strategies also resulted in lower costs. T he New York Power Authority (NYPA) is the nation's largest nonfederal public power organization, providing about one-fourth of the electricity used in New York State. The NYPA owns 12 power projects. Approximately one-fifth of its electrical power is generated by the Indian Point 3 Nuclear Power Plant (IP3), lo-DUNNING ET AL. INTERFACES 31:5 122 cated on the Hudson River. Since 1975, IP3 has saved electric power users in Westchester County and New York City more than $1 billion. IP3 withdraws 840,000 gallons of water per minute from the Hudson River for cooling steam and then returns it to the river. When water is withdrawn from the river between February and September so are fish eggs and small fish. Some of the fish do not survive as they pass through the cooling water system of IP3 (entrainment) because of temperature increases, pressure changes, and shear forces. The effect of entrainment on fish populations in the Hudson River has been the subject of both extensive litigation and scientific research [Dunning et al. 2000; Barnthouse et al. 1988] . The NYPA can reduce the effect of entrainment by scheduling plant shutdowns to refuel IP3 when fish eggs and small fish are most abundant in the Hudson River and taking those outages as scheduled. In the past, the NYPA prepared its refueling outage schedules for IP3 using a 10-year planning horizon, assuming the operation and refueling of IP3 went exactly as planned. However, unforeseen events often altered operation and refueling, causing refueling outage schedules to deviate from what was planned. In the future, there is uncertainty about when New York State will deregulate the electric utility industry and the effect on the cost of replacement power that the NYPA will have to buy during refueling outages. NYPA must systematically consider these uncertainties to ensure that it can continue to provide low cost power and can reduce the environmental effects of operating IP3. For these reasons, the NYPA decided to use decision analysis rather than other approaches, such as mixed-integer programming [Fourcade et al. 1997] , to help it to schedule refueling outages at IP3. The decision analysis project started on October 23, 1997, and the NYPA required that it be finished by December 4, 1997. Our decision analysis approach consisted of four phases: framing, modeling, data collection, and evaluation. During the framing phase, we determined who would be involved in the project, the scope of the analysis, and the important factors (alternatives, uncertainties, and objectives) of the refueling outage plan. The project team consisted of a core group, subject matter experts, and decision makers. We served as the core group; our role was to conduct a decision analysis that was responsive to the needs of decision makers. We used various decisionanalysis tools to organize and record the results of the framing phase. These included a decision-quality spider diagram, a decision pyramid, an objectives hierarchy, a strategy table, a decision tree, and an influence diagram. We determined the scope of the project by developing a mission statement and by clearly defining the decisions and their boundaries. We constructed the mission statement by considering four questions: what are we going to do, why are we doing this, how will we know we are successful, and how can we fail? The resulting mission statement was as follows: -Identify strategies for scheduling refueling outages at IP3 over the period 1999 through 2008 that range from unrestricted NEW YORK POWER AUTHORITY September-October 2001 123 operation, in which outages can begin at any time and protection of fish eggs and small fish is not a concern, to restricted operation, in which outages must begin during a specified week and the primary concern is maximizing protection of fish eggs and small fish, -Develop a decision-analysis model for comparing the cost and amount of fish protection associated with the refueling strategies based on information provided by the experts at the NYPA and the level of confidence they have in that information, -Compare the refueling strategies in terms of cost and fish protection, and -Provide, by December 4, 1997, the results of the decision-analysis model to the NYPA decision makers. During the first week of the project, we listed decisions that could affect the mission. We then categorized each decision as policy, strategic, or tactical to help bound the NYPA's problem and focus our attention at the right level. A decision pyramid [Matheson and Matheson 1998 ] helped us to organize the three categories of decisions (Figure 1) . The strategic decisions were the focus of our project. The two key decisions were the time of year that refueling outages should occur and the amount of fuel that should be ordered for loading into the nuclear reactor core of IP3 at the start of an operating cycle to allow operation for a target number of days. Refueling outages for Cycles 10 through 14 were scheduled to occur in the years 1999, 2001, 2003, 2005, and 2007. The time of year that the outages occur affects the level of fish protection, while the amount of fuel loaded affects the cost of operation. Policy decisions represented corporate philosophy at the NYPA. They provided guidance for the decision analysis but were not to be evaluated in our study. The policy decisions were as follows: the NYPA would continue to operate IP3 for at least 10 years; it would refuel IP3 once during every two-year operating cycle; it would schedule refueling outages to last 50 days during the three consecutive twoyear operating cycles starting in 1999 (Cycles 10-12) and 35 days in Cycles 13 and 14 to allow for anticipated refueling efficiency improvements; and it would not schedule maintenance outages at IP3 in addition to the refueling outages. The tactical decisions were beyond the scope of our project. They represented alternatives that may have to be considered subsequent to the strategic decisions. By organizing the set of decisions into a decision pyramid, we benefited from a simple tool that focused our attention on the two important strategic decisions: timing of outage and amount of fuel loaded in each cycle. This prevented us from creating a large, unmanageable decision model that we could not finish on time. It also prevented us from solving the wrong problem. We identified the important issues related to scheduling of refueling outages and categorized them as objectives, strategies, or uncertainties. We organized our objectives into a hierarchy [Keeney 1992 ]. The fundamental objective was to maximize the overall benefit from planned refueling outages (Figure 2 ). This objective was composed of two subobjectives: to minimize the total cost of the outages and to maximize the fish protection that the outages provided. Although we could have included factors other than cost and fish protection in our analysis, these two were most important to the decision makers. Also, we thought adding further factors might prevent our completing the project by December 4. The cost objective was itself composed of three objectives: to minimize the cost of buying replacement electricity when IP3 was shut down for refueling, to minimize the amount of unused fuel in the reactor at the end of each operating cycle, and to minimize the amount of time that IP3 would operate at less than full power before the next refueling outage. The performance measure we used for the cost objective was net present value (NPV) of future costs for 10 years, using an annual discount rate of 6.5 percent. The NYPA has contracts to provide electricity to its customers. When IP3 is shut down for refueling, the NYPA must buy electricity to replace the electricity it is not producing. The NYPA operates IP3 so as to minimize the purchase of replacement electricity. Therefore, we included the cost of replacing electricity during refueling outages in our analysis. We did not include the cost of replacing electricity during unscheduled outages because we assumed that it would be the same for any refueling strategy. Before each cycle, fuel is loaded into the nuclear reactor core of IP3 to allow operation for a target number of full power days (FPDs). If a refueling outage occurs before all of the fuel is used, some of the remaining fuel cannot be used for future operation. The NYPA strives to minimize the amount of unused fuel and thus the cost of buying fuel that is not used for producing electricity. The unused-fuel cost is the expense of not using all of the FPDs. If IP3 operates at full power for a greater number of days than expected, the fuel will not last until the next scheduled refueling outage. Starting a refueling outage before it is scheduled creates problems NEW YORK POWER AUTHORITY September-October 2001 125 that the NYPA does not consider acceptable. Rather than start a refueling outage earlier than scheduled, the NYPA can extend the operation of IP3 to the scheduled starting date by reducing power generation from 100 percent to 70 percent in the 20 days before the outage, a process known as coast down. The cost of coast down is the penalty associated with running IP3 at less than full power, measured by the cost of replacement power needed to compensate for reduced power production. The performance measure for fish protection is the sum of the average percent reduction (APR) in the conditional entrainment mortality rate (CEMR) at IP3 across five taxa (types) of fish over the 10-year period from 1999 through 2008. CEMR is the fractional reduction in abundance of a fish taxon in the Hudson River due to entrainment of fish eggs and small fish, assuming other sources of mortality are density independent, that is, there is no compensatory increase in survival or growth that would offset entrainment mortality. We considered five taxa of fish in this project: white perch, striped bass, river herring, bay anchovy, and Atlantic tomcod. Prior to the start of this study, the stakeholders (the NYPA, state and federal regulators, and environmental advocacy groups) jointly selected these five taxa to evaluate the effects of operating power plants on the Hudson River [Coastal Environmental Services, Inc. 1996]. Eggs and small fish of these five taxa are found in the Hudson River near IP3. We defined the APR in CEMR as the simple average of the annual percent reductions for the five taxa of fish. The APR in CEMR for each taxon is the annual CEMR at IP3 resulting from plant shutdown during an outage divided by the maximum annual CEMR assuming no outage occurred. Using this definition, each taxon contributes equally to the annual average, regardless of the number of fish of that taxon in the Hudson River, that is, taxa are inversely weighted by their abundance. This inverse weighting was selected by default after the stakeholders were unable to agree on a system for prioritizing taxa based on their relative abundance or other criteria. However, our analysis was intended to address the perceived need for fish protection agreed to by the stakeholders, not to debate the merits of their decisions. We summed the APR in CEMR at IP3 across the 10-year period of a refueling strategy to estimate the cumulative APR (CAPR) in CEMR. We did not discount APR in CEMR as we did with total cost because we assumed that fish would have the same value in future years. The maximum annual CEMR at IP3 differs among taxa as do the weekly contributions to the annual CEMR (Figure 3) . The percent reduction in CEMR for a taxon will be highest if an outage occurs during the weeks that contribute most to the maximum annual CEMR. However, those weeks differ among taxa. The weekly contributions for all taxa occur within a 32-week period (window) from week 7 (the second week in February) to week 38 (the second week in September). Thus, a 32-week outage scheduled from week 7 to week 38 would result in an APR in CEMR of 100 percent; all other outages DUNNING ET AL. INTERFACES 31:5 126 would result in a value less than 100 percent. If no outages occurred during that 32-week period, the APR in CEMR would be zero percent. For short-duration outages, moving the starting week of the outage to late spring will produce a relatively higher APR in CEMR. The highest percent reduction in APR of CEMR for a 35-day or a 50-day outage occurs if the outage begins during the last full week in May. We used strategy tables [Clemen 1996 ] to organize the refueling strategies we developed and selected five that represent a range of operating restrictions for further analysis: unrestricted operation, 32-week, 16-week, fixed 2001, and fixed 1999. In the unrestricted-operation strategy, refueling outages are scheduled every two years, but actually occur at the end of fullpower operation, whenever that happens. Thus, the starting date of the outage does not necessarily adhere to the schedule. This strategy represents how IP3 has operated in the past and serves as a baseline for comparison with the other strategies. In the 32-week strategy, NYPA schedules refueling outages within the 32-week period when fish eggs and small fish of all five taxa would be entrained, that is, a window of weeks 7 to 38 (Figure 3) . A refueling outage for the 32-week strategy starts at the end of full power operation or as close to the end as possible with the constraint that the entire outage occurs within the window. That is, refueling outages start from week 7 through week 31 for a planned 50-day outage and from week 7 through week 33 for a planned 35-day outage. This is the second least restrictive strategy and is intended to provide NEW YORK POWER AUTHORITY September-October 2001 127 more fish protection than the unrestrictedoperation strategy. In the 16-week strategy, NYPA schedules refueling outages within a 16-week period beginning with week 21. Refueling outages scheduled to last 50 days can start in weeks 21 through 29 and those scheduled to last 35 days can start in weeks 21 through 31. During this 16-week period, more fish eggs and small fish would be entrained than in the remaining 16 weeks of the 32-week period. This strategy is more restrictive because the window is half the size of the 32-week strategy and is intended to provide greater fish protection. In the fixed 2001 strategy, the 1999 refueling outage was expected to start near the beginning of September 1999 (depending on how well IP3 operated). Subsequent refueling outages are constrained to start during the last full week of May, beginning with the 2001 refueling outage and then in odd years thereafter. The fixed 2001 strategy is more restrictive than the 16-week strategy and is intended to provide greater fish protection. This strategy allows IP3 to operate until the end of FPDs in 1999, starting a refueling outage then, and imposing a strict outage window for future outages. The fixed 1999 strategy differs from the fixed 2001 strategy only in that the 1999 refueling outage was expected to start during the last full week of May 1999 rather than in September 1999. This strategy is more restrictive than the fixed 2001 strategy and is intended to provide maximum fish protection for the 10-year planning horizon with the planned outages. The fixed 1999 strategy stops operation of IP3 before the end of FPDs in 1999, but it also provides maximum fish protection immediately. We identified three uncertainties that were likely to have the largest effect on total net present value (NPV) cost for the five refueling strategies: how well IP3 operates (operating factor), how long it takes to refuel IP3 (outage length), and when New York State is likely to deregulate the electric utility industry (Figure 4) . The two key strategic decisions concern the timing of the refueling outages and how much fuel to order for the next cycle (Figure 1 ). Since their timing determines the fish protection outages provide, we addressed the alternatives explicitly in a decision tree ( Figure 5 ). We modeled the amount of fuel loaded at the beginning of each cycle as a calculated value in the cost model, assuming that IP3 operated at full power in that cycle except during the refueling outage. For each refueling strategy, we computed total cost (NPV) and fish protection (CAPR in CEMR) for all combinations of uncertainty states. We then weighted each scenario through the tree by its probability, resulting in a distribution of total cost and fish protection for each refueling strategy. We used the expected values of the resulting probability distributions to compare refueling strategies. We computed these scenarios with DPL [ADA Decision Systems 1995] linked to an Excel spreadsheet. We used the simulation and codeconversion features of DPL to reduce our run time on a 486 machine from approximately 30 minutes for complete enumeration to less than two minutes with negligi-DUNNING ET AL. INTERFACES 31:5 128 . We also fixed the Order i date since it is 396 days before the scheduled refueling outage date (RO i date). We divided the cycle into two periods, one before the Order i date (DaysBefore i ) and one after that date but before the RO i date (DaysAfter). IP3 can actually shut down for refueling (OutStartAct i ) before, after, or on the RO i date, depending upon how well it operates relative to its target. If the actual operating factor falls below the target operating factor for a cycle, IP3 will continue to operate beyond the scheduled refueling date until it must shut down to stay within the window of a refueling strategy. If the actual operating factor exceeds the target operating factor, it will have to shut down before the scheduled refueling date if it maintains full power throughout the cycle. In this case, IP3 operated more efficiently than expected and can continue to operate only at a reduced power level (derated). If IP3 does not coast into the refueling strategy's window, then it must shut down early and wait until the planned outage begins. protection for each scenario of the decision tree for any setting of model variables. The appendix contains additional details on the model variables and logic. We collected data on the operating factor, the outage length, the expected year for deregulation, and replacement-power cost from experts in the NYPA, including core group members, senior managers, and nuclear plant operators. For most of the uncertainties, we used a six-step probability assessment process [Merkhofer 1987 and Spetzler and Staël von Holstein 1975] to encode probability distributions from experts. The process consisted of the following steps: (1) Motivating, in which we described the DUNNING ET AL. INTERFACES 31:5 130 assessment task and its importance to the subject-matter experts, and counteracted motivational biases; (2) Structuring, in which we clearly defined the variables, assumptions, and measurement scales; (3) Conditioning, in which we drew out the subject-matter expert's knowledge concerning the variable and counteracted cognitive biases [Russo and Schoemaker 1990 and Kahneman, Slovic, and Tversky 1982] ; (4) Encoding, in which we quantified the variable's uncertainty as a cumulative probability distribution by assessing extreme values (first and 99th percentiles) and five to 10 values in between the extreme values; (5) Verifying, in which we confirmed that the distribution accurately reflected the beliefs of the expert; and (6) Discretizing, in which we selected the 10th, 50th, and 90th percentiles of the encoded cumulative probability distribution. The NYPA establishes target operating factors and target outage lengths for each cycle as part of its business plan. However, it cannot forecast actual operating factors and actual outage lengths for each cycle with complete certainty. To reflect this uncertainty, we encoded values for the actual operating factor and outage length that the operators of IP3 and other NYPA experts expected, and their confidence in those values. From the resulting cumulative probability distribution, we selected the 10th, 50th, and 90th percentiles for use in the decision tree. We conducted each probability assessment in approximately three hours using the six-step process. For all cycles, the likelihood of operating at an operating factor lower than the target is greater than that of operating at or above the target. As a result, to maintain a predetermined schedule, the NYPA has a higher probability of discarding unused fuel than of running out of fuel and coasting down. The probability assessment for outage length indicates that the NYPA experts may be overconfident because the spread in the distribution narrows in the future (Table 1) . Generally, experts are less sure of events far into the future than they are of near events. In addition, they may show motivational bias because the actual values of operating factor and outage length Operating Factor (%) Outage Length (days )   Cycle  10th  50th  90th  Target  10th  50th  90th  Target   10  80  85  92  90  50  65  90  50  11  83  88  95  90  40  55  70  50  12  85  90  97  95  35  50  65  50  13  85  90  97  95  35  45  60  35  14  85  90  97  95  35  40  50  35   Table 1 move toward target values. We made the NYPA experts aware of these biases, but they stood by their assessments because they expect IP3 to become more efficient in the future. Even though the NYPA could not forecast the precise timing of deregulation of the electric utility industry, we did not use the six-step probability assessment process because of time constraints. Instead, we assessed the 10th, 50th, and 90th percentiles directly from the core group, resulting in years 1999, 2000, and 2002. Approximating the probability distribution of a continuous variable with these three percentiles is common in practice. For future revisions of the analysis, we recommend a more thorough review of the factors surrounding the deregulation year and resulting purchase power prices. Before deregulation, the NYPA forecasts that replacement-power cost will continue to vary monthly (Figure 7) , It expects the highest values from November through March. It also expects higher values during July and August than in April, May, June, September, and October. We thought it impractical to conduct probability assessments on replacement-power cost for each month of each cycle. The electric utility experts we interviewed, both within and outside of the NYPA, believed that the current seasonal differences in replacement power are likely to diminish in the area IP3 serves as energy providers try to capitalize on the higher price that they can get for power during certain seasons. As the supply of electricity increases to meet demand during the higher-priced seasons, prices will come down. Precisely how much seasonal replacement-power costs will change will be known only after deregulation. For our analysis, we assumed that after deregulation the cost of replacement power would be constant throughout the year. We also assumed that the cost of replacement power after deregulation would be equal to the average across all months before deregulation. As a result, the month during which a refueling outage starts is a factor in determining the replacement power contribution to total NPV cost before, but not after, deregulation. We also assumed that the cost of replacement power after deregulation would not escalate during our 10-year planning horizon to reflect the price control that deregulation is expected to provide. We performed analyses to explore the performance of each strategy against the two primary objectives of cost and fish protection. We presented the results to decision makers and gave compelling evidence that highlighted the key drivers of the results and the differences between the DUNNING ET AL. INTERFACES 31:5 132 strategies. Although the analysis accounted for the range of possible outcomes for each strategy, in our presentation we emphasized the expected values of the outcome distributions. The rank order of the refueling strategies differs depending on the performance measure. No single strategy simultaneously minimizes the expected NPV cost and maximizes the expected CAPR in CEMR (Figure 8 ). Early in the project, we tried to convince the NYPA decision makers to specify the trade-offs between cost and fish protection but were unable to do so. Rather than force this, we recognized that we could infer this trade-off later since the analysis was limited to two objectives. We presented the rankings of the strategies to the NYPA decision makers without explicitly weighting cost and fish protection. The 16-week and 32-week strategies were dominated by the fixed 2001 strategy. Therefore, we compared the remaining three strategies. Unrestricted operation cost $5.3 million less than the fixed 2001 strategy because the savings associated with operating until the end of FPDs outweighed the cost of buying more expensive replacement power. However, it provided fish protection well below a standard previously accepted by the NYPA, a CAPR in CEMR of 204 based on the Hudson River Cooling Tower Settlement Agreement [Barnthouse et al. 1988 ]. Fish protection above this standard was provided by both the fixed 2001 and the fixed 1999 strategies. The NYPA decision makers chose the fixed 2001 strategy because it met the previously accepted standard for fish protection at a cost savings of nearly $10 million. Despite the absence of explicit weighting among objectives, the NYPA decision makers drew on the insights from the decision analysis modeling to reinforce their choice of a refueling strategy. The decision analysis provided new insights for scheduling refueling outages at IP3. Historically, the NYPA decision makers believed that strategies allowing refueling outages to start within a fixed period of many weeks were better than those that required refueling outages to start within a particular week. They wanted refueling outages to start when scheduled, and past refueling outages had often started at times very different from those initially scheduled. For the future, it seemed reasonable to assume that the probability of starting refueling outages at the end of FPDs within a fixed period of weeks was considerably higher than the probability of starting them during a particular week. However, the 16-week and 32-week strate- gies did not guarantee that IP3 could operate until the end of FPDs. These strategies also resulted in two of the five outages that occurred during weeks when replacement power was higher than during the weeks associated with the fixed 2001 strategy. Another insight from the analysis was that operators and decision makers differed in their forecasts of operating factors and outage length durations, an insight that fueled discussion and attempts to resolve these differences. Finally, even though NYPA decision makers had been comfortable with their assumptions about replacement power, they were surprised at how much replacement power affected the results of the analysis. Consequently, they are considering further study of replacement power. The NYPA accepted our decision-analysis model as a major improvement in the approach it used for scheduling outages to refuel IP3 because the model accounts for key uncertainties in the refueling schedule and provides a systematic approach for simultaneously evaluating the cost of refueling outages and fish protection. It also provides the NYPA with a tool for explaining its selection of a refueling outage schedule to other stakeholders. As a result, the NYPA used our model in developing a schedule for refueling outages at IP3 over the 10-year period 1999 through 2008. The approach we used is generally applicable to other electric utilities that must evaluate options for operating power plants when faced with competing objectives and significant uncertainty. We thank the two reviewers and the associate editor for improving the quality of this manuscript. We also thank Craig Kirkwood and William A. Harkins for reading an earlier draft. The appendix contains the details of the quantitative model that computes cost and fish protection for any scenario in the decision tree, including the equations and variable definitions that describe the underlying cost model. We implemented the cost model in an Excel spreadsheet and evaluated the scenarios for each strategy with DPL linked to Excel. DaysBefore i is the number of days in cycle i before the Order i date, and it depends on the scheduled refueling outage date for cycle i (RO i date), when the previous cycle ended (OutEndAct i‫מ‬l ), and the number of days in cycle i after the Order i date (DaysAfter):
Nuclear maintenance and refueling is mentioned by Dunning et al. in REF , but the problem considered is to minimize the environmental impact.
New York Power Authority Uses Decision Analysis to Schedule Refueling of Its Indian Point 3 Nuclear Power Plant
Interfaces
Interfaces
Computer Science