## Roadmapping Process Improvements by Experience at the Idaho National Engineering Laboratory' High Level Waste Program and Synergistic Interfaces with Decision-Making

By

James Murphy

# Arlin Olson

Idaho National Engineering and Environmental Laboratory (INEEL)

## 

## Abstract

Six technology roadmaps were developed for various technologies under
consideration for the treatment of sodium bearing liquid and calcine
wastes. In the process of creating these roadmaps, a number of process
improvements were identified for each of the formal roadmapping phases
as described in the Department of Energy's draft roadmapping guidance.
The lessons learned, presented as beneficial improvements to the Idaho
National Engineering and Environmental Laboratory (INEEL) High Level
Waste Program, are proposed to be added to the draft guidance.
Additionally, synergistic interfaces between the roadmapping and
decision-making processes were observed and reported on. With these
improvements, technology roadmapping has become an effective integration
tool at the INEEL for planning technology development.

## Introduction

The High Level Waste (HLW) Program at the Idaho National Engineering and
Environmental Laboratory (INEEL) has treated acidic HLW liquid using a
solidification process called calcination. The solid product is stored
temporarily in stainless steel bins awaiting final treatment for
ultimate disposition to a federal repository. The acidic liquid is
temporarily stored in stainless steel tanks until it is treated.
Although the "calciner" was effective in converting acidic liquid HLW to
solids, the effectiveness was significantly reduced when all the liquid
HLW was calcined and only acidic decontamination-type solutions called
sodium bearing waste (SBW), remained in the tanks. Added to the
reduction in effectiveness was the applicability of the federal maximum
achievable control technologies (MACT) regulations. With the new
regulations, a major upgrade project would be required to continue the
use of the calciner. An evaluation was conducted to decide if some other
alternative was more desirable than upgrading a twenty-year-old
facility. The alternatives evaluation was conducted formally as part of
an Environmental Impact Statement (EIS)^1^. The EIS scope included both
the SBW remaining in the tanks and the calcine in the solids storage
bins.

The several year EIS process was increasing the difficulty to be able to
complete the treatment of all the liquid SBW stored in the tank farm by
the Department of Energy and the State of Idaho Settlement Agreement
milestone of 2012. If a major project were required to make processing
possible by this date, a final technology choice, plus an expedient
start of the project would be essential. The milestones associated with
the solid HLW calcine stored in the bins has a Settlement Agreement
completion date of 2035, leaving a little more time to choose and
develop the technologies. However, since the wastes have common chemical
characteristics, potential integration efficiencies might exist.
Therefore, the technology alternatives were evaluated such that the
waste streams could be integrated into a common facility or processed
separately.

The EIS was on course for a late fiscal year 2000 selection of the
preferred alternative for treatment of the SBW. However, the urgency to
get the right technology developed caused discomfort with the HLW
program baseline schedule. In an attempt to get a head start on the
development needed, a technology roadmap was requested prior to the
technology selection. This left the option to roadmap the candidate
technologies or pick one knowing that it might be the wrong choice. DOE
decided to ask the contractor to look at the alternatives and available
discriminating data to see if a technology choice was obvious. This
pre-analysis would help limit the scope that needed to be roadmapped to
something manageable, decreasing the chance of having to redo it later.
Thus, the contractor began an alternatives evaluation^2^ that would then
lead to a roadmapping effort. The alternatives evaluation was to be
completed prior to DOE's formal review so that DOE could take advantage
of any data and discriminators identified by the contractor.

The technology evaluation was completed over about three months with
twelve people working full-time. The decision support process was
performed by several of the same people who later performed the
technology roadmapping. It was this coincidence that helped identify a
number of interfaces between the roadmapping and decision support
processes that will be shared in this paper.

Additionally, six separate technology flowsheets have been roadmapped to
date; three for SBW^3^ and three for calcine^4^. Many significant
"lessons learned" have been identified over this time. These
improvements have increased the effectiveness of roadmapping as well as
the efficiencies in creating roadmaps. The improvements in the
processes, coupled with the synergistic interfaces between
decision-making and roadmapping, have made the concept of roadmapping an
extremely useful tool to INEEL's technology development.

The first two SBW technology roadmaps took about \$300 K and three
months. The third SBW technology roadmap cost \$190K and took eight
weeks. These were followed by three calciner flowsheet roadmaps that
were developed in parallel for a total cost of \$150K, taking six weeks
to complete. These costs and times demonstrate the value of some of the
process improvements that are shared in this paper. In addition, some of
the roadmapping products that were found to be useful are discussed.

The roadmapping process used by the HLW program closely followed the
draft DOE guidance provided to Environmental Management science and
technology programs.^5^ The guidance provides a basic four phase process
namely, initiation, technical needs, technical responses, and technology
implementation. The process and product improvements found over the past
year is discussed relative to the phase of the process these
improvements are focused towards. These discussions are followed by the
discussion of the synergistic interfaces between decision-making and
roadmapping processes.

## Phase I Improvements - Roadmapping Initiation

During Phase I, the scope of the roadmap is defined. A few key tools
were found to help keep the scope of the roadmap manageable. The draft
guidance recommends defining the charter, mission, system boundary,
scope, product definition, and team participants prior to actually
creating the roadmap. In the course of performing the six roadmaps noted
earlier, a few additional considerations were found to be valuable in
defining the scope.

There were two purposes for HLW program technology roadmaps created over
the last year. The first purpose was to help select a technology for
implementation. The second purpose was to determine what development was
necessary to select a technology. The difference in these two purposes
has a great impact on the level of detail included in the roadmap. In
the case where a technology is being implemented, the technical needs
and risks can be prioritized by the project phases. Definition and
resolution of the highest risks is desirable early in the project
design, whereas lower risks can be dealt with later. In the case of
technology selection, the research and development is focused on
demonstrating that the technology can work and needs to include enough
preliminary performance data that shows that the new technology can
compete with existing technologies. Any technology alternative that
lacks data to demonstrate application and performance will not likely be
selected.

By knowing the purpose, the decision being affected can be defined.
Every program implementing a technology passes through decision points.
A project has pre-conceptual phases that are used to evaluate multiple
alternatives and a conceptual design phase where a selected technology
is adequately designed to understand the construction and operational
aspects. Additionally, other decision points for construction and
operations exist. As the program or project moves through these phases,
risk should be decreasing. It is desired to resolve the highest risks
first, followed by intermediate risks, followed by lower risks. Knowing
the phase of a project helps to focus on the right set of technical
needs that would support the program or project's next decision point.
For example, development directed to resolving operational optimization
needs should be a minor concern during a technology selection when large
needs or uncertainties that could result in technology failure or major
cost escalation exist. By defining the project decision points that
technology development can support, the focus of the development
activities can be commensurate with the needed risk reduction. This will
be further discussed in a prioritization step under the Technical
Response phase. If the decision point is left undefined, the roadmap
scope could be much larger than needed to support the project and this
could lead to schedule delays, cost overruns, or lack of responsiveness
to the project needs.

In summary, the suggested improvements to the roadmapping initiation
phase are:

1.  Identify the purpose of the roadmap at the beginning; and

2.  Define which decision points the roadmap needs to influence during
    this phase of roadmapping.

## Phase II Improvements - Roadmapping Technical Needs

Technical needs vary in importance. The needs, as identified by the
technical experts, were found to have a great deal of subjectivity. As
these needs were identified, it became important to create a process to
decrease the subjectivity. There was some disagreement as to what was a
need and how to identify it consistently between technologies. There
were also issues recognizing what were assumptions versus facts.

The first improvement was to change the term "need" to "uncertainty" and
define it in terms of lack of information or data that could lead to a
potential consequence that would have either positive or negative
effects on the design or operation of the technology. A consequence was
identified instead of assigning a risk category to avoid the
subjectivity of evaluating probabilities. This definition helped clarify
what was an uncertainty and what wasn't. Levels of consequences were
useful in a prioritization method performed as part of Phase III
described in the Technical Response section.

Many of the uncertainties that were found were associated with the
technologies in the supporting unit operations. Often, the main process
technology gets most of the attention but it was not where most of the
uncertainty was found.

Using the uncertainty definition, flowsheets were evaluated for areas
where there was a lack of knowledge. The potential consequence for that
lack of knowledge was documented by the team to help determine the level
of importance of the uncertainty. Consequence statements helped define
each uncertainty, examining what was [not]{.underline} known about each
unit operation's performance, specific applications, and interfaces
between unit operations. Uncertainties with a large potential
consequence were often discovered while evaluating smaller
uncertainties. It was important to not limit the identification of
uncertainties during this phase but to be sure to describe them in terms
of potential consequence.

In summary, the suggested improvements to the roadmapping technical
needs phase are:

1.  Define needs in terms of uncertainties where the potential
    consequence is documented;

2.  Ensure data availability or the issues are uncertainties. Avoid
    accepting long-standing "assumptions" that have no data backup as
    "fact";

3.  Look for uncertainties throughout the flowsheet especially in the
    supporting technologies; and

4.  Do not limit the uncertainty identification to the current decision
    point being supported by the roadmap. Otherwise, important
    uncertainties might not be identified.

## Phase III Improvements - Roadmapping Technical Responses

The problem with attempting to identify all the uncertainties is the
amount of work it takes to define the technical responses for them. The
technical responses were defined by the technologists or engineers that
would need to perform the work to resolve the uncertainty. Technology
and engineering personnel created, in early roadmaps, a response for
each and every uncertainty. There were two major problems with this
approach, time and money. The amount of scope identified would take
years to complete and cost too much. There were two improvements
identified to help these technical responses to be worthwhile.

The first improvement was to prioritize the uncertainties. Using the
potential consequence, uncertainties were placed into viability,
footprint, and optimization categories. Viability uncertainties had a
potential consequence that could lead to failure of a technology to meet
the functional requirements of the system. The footprint category were
those that have a potential consequence of changing the size of the
major unit operations and effectively, changing the implementation cost.
The third category, optimization uncertainties, were those that may
affect how the technology is operated and other small changes but would
not influence the viability or largely affect the implementation costs.
These categories were selected because they lined up well with the major
project decision points. The viability uncertainties usually must be
resolved before a technology can be seriously considered or selected.
The footprint-type uncertainties must be resolved to properly baseline
the project cost, and the optimization uncertainties support operations.

The second improvement was to group the uncertainties into common
problem statements and technical responses. An evaluation of the
uncertainties and grouping by a problem statement yielded technical
responses that resolved more than one uncertainty. Prioritizing and
grouping the uncertainties defined the right scope and yielded technical
response tasks whose costs were less than expected.

As the roadmaps were developed, it was unclear as to what relationship
information needed to be maintained. Each roadmap was basically a series
of activities to resolve uncertainties. Each activity was placed into a
project schedule. The schedule logic and resource needs were kept with
each activity. Initially, it was difficult to ascertain the
uncertainties and potential consequences and to define the problem
statement and relationships. Each roadmap continued to improve upon
these data and interrelationships. The final roadmap included
improvements in maintaining the data structure that is needed as well as
resource loading the activities in the schedule.

The data structure for the calcine roadmap has been most useful in
capturing the important relationships thus far and is shown in Figure 1.
The project decision points, defined in the roadmap initiation phase,
became the main organization points. In this example, the technology
selection decision was planned using three critical decision points,
numbers 1-3. These decision points represent the points in time when a
program or project must have certain information to be able to move
forward. By defining these specific decision points, the related
uncertainties can be prioritized. Listed under each decision are each of
the relevant unit operations of a flowsheet. All unit operations were
included, even those without uncertainties, for completeness and
understandability. Within each unit operation, the prioritized
uncertainties are connected to the technical responses that resolve
them. If more than one activity is needed to resolve the uncertainty,
schedule logic ties are used to keep the tasks in the correct order.

![](media/image1.wmf)

Figure 1. Calcine Roadmap Example

The SBW roadmap was focused on resolving the uncertainties needed to
support conceptual design and to establish the cost baseline.
Uncertainties that are potentially significant to the cost baseline were
used to define the scope of the roadmap. Uncertainties and activities
needed to support later project decision points were set aside until
later in the project schedule. As development continues, it is expected
that these issues will be addressed.

The calcine roadmap was developed to support technology selection. Those
uncertainties not associated with technology viability or significant
cost issues were set-aside until later in the project schedule. With
three technologies being considered, the amount of effort that could be
expended without prioritization could delay technology selection for
many years. Once the selection is made, the chosen technology can be
further developed, starting with the list of uncertainties that were set
aside.

In summary, the suggested improvements to the roadmapping technical
responses phase are:

1.  Prioritize uncertainties before building technical responses;

2.  Prioritize consequence categories and align with the project
    decision points. Place only the necessary uncertainties in each
    consequence category;

3.  Group the uncertainties and describe them with problem statements.
    This grouping of the uncertainties can help reveal if development
    activities can solve more than one uncertainty at a time;

4.  Be sure that the relationships that need to be maintained among the
    roadmapping elements improve the likelihood that the development
    work will accomplish what is needed; and

5.  Develop technical responses, in an iterative fashion, that follow
    the project decision points and schedule. Save the next decision
    point's uncertainties for later consideration, after the issues for
    the current decision points are nearing completion.

## Phase IV Improvements - Roadmapping Technology Implementation

Resource loading of the technology tasks often present yearly funding
needs that has large peaks. In order to level the budget, activity
schedules are shifted until allowable funding levels are achieved.

Building the roadmap using project management software provides some
help for resource loading the development schedule and performing
funding analysis. For the roadmaps discussed in this paper, the roadmap
schedule, even though it was input into a project management software
package, is not used to manage the work. However, by building the
relationship of the roadmap activities to the company responsibility
matrix and the work breakdown structure (WBS), the roadmap was useful to
summarize the work, funding, milestones, interfaces and resources for
each technology development activity for use in the company project
management/control system.

In summary, the suggested improvements to the roadmapping technology
implementation phase are:

1.  Build the roadmap in a project management software system so
    resource loading, budget profiles, and schedule logic can be
    manipulated and used to analyze the technology development
    schedules; and

2.  Build a WBS responsibility matrix into the roadmap to allow direct
    reporting of the roadmap information to the company work package
    managers.

## Synergistic Interfaces Between Decision-Making and Roadmapping Processes

As mentioned in the Introduction, the coincidence of roadmapping and the
decision process by some of the same people proved useful to both
processes and products. In decision-making, understanding the
discriminators for choosing among alternatives is important. When
evaluating technologies, the criteria used, such as technical maturity
or implementation confidence, are often very subjective. Though
subjective, these judgements generally are made by competent and
knowledgeable personnel. However, as other criteria are evaluated, the
results usually don't get any better. This can effectively slow down a
selection if more effective discriminators cannot be further defined.

The roadmapping process identifies uncertainties and defines what is
necessary to resolve them. The first evaluation phase of the roadmap
identifies the uncertainties. The SBW roadmap team defined the
uncertainties for many technologies being considered by DOE. Although
the uncertainty definition was necessary for the roadmapping process, it
was the more detailed technical maturity data that provided some of the
necessary discrimination between the alternatives. By this process the
technologies were selected for consideration during the next phase of
roadmapping.

Phase III of the roadmapping process prioritized the uncertainties and
identified the technical responses and schedule logic and duration. The
results of this phase were optimistic schedules for each of the
remaining three alternatives in the SBW alternatives analysis. A
consequence reduction curve was drawn from these schedules. Figure 2
shows how the three finalist alternatives compare. These graphs provided
information for another level of discrimination since one of the three
remaining technologies was much slower to mature than the other two.

![](media/image2.wmf){width="5.995833333333334in" height="4.05in"}

Figure 2. Consequence Waterfall Chart Expressing Roadmap Schedules for
Three Technologies.

If at all possible, it would be good to select the final technologies
before progressing through the resource loading and preparations for
implementation. In the case of the SBW roadmaps, there was concern that
the roadmaps would take too long. During the roadmapping process, an
apparently good alternative began to surface as a potential favorite;
therefore the team created a resource-loaded schedule a couple of months
ahead of the actual selection of a preferred alternative. The technology
that was already roadmapped through Phase IV was not the one selected as
the preferred alternative. The team worked quickly and thoroughly to
produce the technology roadmap for the chosen preferred alternative in
less time than it took to get the approval through the DOE
chain-of-command. Thus, moving forward ahead of the decision process did
not save time but rather caused a detailed plan to be generated that was
not used.

In the calcine technology selection process, it was found that there
were two key uncertainties that lacked sufficient information to make a
choice. These key uncertainties became the basis to prioritize the work
required. All work that could be delayed was placed in later decision
points and will be addressed later.

In summary, the recommended improvements are:

1)  Provide the data from the roadmapping process to help with further
    discrimination of technologies during the selection process; and

2)  Make alternative down-selection and detailing of alternatives'
    uncertainties complimentary to each other, reducing the scope of
    roadmapping and increasing the confidence in the technology
    selections.

## Overall Summary

The preparation of several technology roadmaps has led to process and
product improvements. Although these improvements have made technology
roadmapping more efficient and effective, additional improvements are
likely and will be identified in future INEEL roadmapping efforts as
technical development and project designs are integrated.

**References:**

1)  Federal Register, 1995, 60 FR 28680, *Record of Decision for the
    Programmatic Spent Nuclear Fuel Management and Idaho National
    Engineering Laboratory Environmental Restoration and Waste
    Management Programs*, U.S. Department of Energy, June 1.

2)  *Pre-Decisional Sodium Bearing Waste Technology Development
    Roadmap*, Arlin Olson, James Murphy, Keith Perry, INEEL/EXT --
    2000-01299, 10/2000.

3)  *Draft Calcine Treatment Technology Development Roadmap*, James
    Murphy, James Melton, Arlin Olson, Adam Rogers, INEEL/EXT --
    2000-01620, 12/2000.*nt*, U.S. Department of Energy, September 19.

4)  *Sodium Bearing Waste Processing Alternatives Analysis*, James
    Murphy, Brent Palmer, Keith Perry, INEEL/EXT -- 2000 -- 00361,
    12/2000.

5)  *Applying Science and Technology Roadmapping in Environmental
    Management, Draft B, DOE Draft Guidance, 2000.*