# Presentation: 070262

## Adoption of Uniform Cost Estimation Procedures for Evaluation of Innovative Technologies 

**Adoption of Uniform Cost Estimation Procedures for Evaluation of Innovative Technologies **

**Germantown, Maryland**
**August 23, 2001**

**Notes:**
I became technical coordinator in November 1999.
The scope of the project expanded in that FY and I formed a team composed of project leads and technical support.
This talk describes the lessons we have learned during the last two fiscal years and the changes in the approach ITRD is taking.

## Meeting Objectives

*Promote collaboration to improve methods to estimate cost savings associated with the use of innovative technologies for environmental remediation at DOE sites.   *
*Help ITRD and TechCon programs obtain an accepted tool for cost estimation in their technical assistance to the sites. *
*Produce a mechanism by which the TechCon and ITRD programs can provide cost information to the ACE Team**.   *

## Adoption of Uniform Cost Estimation Procedures for Evaluation of Innovative Technologies 

**Adoption of Uniform Cost Estimation Procedures for Evaluation of Innovative Technologies **
**Perspectives from the Innovative Treatment and Remediation Demonstration (ITRD) Program**

**Malcolm Siegel, Ph.D. **
**Technical Coordinator ITRD**
**Sandia National Laboratories**
**Albuquerque, NM. 87185**

**August 23, 2001**

**Notes:**
I became technical coordinator in November 1999.
The scope of the project expanded in that FY and I formed a team composed of project leads and technical support.
This talk describes the lessons we have learned during the last two fiscal years and the changes in the approach ITRD is taking.

## Outline

***Overview of ITRD Program Goals***
***Summary of Current Projects***
***Barriers to Innovation***
***Technical Challenges***
***Poor Cost Estimation***
***Worker Safety vs. Perceived Site Risks***
***Potential Solutions***

**Notes:**
We’ve had a display booth with descriptions of the projects that you can pick up for more information

I’ll spend most of my time talking about the non-technical lessons lessons learned and challenges

## ITRD and Technical Assistance Program

**ITRD and Technical Assistance Program**

**Notes:**
Itrd is part of SCFAs technical assistance program. 
There are three legs to the program, each component has a unique role that complements that of the other two. 

## Accelerate deployment of innovative characterization and remediation technologies

Reduce barriers to use of innovative technologies by reducing risks to DOE and Industry
Design combinations of technologies for innovative solutions to site-specific problems
Reduce regulatory  barriers by involving all stakeholders early in the evaluation process

Generate cost and performance data to support adoption of innovative technologies to address specific site needs

***Reduce barriers to use of innovative technologies by reducing risks to DOE and Industry***
***Design combinations of technologies for innovative solutions to site-specific problems***
***Reduce regulatory  barriers by involving all stakeholders early in the evaluation process***
***Generate cost and performance data to support adoption of innovative technologies to address specific site needs***

**ITRD**** ****Program**** ****Goals**

## On-going Projects

_***Hanford 100N***_*** - Strontium-Contaminated Groundwater***
_***Hanford 200 W***_*** - Carbon Tetrachloride in Soil and ******	******	******	******     Groundwater ***

**On-going Projects**

_***Paducah***_*** - TCE and ******99******Tc Contamination in Soil and ******	******	******      Groundwater***
_***DNAPL***_*** - VOCs in Deep Fractured Rock at Y-12 ******	******	******	******    and Portsmouth***

_***Mound ***_***- VOCs at OU1 and Radionuclide-Contaminated ******	******   Soil at PRS 66***

_***Explosives ***_***- Contaminated Soil, Surface Water and ******	******	******          Groundwater at  LANL and Pantex***

**Notes:**
At Paducah  the technical advisory group recommended 6 phase heating, C-sparge and PRB pilots based on ITRD meetings, vendor input and ITRD funded treatability studies.

## ITRD Technology Selection Process

**Potential Technologies**

**Suggested System Concepts**

**ITRD Technology Selection Process**

**Technical **
**Advisory **
**Group**

**Applicable Technologies**

**TAG**

## Some Criteria for Technology Selection

*** ******Site Objectives***
***  ******Source removal***
***  ******Prevent further migration off-site.***
*** ******Site Clean-up Criteria***
***  ******Groundwater MCL and Soil Clean-up Levels***
*** ******Site-specific Regulatory Issues***
*** ******Regulators prefer low waste  generation***
*** ******Citizens do not want incineration or catalytic oxidation***
*** ******System Impact***
***  ******Treat vadose zone, unconfined aquifer and confined aquifer   ***
***    ******with same technology***
***  ******Address both organics (e.g. TCE) and rads (e.g. Tc-99)***
***  ******Utilities and overhead clearance issues.***
*** ******Protection of Cultural Resources ***
*** ******Burial grounds and tribal access***

## Challenges at ITRD Sites

***Problems with Innovative Technologies***
***Soil flushing - excursions, waste generation***
***Permeable Reactive Treatment Walls - plugging***
***Air sparging- off-site migration due to preferential paths***
***Biostimulation/Bioaugmentation - poor delivery***
***Monitored Natural Attenuation***
***Perception that MNA is a “do-nothing” approach.***
***Perception that long-term monitoring costs are prohibitive***
***Regulatory issues***
***Cleanup levels may not be risk-based and may not consider background and method DL.***
***Cost estimates are not stable***
***Estimates used by TAG to make recommendations differ from final cost estimates made before pilots.***

**Notes:**
At one stage in the ITRD process vendors provide cost estimates for pilots and full scale deployments of their technologies.  The cost is an important criterion for selection.  We have found that in some cases the cost estimates used by the TAG to select a technology are not realistic. They can change drastically during the next phase of the ITRD process; which is preparation of detailed work plan  that involves the vendor and the M&I or M&O contractor subcontractor. 

## Initial versus Final Cost Estimates

**Initial versus Final Cost Estimates**

**Notes:**
At a third site, excavation is the proposed remediation technology. A vendor has estimated the cost to be about $12M wheras the site contractor estimates the true cost to be almost 96M.

## Vendors not familiar with DOE sites
Waste generation/handling 
Operational, utility, security constraints
Sites present novel technical challenges
Vendors may provide inaccurate or incomplete information about their technologies.
Risk to workers initially not considered
Poorly defined definition of end-point  for clean-up
DOE and M&I generally have no incentives to conduct remediation better, faster, cheaper.
Liability concerns
Lack of teamwork and communication
Procurement and budget barriers

***Waste generation/handling ***
***Operational, utility, security constraints***
***Sites present novel technical challenges***
***Vendors may provide inaccurate or incomplete information about their technologies.***
***Risk to workers initially not considered***
***Poorly defined definition of end-point  for clean-up***
***DOE and M&I generally have no incentives to conduct remediation better, faster, cheaper******.***
***Liability concerns***
***Lack of teamwork and communication***
***Procurement and budget barriers***

**Why Cost Estimates Change**

**Notes:**
At one site, the  cost estimate was very sensitive to the definition of clean up level and hot spot criteria because it was very important in determining the volume of soil that needed to be excavated. 

## Consequences of not Considering Worker Safety at Start of Evaluation

***Technologies are “not inherently UNSAFE”***
***Costly changes to address safety concerns are made late in technology development process***
***Estimated 1993 national costs of accidents and adverse health effects range from $100B to  $173B.***
***Remediation may do more harm than good***
***Cost/life saved excessive***
***“******Statistical” lives are saved at expense of “real” lives******.****** ***

## e.g. Mound PRS 66: Smart and Adaptive Sampling, used to define end point and hotspot clean-up goals for Th-232 and Pu-238, could lead to >50% reduction in volume.

***e.g. Mound PRS 66: Smart and Adaptive Sampling, used to define end point and hotspot clean-up goals for Th-232 and Pu-238, could lead to >50% reduction in volume***.
***e.g.*** ***Oak Ridge Y-12: Relative bioavailability of HgCl******2****** vs. HgS in sediments used to propose higher clean-up level and major cost savings*** ***($1400M vs $30M).***
***Key element in successful negotiation of higher risk-based clean-up levels was Stakeholder involvement in various stages of risk assessment and decision process.***
***User-friendly risk analysis tools could convince stakeholders to accept higher clean-up levels.***

**Importance of Risk-based Clean-up Levels**

**Notes:**
In some cases the cleanup goals established for a site require expensive or impossible clean ups and  nothing is done due to lack of clear-path forward. 

Risk based cleanups may lead to resolution of such impasses.
Key elements of establishing risk-based cleanups include recognition of natural background levels of contaminants and relative bioavailability of the metals in the soils compared to soluble salt used in toxicology studies. 
Thus, higher cleanup levels may be appropriate for soils in which the metals are in forms that are not readily absorbed by the GI tract. 

In order for stakeholder to be comfortable with risk-based levels they must be involved in many aspects of decision-making process. 

## Activities to Address Challenges

***Adoption of uniform accepted cost estimation techniques***
***Use of user-friendly risk assessment tools with TAG to build trust.***
***Analysis of effect of DOE-Contractor relationships on acceptance of innovation by M&I/M&Os***
**Interviews with key players**
**Reviews of contracts to define spectrum of interests.**

## Improving Cost/Performance Estimation

***Peer review of technical basis of candidate technologies***
***Lead Lab Technical Teams***
***Institute for Regulatory Science peer review***
***Use of Standard Cost Estimation Tools***
***Federal Remediation Technologies Roundtable (1995)***
***FETC Cost Savings Analysis Method (1998)***
***Net present value comparisons (constant present year dollars)***
***Applied Cost Engineering Team Concepts (EC******2******)***
***Standard cost elements (ECES/HTRW WBS)***
***Graded approach fits ITRD process ***
***single scenario and cost-analysis models***
***Include worker safety issues early in cost evaluation***
***Cost of compliance with OSHA standards***

**Notes:**
FETC = Federal Energy Technology Center.  Document produced in response to a memorandum issued in July 1997 by the Asst Secretary of Environmental Management calling for development of a standardized cost-savings analysis methodology.
EC2 = Environmental Cost Engineering Committee – interagency committee including reps from EPA, DOE, USAF, Army Corp enginers, Dept Interior, Naval Facilities Engineering Command and NASA.  It was formerly known as the Interagency Cost Estimating Group.
ACE 
ECES = Environmental Cost Element Structure – an descendent  of the Hazardous , Toxic and Radioactive Waste Work Breakdown Structure (HTW WBS) : = a list of elements (tasks, items etc) that need to be considered in a remediation project.  Has 5 hierarchical levels that describe life cycle elements at the highest level and detailed items at the lowest level.   Designed to provide a consistent framework usable by Site mangers, vendors and and might be used by ITRD TAG members to prepare and then compare cost proposals. 

## Safety and Health Costs

***Compliance costs***
***training & hazard communication***
***personal protective/emergency equipment***
***engineering controls***
***medical surveillance***
***monitoring, certification & record keeping***
***Catastrophic costs***
***accident, illnesses, premature deaths***
***damages to equipment and environment***

## Resources for S&H Cost Estimation

***Job Hazard Analysis (JHA)***
***Technology Safety Data Sheets (TSDS)***
***Integrated Safety Management (ISM)***
***Oct. 2000 OENHP Workshop***
***Compliance Cost Matrix***
***Analyses and guidance by:***

**Notes:**
Job Hazard Analysis used by employers to analyze hazards associated with all phases of an activity : guidance available from OSHA, Mine Safety
TSDS provide similar information and  is intended to describe relative risk of safety and health hazards associated with a technology; intended to be technology version of MSDS for chemicals
OEHNP = Operating Engineers National Hazmat Program
Compliance Cost Matrix contains list of hazards identified in the TSDS, applicable OSHA regulation (if one exists); the nature of compliance activities associated with the hazard and regulation, and costs associated with compliance. 

## Safety Cost Estimation in the ITRD Process

_**ITRD Stage**_
***Technology review***
***Detailed cost estimates***
***Cost/Performance Report***

_**Cost Tool**_
***Screen for TSDS’s; use ISM for baseline***
***S&H Compliance Cost Matrix***
***Validate compliance costs***

** **

**Methodology**

**Software**

** ****Probabilistic**
** ****Quantitative**
** ****Iterative**
** ****Model Based**

** ****Transparency**
** ****Consistency**
** ****Easy documentation**
** ****Seamless info. transfer **
** ****Robust / flexible**

**Notes:**
User-friendly risk assessment and decision support tools are available and community members should be trained in their use. 

## over-the-counter models for risk assessment and decision support 

***uses Analytical Hierarchy Process ***
**Sandia:*****  ******RADTRAN, PRECIS, Smart Sampling***
***SEDSS (Sandia Environmental Support System)***
**Argonne:**  ***RESRAD, RISKIND****** ***
***ASAP (Adaptive Sampling and Analysis) characterization and remediation tool***
**EPA/NCEA National Center for Exposure Assessment**
***RIS (Integrated Risk Information System)***
***IMES (Integrated Model Evaluation System)***
**Others:**  ***ITER, SmartRISK, RiskAssistant, and CalTOX***
***over-the-counter models for risk assessment and decision support***

**Risk Assessment/Decision Support Tools**** **

## Analysis of DOE-Contractors Relationships

- James Studer

## Contract Features Influencing Innovation

- Management and Operations (M&O)
- DOE structured for it
- Entire work scope held by contractor
- Cost Plus Award Fee
- DEAR - based contract

- Management and Integration (M&I)
- New to DOE, 1995
- Majority of scope competitively subcont.
- Performance-based Fee
- FAR-based contract

## M & I Contracts are New

- DOE knows M&O
- Almost all Sites are M&O, including the recently signed BWTX contract at Pantex
- First true M&I contract Rocky Flats/ Kaiser-Hill LLC, 1995, new Award 2000.
- Bechtel-Jacobs at Oak Ridge is M&I
- RF is Closure Site (2006) and OR is not

## Self-Performance vs. Subcontracting

- M&O ~ Self-Performance
- M&I ~ Subcontract most of scope
- In 1995-1997 KH subcontracted 90% of scope to 4 team members and 160 subcontractors
- Intense competition the rule

## Fee Basis

- M&O contracts are Typically Cost plus award fee
- Base Fee  and  Award Fee
- Focuses on the Cost of Doing Something
- M&I such as RF and KH is Performance Based Fee
- Focuses on Results and Cost Savings (but not necessarily to advantage of Innov. Tech)

## DEAR or FAR?

- Department of Energy Acquisitions Regs
- CFR Title 48 Ch 9 SubChapter I, Part 970
- Implements and Supplements FAR in DOE
- FAR-based means contractor has liability for safety and env. violations, no general liability protection, no letter of credit, and “killer clause” that allows massive DOE penalties for serious safety incidents.

## Where are DOE Contracting Practices Headed? 

- Contract Reform as exemplified by RF Performance Based Contract is alive and ...
- Robert Card, former President of Kaiser-Hill LLC, has recently been confirmed as Under Secretary at the department.
- Restructuring putting more control with Under Secretaries as opposed to procurement administrators.

## What Does This Mean?

- M&I and performance-based contracts will be the preferred route for closure sites if not most sites.
- Innovative financial incentives will be better matched to contractor risks.
- Life-cycle funding commitments will support life-cycle management behavior at the Sites (including LCCA and Cost Savings Evaluations).  So....
*Environment for integration of innovative C&R technologies may be improving. Right???*

## What Does This Mean?

- Maybe....if the contractor perceives that their incentives are effectively balanced against their perceived risks.
- Possible downsides:
- Penalty for use of DOE devloped technologies
- Greater risk aversion to protect fees

## Summary

***Technical issues may be the least significant barrier to adoption of new technology. ***
***Recommendation by majority of stakeholders in Technical Advisory Group is not sufficient to ensure adoption of innovative technology.***
***Interactions among M&I contractor, DOE and vendors may be controlling factor******.***
***Meaningful comparisons among technology alternatives require use of a uniform and comprehensive cost estimation method.***
***It is necessary to work closely with (fund) vendors and Site staff to prepare useful cost estimates. ***
***Clear separation of “real” costs from “risk aversion” cost may support activities to reduce risks of innovation.***

## References

- Robert Card Testimony before Congressional Sub Committee (1997)
- http:\\com-notes.house.gov/cchear/hearings.nsf/d71c5b34d01aa173852
- Clair Sink, OST, pc 8/01
- “Analysis of the DOE Contractor Base. Readiness, Willingness, Profitability and Trends” Jan. 2001, Contract Reform
- DEAR CFR Title 48 Ch 9 SC I Part 970

## Challenges to Making Progress

**Challenges to Making Progress**

***Our Goal***
***Collaboration among personnel from ITRD, TechCon, ACE, OEHNP, IRS, CRE, citizen groups and site workers  to ensure that deployment decisions are based on reliable engineering, economic, and health evaluations.***

**Notes:**
ACE = Applied Cost Engineering Team
OEHNP = Operating Engineers National Hazmat program
IRS = Institute for Regulatory Science
CRE =  Center for Risk Evaluation

## Discussion

## How to get everyone on the same page?

- Build database of historical data
- Require DOE vendors to report data in consistent manner
- Use TechCon database for vendor entry into DOE complex.
- Capture/report data by technology not vendor
- Use seconday (physical, chem. Location) to provide site specific attributes.

## Obstacles/Solutions

- Vendor concerns about proprietary info.
- Use Web to make baseline schedule and technologies available to all (WalMart model)
- XML structure allows for control of data access as appropriate.

## Relationship among efforts discussed today

- Database vs. methodology
- Graded approach for vendors, sites
- Link to Paths to Closure?
- Efforts to establish baseline technology by EM.
- Need for tool to compare potential pilots for ITRD

## Questions 

- Use of NPV in the government world?
- How to get data from uncooperative companies?
- Next step?
- Links to other EM focus groups, EM-30, EM-40, EM-50.