# Presentation: 316955

## Estimating the  Value of R&D: Using Estimates in R&D Program Planning and Evaluation

**Norling@rand.org**

**Notes:**
I certainly appreciate the chance to participate in this panel discussion and share some perspectives from DuPont and also RAND where I’m spending a year

## Topics

**An evaluation we did not want to see**
**A look at the R&D and innovation processes**
**Some estimating approaches: depends where you are in the process**
**What about risk?**
**Applying evaluations to R&D planning**

**Notes:**
Here’s what I’d like to cover as time allows

## Wall St. Journal-1992

**Notes:**
It was ten years ago when we in the DuPont R&D community found our performance review on the front page of the Wall St. Journal.

## Wall St. Journal : Front Page

**“****despite spending of more than $13 billion on chemical and related research over the past 10 years, DuPont’s 5,000 scientists and engineers were a technological black hole”**

**Notes:**
The words rang out: “despite spending of more than 13 billion on chemical and related research over the past 10 years, DuPont’s 5000 scientists and engineers were a technological black hole”

##           More:

**“****They sucked in money but, Company officials concede, didn’t turn out a single all new blockbuster or even many innovations.”**
**“****The technology is great, but where’s the payoff?”**

**Notes:**
The Journal continued:

 They sucked in money but, Company officials concede, didn’t turn out a single all new blockbuster or even many innovations”  The technology is great but where’s the payoff?

## Another Nylon or Teflon ®?

**1938: Nylon, Teflon® and Butacite®**

**1940**

**1935: Better things for better living... through chemistry”**

**Notes:**
The Wall St. Journal wanted to know where was the next Nylon or Teflon?

Here was a concern that R&D was not delivering and raising questions on gaining benefits from the costs incurred.

Much has changed in DuPont R&D for the better since 1992 -- I’m glad to say that others have more recently given  favorable performance reviews.

But I’m not here today to review the R&D programs and results for any particular company. I’m here to understand why companies are getting more for their R&D and why they continue to invest more and more each year in their R&D.

Let’s take a look at the current level of support for this R&D--

## R&D as  a System

**Processing system**

**R&D Lab**

**Activities: Research,Development,testing,knowledge building...**

**Inputs**

**people**
**ideas**
**equipment**
**facilities**
**funds**
**information**

**In-process measurement**
**and feedback**

**Outputs**

**patents**
**products**
**processes**
**publications**
**facts,**
**knowledge**

**Receiving system**

**Marketing**
**business planning**
**manufacturing**
**Operations**

**Outcomes**
**cost reduction**
**new businesses,products, processes...**

**Brown and Swinson RTM July-Aug.1988**

**Output measurement and feedback**

**Outcome measurement and feedback**

**Notes:**
To understand the role that benefits play in launching R&D, we need to understand something about  the R&D process itself.
Industrial R&D  can be pictured in this way. Inputs-- people, ideas, funding flow into the R&D laboratory where  testing, exploration, and knowledge building takes place. Out of the lab flow outputs -- patents, designs for products, processes, services, and knowledge. The receiving systems such as a business unit, a manufacturing function,  engineering ,or other operations take the outputs and convert them to outcomes -- cost reductions, new businesses, new products, new services, and improved products, processes, and services.

## R&D as  a System

**Processing system**

**R&D Lab**

**Activities: Research,Development,testing,knowledge building...**

**Inputs**

**people**
**ideas**
**equipment**
**facilities**
**funds**
**information**

**In-process measurement**
**and feedback**

**Outputs**

**patents**
**products**
**processes**
**publications**
**facts,**
**knowledge**

**Receiving system**

**Marketing**
**business planning**
**manufacturing**
**Operations**

**Outcomes**
**cost reduction**
**new businesses,products, processes...**

**Brown and Swinson RTM July-Aug.1988**

**Output measurement and feedback**

**Outcome measurement and feedback**

**Notes:**
At each step we can measure the effectiveness of the effort. There are  input metrics, in-process metrics, output metrics, and outcome metrics. Often different constituencies are interested in measuring at different points in the process. The Wall St. Journal was looking at Outcomes to be sure.
The metrics may be used to help determine the R&D budget for the coming years.

## Innovation : Not a Linear Process

**Market knowledge**

**Technology**
**developments**

**Changes in**
**the external**
**environment**

**Knowledge of **
**customer**
**needs**

**Company **
**strategies**

**Scientific advances**

**IDEAS**

**OPPORTUNITY**
**IDENTIFICATION**

**CONCEPT/**
**TECHNOLOGY**
**DEVELOPMENT**

**SCREENING AND**
**BUSINESS ANALYSIS**

**DEVELOPMENT**
**AND**
**COMMERCIALIZATION**

**MARKET**
**PENETRATION**
**AND **
**DEVELOPMENT**

**Notes:**
The R&D process is in turn  part of the overall more general  innovation process --  where innovation is the  taking of  ideas and turning them into something unique and  tangible  and something that has that has utility.

Several points are worth mentioning while looking at this chart:
One -- the innovation process is not a linear one -- there are many recycle loops, often  the idea must be reworked, or  put on the shelf for a year or two.

Also there are many influences on the process, new company strategies, changes in regulations, marketplace forces that change over time -- it is here that governmental S&T policies can also have an impact.

## Framework for Benefits

**Realized**

**Expected**

**Option**

**knowledge**

**economic**

**environmental**

**defensive**

**X**

**X**

**X**

**X**

**X**

**X**

**X**

**X**

**X**

**Notes:**
Using the framework suggested by Russell Lee we can look at a number of techniques used to determine benefits -- with some examples from my experience in R&D planning at DuPont.

Note that “security” in the framework translates into defensive R&D in the private sector/

## Estimating Techniques

**Net present value**
**What discount rate?**
**Many uncertainties**
**“****save the business projects” – how many times?**
**Value in the business plan**
**Know the possible value of the new product or cost reduction**
**How sure – in the earnings forecast?**

**Notes:**
Discuss from chart

## An Energy R&D Example

**To strengthen its position as a world leader in fuel cell technology, the Government of Canada announced a $19 MM Technology Partnerships Canada (TPC), repayable investment (royalty based on sales) with DuPont Canada Inc. (DPCI).  The government  project will further generate ca.  $30 MM - $40 MM in R&D activity.**
**TPC and DPCI have signed a four-year program to develop key components for the FC  industry, particularly the PEM variety. Over the next four years of development, FC staffing is forecast to grow from 27 to 80, including engineers, scientists, and technical and skilled trades people. **
**Success in DPCI's initiatives could lead to cumulative investment in development and commercial capital facilities totaling $45 MM by 2009, creating 500 jobs in production and continuing development of FC technology.**
**The TPC-DPCI project will create/maintain about 56 jobs during the R&D phase, and an additional 443 jobs in the post-development phase.**

**Notes:**
A most recent example of spelling out the expected benefits of a fuel cells program by DuPont in Canada

## Estimating Techniques

**Estimating “in-process R&D”**
**Appraisals made for acquisitions:spelled out in DuPont Annual Report**

**Purchased in-process R&D represents the value assigned in a purchase business combination to R&D projects of the acquired business... that were not yet complete for which technological feasibility has not yet been established... In accordance with Statement of Financial Accounting Standards No. 2 with Interpretation # 4, amounts assigned to purchased in-process R&D meeting the above criteria must be charged to expense.**
**In 1998 there were revisions to values established in previous years.**

**Notes:**
We’ve had some experience in the past five years of determining to expected benefits of R&D that is in progress -- as an acquisition takes place. This is neither to justify the research nor to determine retrospective benefits-- maybe this is close to purchasing an option.

## Estimating Techniques

**Compare with similar projects**
**Some rules of thumb 3x return on standard cost reduction effort**
**Data base of assessments of previous projects**
**“****This project has the characteristics of project X in 1995”**
**Studies by  Independent Project Analysis Inc.**

**Project 80: All new process: In retrospect it is clear that this process was not ready for commercialization. The key decision was not to spend the considerable sum required for a fully integrated pilot plant. If it had been built the project would not have gone forward because data would have shown the process to be uneconomic.**

**Notes:**
A number of cases at retrospective analysis

## Estimating Techniques

**Value of Intellectual Property**
**Trend for orphan patents to be donated to universities**
**Adds to the bottom line**
**Accounting rigor needed**
**Financial analysts (Wall St.) estimate value of research in the pipeline**
**Primarily in pharmaceuticals**
**Other sectors now sharing pipeline information- important driver in the 90’s**

**Notes:**
Two other type of analyses

## Estimating Techniques: Non-financial

**Merrifield scoring for commercial success – validated with existing businesses**
**D. Bruce Merrifield, “Corporate Renewal Through Cooperation and Critical Technologies”, Research-Technology Management, July-August 1993**
**Business attractiveness**
**	****sales/profit potential**
**	****growth potential**
**	****competition**
**	****distribution risk**
**	****restructuring potential**
**	****political/social factors**

**Business Fit Factors**
**     ****Capital availability**
**      ****Manufacturing**
**      ****Marketing-distribution**
**      ****technical competence**
**       ****access to components**
**       ****management**

**Notes:**
The Merrifield approach is also useful

## Estimating Techniques: Non-financial

**Venture capital: the entrepreneur**
**NRC panel evaluation of research at Army lab; measures: lives saved, battles avoided....**
**Strategic value to a business**
**Environmental, health, and safety**
**Project itself adds value to other products or enables another technology**
**Stine approach: guess that area of research would yield important knowledge for the company**

**Notes:**
Some non-financial approaches

## Discovery Research in DuPontHow Did it Start?December 18, 1926

**“****We are including in the central Chemical Department’s budget for 1927 an item of $20,000 to cover what may be called , for want of a better name,**_** pure science or fundamental research work**_**...”**
**“****Work undertaken... with the **_**object of establishing or discovering new scientific facts**_**...”**

**Fundamental work was declining and applied research faced a shortage of its raw material... IDEAS**

**Notes:**
We have an example from our history --
At the end of 1926, Charles Stine - then head of Research for DuPont, called for the beginnings of “pure science” or fundamental research in DuPont. He sought $20,000 and got it to launch a multi-year program.

He recognized the relationship between basic research and business results. In making his case he said that applied research was running out of its raw materials -- ideas -- the new  knowledge created through fundamental research.

## Stine: 1927

-    “It is important to distinguish between fundamental research.... and pioneering applied research.
- 	Pioneering applied research might result in the development of something of great value or might come to naught...
- 	Fundamental research is bound to result in the discovery of new highly useful, and in many cases, indispensable knowledge.”
***	***

**Notes:**
In making his case he pointed  out that  while applied research may or may not create something of great value,fundamental research  is bound to create something highly useful and in many cases -- indispensable knowledge.
(Take a moment to read slide) It applies today

## Stine’s Observation

*“**It is an interesting fact that none of our competitors in this country, so far as we know are carrying out fundamental research... *_*This should give us increased assurance of the wisdom of our plan to undertake work of this nature.”*_

**Notes:**
Stine recognized that others were not carrying on fundamental research; he saw  an opportunity for  competitive advantage.

In today’s terms, Stine was bold in going against his benchmarking results -- going against what others were doing. 
He helped create  new competitive space. The fundamental research program brought Wallace Carothers to DuPont , led to the invention of neoprene, nylon, polyesters -- and the establishment of a new DuPont and  an entire new industry.

## But what about basic research?

**Advances in the sciences – can make some wild guesses: credibility?**
**Many times it takes a number of research products to develop one technology – how to assign a value to each?**
**Often difficult the track the history of  a development**

**Notes:**
Discuss from slide

## Radical Innovation

**Collaboration -- IRI-RPI Lally School of Management**
**12 projects - 5-10X+ performance improvement, 30-50% reduction in costs**
**Non-linear, can this disorderly process be managed?**
**Some useful approaches: clear articulation of strategic intent, deliberate spelling out of “holy grails”, targeting potentially rich domains, exposing R&D to the marketplace -- and more**

**Notes:**
Over the past six years IRI’s Research-on Research Committee has been working with a team of professors from the Lally School of Management at RPI (led by Professor Mark Rice) to study Radical or Discontinuous Innovation (see definition). Projects from GE, DuPont, IBM, Nortel, Texas Instruments, Otis Elevator, and others were studied in real time. Some interesting findings have emerged and are included in a just-published book by Harvard Business School Press. The team found that many of the management techniques such as tight control, or key checkpoints -- just don’t work and can be counterproductive. While they did not find a common recipe -- they did find that several “approaches” seemed to help (read from slide)

## Technology Supply Chain

**19**

**Technology Supply Chain**

**Science Base**

**Product Development**

**Market Knowledge**

**Invention**

**Process Development**

**Application Development**

**Customer Process Development**

**Plant Support**

**Customer Support**

**Notes:**
It is very important for any industrial R&D organization to maintain close contact with the science base -- whether that is work in academia or government laboratory. 

The importance of maintaining  federal funding for basic research is more than apparent. CEO’s know that funding for basic research is one of the drivers for a healthy economy.

## Knowledge Drivers of the Future

**Knowledge Drivers of the Future**

**Chemical &**
**Material Sciences**

**Biological Sciences**

**Information Sciences**

**Bio-computation**

**Smart Materials**

**Bio-engineered Materials**

***Genomics***

***Combichem/HTS***

***Rational Design***

***DNA Manipulation***

***Informatics***

***Modeling/Simulation***

***Data Mining***

***Nanoscale***

***Chiral Chem.***

***Structure/Property***

***Combi-methods***

***Poly Catalysis***

***Opto-electric Materials***

***Cell Biology***

**Notes:**

For many businesses, the scientific advances being made in the material, biological, and information sciences all have important roles to play.
There may be a special interest in catalysis, modeling and simulation, and bioprocessing, but there are also  interesting things  happening where the sciences overlap or interact, such as bioengineered materials, biocomputation, or “smart” materials.

## Basic Research and Innovation

**Basic Research and**** Innovation**

**A better understanding of the relationship of basic science and technological innovation: there is no linear process**

**Notes:**
One of the best books to describe the relationship between science and technology is -- Pasteur’s Quadrant: Basic Science and Technological Innovation by  the late Donald Stokes -- Woodrow Wilson School at Princeton
Stokes pointed out that basic research and technology development should be viewed differently than it has been in the past. The model that he presents is most important for us to understand.

## Linear  Model

**Linear  Model**

**Applied **
**R&D**

**Technology**
**Development**

**Basic**
**Research**

**Existing**
**Understanding**

**Improved understanding**

**Improved Technology**

**Existing Technology**

**Economic Growth**

**Notes:**
Stokes  makes the point :

The linear model of research doesn’t seem to apply -- 
The linear model says that basic science or research leads to applied science which then leads to technology development and onto wealth creation.  
It is not that simple

The fundamental premise  here is that science informs  and determines technological advancement-

## Quadrant Model: ResearchInspired by:

**Quadrant Model: Research****Inspired by:**

**consideration of use?**

**Quest for understanding?**

**yes**

**no**

**no**

**yes**

**Notes:**
Stokes asks two questions about motivations for basic research (read them)
The upper left hand quadrant includes basic research -- the quest for understanding without thought of practical use. He calls it Bohr’s quadrant -- Niels Bohr’s quest for a model  structure of the atom was a pure voyage of discovery .
The lower right hand cell includes research  guided  by applied goals without seeking  more general understanding. This is Thomas Edison’s quadrant --( he kept his co workers at Menlo Park -- in the first industrial lab in the US from pursuing the deeper scientific implications.)
The lower left hand quadrant is not empty -- this includes research that is inspired neither by the goal of understanding nor the goal of use -- studies of biological classification, (taxonomy) or work to enhance the skills of a researcher are examples.

## Pasteur’s Quadrant

**Pasteur’s Quadrant**

**Notes:**
Pasteur’s  quadrant is the upper right one.
This includes basic research that seeks to extend the frontiers of understanding but is  inspired by considerations of use. 

This is the quadrant we aspire to fill -- to solve daunting problems with scientific inquiry.

The descriptor for this model is Louis Pasteur. Pasteur was a scientist seeking fresh fundamental challenge in perfecting in what his day was called “the useful arts” His studies on fermentation led to still further basic research for understanding

What this really means is that for basic research to be highly effective in the industrial environment one needs a focusing problem.

The challenge is to be scientific in solving that problem. (I might add that two years ago DuPont changed its slogan from “Better Things for Better Living (through Chemistry)” to Miracles of Science

## A Revised Dynamic Model

**A Revised Dynamic Model**

**Use-inspired**
**Basic research**

**Applied **
**R&D**

**Pure Basic**
**Research**

**Existing**
**Understanding**

**Improved understanding**

**Improved Technology**

**Existing Technology**

**Economic Growth**

**Notes:**
Stokes therefore gives us a  richer picture of the relationship between  basic research and technology development. 

The whole network is connected and each affects each other.

Technology developments do indeed inspire further basic research  -- and industry needs to be connected to the basic research funded or conducted by the government.

## Application to R&D Planning

**Portfolio Planning**
**Strategy tables**
**Decision and Risk analysis**
**Scenario planning**
**Ten Year Technology plans**

**Notes:**
Now for the specific topic at hand

## Example: Ten Year Technology PlanOutline

- I.	Business Strategy Brief
- II.	Technology Time Line
- III.	Key Programs Details
- IV.	Resources Table
- V.	Metrics
- VI.	Competitive Position
- VII.	Key Issues

**Notes:**
To assure that we focus our resources and the research and technology development in support of each of our businesses we create 10 year technology plans -- and these plans are updated each year. 

## Technology Time Line Additive Chemicals

_**Technology Area**__**	**__**Programs**_

** ****Catalyst Optimization**

**1996****	****1998****	****2000****	****2002****	****2004****	****2006**

**cosmetics thrust**

**Feedstock**
**Monomer**
**Polymerization**
**Compounding**
** ****Uniformity**
**Dispersion Transport**
**Renewed Products**
**New Applications**
**Waste Treatment/ ****Recycle**

**Antimicrobial  Compositions**

**“****Z” monomer process**

**RDT**

**New emulsion polymerization**

**“****CD” polymerization**

**dispersion Control**

**Polarity control**

**Redundancy Development**

**Transfer Line Chemistry and Modification**

**Dispersion  Recycle**

**Biodegradable compositions**

**No Waste Process**

**holographic coloring**

**Zero Defects**

**Integrated Technology ****Development**

**Example**

**Notes:**
While, I won’t  go into specifics -- here is an example of how we lay out technology roadmaps over 10 years -- in this case for a hypothetical business “Additive chemicals”

We lay out three generations of technologies-- three horizons in   a number of areas important to the business. Near term we have programs that are underway and well defined. Mid term are those that must be developed and defined -- and longer term there is little definition. Here often the need is to explore the science base for the new advances that can be translated into important breakthrough technologies over the ten year period -- and possibly change the nature of the business.

## Segmenting Technology Spending

**Growth Options:**
**Create and establish bases for future growth**

**Base Growth****: Extend and improve current performance**

**Base Necessity:**
**Sustain current performance**

**Investment**

**Cost**

**Strategic Growth**
**Initiatives**

**short term**

**Mid- term**

**Long  term**

## Growth Dynamics

**300 proposals**

**40 expanded investigation**

**9 major developments**

**5 scale ups**** **

**2**** ****successes - $500MM/YR**

**10 **
**   ****Years**

**Apex**** ****targets**

**Will need a steady state pipeline of 600 proposals for 4 successes**

## Portfolio

**300 Submitted ideas**

**40 Early Investigation**

**9 Feasibility Development**

**5 Scale-up/Commercialization**

**2 Commercial**
**Successes**

**3-4 successes/yr.**

**“****APEX” proposals**

**Breakthru TiO2, direct Nylon, biomaterials, Nutraceuticals.............**

**Fuel cells, batteries, Versipol® Bio 3G.....**

**CPC’s,Quality Grains,Sustiva® NG-3,PTI**

## Management Tools and Techniques Through the Years

**1960**

**1970**

**1980**

**1990**

**2000**

**Intrapreneuring**

**Theory X-Y**

**MBO**

**Cost of Gov’t reg.**

**Org. Effectiveness**

**MBWA**

**TQM**

**Pecos River**

**Excellence**

**Benchmarking**

**HPWS**

**Missions& **
**Visions**

**Pay-for-performance**

**COT’s**

**Core **
**competencies**

**Business Process**
** ****Reengineering**

**Experience**
**Curves**

**Empowerment**

**Cycle-time**
**Reduction**

**Portfolio**
** ****Management**

**EVA**

**Managerial**
**grid**

**Balanced**
**Scorecard**

**Matrix Mgt.**

**Learning Organization**
**Knowledge Mgt.**

**Theory Z**

**MBX**

**3rd/4th Generation**
**Management.....**

**ROI**

**Platforms**

**Disintermediation**

**Notes:**

Many of these may be management fads that have come and gone...

## Success Rate New Products

**3000 raw**
**ideas**
**.03%**

**300 submitted**
**ideas**
**.3%**

**125 beginning**
**projects**
**.8%**

**9 Large**
**developments**
**11%**

**4 major**
**developments**
**25%**

**1.7 launches**
**60%**

**1 commercial**
**success**

- DuPont’s batting avg. is 2X

**Notes:**
-- as Stevens and Burley have shown, R&D is risky business

## Product Renewal Matrix: Probabilities of success

- Product Renewal Matrix: Probabilities of success

**Markets**

**Technology**

**Core**** ****Business**

**123projects**

**25**** ****projcts**

**$3 billion**

**$500 MM**

**58**** ****projects**

**$9 billion**

**Familiar**

**New**

**New**

**.05**

**.15**

**.50**

**.75**

**Suicide square**

**Notes:**
Using the familiarity matrix one can look at other probabilities --

## Product Renewal Matrix: number of projects and potential value if 100% successful

- Product Renewal Matrix: number of projects and potential value if 100% successful

**Markets**

**Technology**

**Core**** ****Business**

**123projects**

**25**** ****projcts**

**$3 billion**

**$500 MM**

**58**** ****projects**

**$9 billion**

**Familiar**

**New**

**New**

**3 projects**
**$300 MM**

**5 projects**
**$100MM**

**52 projects**
**$100MM**

**7 projects**
**$400MM**

**1 project**
**$10MM**

**10projects**
**$50MM**

**2 projects**
**$500MM**

**1 project**
**$1MM**

**12 projects**
**$30MM**

**Notes:**
And in our portfolio planning we can look at our collection of research projects to look for gaps-- and the level of risk we are taking

## Summary

**Many evaluation tools to use**
**Many different estimates are made that can affect funding, evaluation of a company, and earnings**
**Order of magnitude estimates may be good enough early in the process**
**Risk becomes a major factor – how to reduce risk is critical factor in carrying out R&D**
**An important part of a research project is to verify or rule out assumptions and establish the value of the product of the research**

**Notes:**
Discuss from slide