# Presentation: 766559

## Estimating the Contribution of a Government Program

- Rosalie T. Ruegg
- TIA Consulting
- ruegg@ec.rr.com

- Conference on Estimating the Benefits
- of Government-Sponsored Energy R&D
- Crystal City, VA
- March 5, 2002

## Attributing Benefits to a Government Program

- inputs

- outputs

- outcomes

- impacts

-   What would have happened without the gov’t  effort?
- 	[counterfactual studies]
-   What happened in similar cases where there was
-     no gov’t intervention?
- 	[control group and comparison studies]

- It’s not enough to measure these effects; evaluation seeks to determine how much is attributable (caused by) the gov’t intervention.

## Overall Net Benefits to Innovation X

- Government +Industry R&D Investment

- NPV Benefits
- computed against a baseline

## To estimate the return on government investment, we need to isolate its effect

- Benefits w/o gov’t
- measured against the baseline

-       Private Actions w/o Gov’t Intervention

## A Gov’t Program Might Affect Benefits By:

-  Accelerating development & deployment
- 	- start earlier
- 	- progress faster
- 	- pursue what otherwise would not be done
-  Increasing scale  raising likelihood of success
-  Increasing scope  widening applications
-  Building new innovation networks/pathways
-  Other effects

- (It might also substitute for private investment)

## Impact of Acceleration on Social Returns

- Source:  RTI, NIST GCR 97-737, 1998, fig. 2-3

##    Illustrative Approaches to Estimating Gov’t Impact (examples from ATP) 		

- Purposes:
- 1. To support specific project benefit-cost case studies
- 2.  To provide program-wide evidence of attribution
- Study Examples:
-  Survey Method
-    	e.g., Silber, Powell, Laidlaw –counterfactual analysis
-            	        Feldman/Kelley--control group study
-  Case Study Method
-   	 e.g., Gompers/Lerner, RTI, Pelsoci, 50 completed Projects, etc.
-  Econometric/Sociometric Methods
- 	e.g., Branstetter/Sakakibara, Zucker/Darby, Fogarty/Sinha/Jaffe, etc.

## Acceleration: Laidlaw Survey of 28 ATP Projects

- Applied research cycle time Reductions:
- 	Median reported values
- 		-  50% reduction
- 		-  3 year cut
- 	Range of reported values
- 		-  % reduction: 25% to infinity
- 		-  years cut:  1-2 to unbounded

## Other Relevant Effects:  Laidlaw Survey	

-   86% expected flow-thru of time cuts to later stages
-   79% gave “ballpark estimates” of the value of cutting time      	by 1 year
-   Respondents listed ways the program cut time
- 	- upfront planning and mangt requirements
- 	- achieving a critical funding mass
- 	- enhancing ability to attract additional funding
- 	- increasing project stability
- 	- encouraging collaboration
-   86% reported carry-over of acceleration to other projects

## Feldman/Kelley Non-Winner “Comparison Group” Analysis as Evidence that Gov’t Program (ATP) Makes a Difference

| Extent of Follow-up Activity | Percentage |
| --- | --- |
| Did not proceed with the project at all | 61.4 |
| Began project on a much smaller scale | 16.4 |
| Began project on a somewhat smaller scale | 11.7 |
| Began project at about the same scale as proposed to ATP | 5.3 |
| Began project at somewhat larger scale | 2.9 |
| Began project on a much larger scale | 0.6 |
| Refused to answer/don’t know | 1.8 |

- Source:  Feldman & Kelley, NRC Report on ATP, 2001, p. 134.

## Discussion Topic: Use of a 5-Year Acceleration Rule      			for DOE Program Evaluation

-   NRC basis for?
-   Is it “conservative”?
-   Is it generalizable?
-   Is it feasible to determine specific time lags
-     on a program-by-program basis or should it be
-     on a project-by-project basis?
-  (ATP’s experience suggests no fixed Rule)