# Presentation: 127160

## Indicators of Knowledge Value

- Conference on Estimating the Benefits of Government Sponsored Energy R&D
- Department of Energy
- At Hilton Crystal City - March 4-5, 2002
- Diana Hicks
- CHI Research, Inc.
- 10 White Horse Pike
- Haddon Heights, NJ 08035
- Tel: 856 546 0600  Fax: 856 546 9633
- E-mail: dhicks@chiresearch.com

## Outline

- Who is CHI?
- Knowledge value and bibliometrics
- Examples: paper-patent citations
- Examples: paper-paper citations
- Patent citations to papers
- Examples: patent-paper citations
- People and knowledge value

**Notes:**
Will be hard to keep straight all the different types of citations.

Including DOE examples

## Who is CHI Research?

- We are a 30 year old consulting firm specializing in development and analysis of science indicators and patent-based technology metrics.
- Francis Narin, founder, developed techniques for analyzing national scientific performance in the 1970’s; then developed techniques to assess corporate technological intellectual property using patent-based metrics in the 1980’s; then extended techniques to relate technology portfolios to stock market valuation in the 1990’s.
- We have a staff of 21 (10 analysts and 11 support people) with diverse technical backgrounds.
- Among the senior staff we have 6 PhD’s and 9 Masters degrees in areas such as Physics, Chemistry, Mathematics, Computer Science, Science Policy, Bibliometrics, and Public Policy.

## CHI’s Databases

- US Patent Database
- Since 1975
- 3 million US Patents, 1.1 million EPO patents
- 24 million citations
- Analytically ready
- Filters
- CHI has developed the expertise needed to cleanly pull sets of patents on any technology
- Unification
- Company names unified to link parent/subsidiary/joint ventures
- Restated for mergers, divestitures, and reassignments
- Indicators
- More than 20 technology indicators
- 2.4 million references to non-patent literature classified
- 1.4 million of these are references to journal articles and have been put in standard form

## Knowledge Value and Bibliometric Techniques

- Technology
- Patents citing Patents

**Knowledge Value and Bibliometric Techniques**

- Science
- Papers citing Papers

- Outcomes
-     Patents		 & 	Stock Market
- citing Papers	 	 Performance

## Extract from the Front Page of a 

**Extract from the Front Page of a **
**U.S. Patent**

## DOE government interest patents CHI indicators

- Patents that indicate government interest and contain the word “energy” in the government interest field. Patents identified by David Eike

**Indicators defined at: http://www.chiresearch.com/about/data/tech/indicator.php3**

**Notes:**
Current Impact Index (CII) - The number of times a company's previous five years of patents are cited in the current year, relative to all patents in the U.S. patent system. Indicates patent portfolio quality. A value of 1.0 represents average citation frequency; a value of 2.0 represents twice average citation frequency; and 0.25 represents 25% of average citation frequency. 

Technology Cycle Time (TCT) - Indicates speed of innovation or how fast the technology is turning over, defined as the median age in years of the U.S. patent references cited on the front page of the company's patents. Companies with shorter cycle times than their competitors are advancing more quickly from prior technology to current technology. In semiconductors, cycle times are short (3-4 years); in shipbuilding they are long (more than 10 years). The average is 8 years. In fast moving technologies, TCT allows you to identify companies that may gain the advantage by innovating more quickly.
Science Linkage (SL) - The average number of science references cited on the front page of the company's patents. High science linkage indicates that a company is building its technology based on advances in science. 

## DOE government interest patents, leading technology areas

## IBM’s patents are valuable

## Does an agency resource support those creating high value technology?

**Incidence of Subscriber and Non-Subscriber Top-Decile CFC Replacement Patents**

## Conceptual Diagram

- 9 U.S. Patents

- 5 Foreign Patents

- IBM Patent No.
- 5, 278,955
- Issued 1994

- 6 U.S. Patents

- 6 Other References, Including 3 Science References

*Backward Citations (References)*

*Forward  Citations*

- Time

- 1985-92			1994		     1995-98

*A Starting Patent references prior art, and is cited by later patents*

## Science Linkage is Increasing in             Many Countries

## Excluding Biotechnology - Science Linkage is Increasing in Many Countries

## Public Sector Science is Valuable for U.S.-Invented Technology

**% of references on U.S. patents to U.S. scientific literature**

- Source: NSF Science & Engineering Indicators and CHI Research, Inc.

**Notes:**
- The institutional origins of research cited on patents were found to lie heavily 
in the public sector.  

- In 1988, 74 percent of the papers cited in U.S. industry patents were authored 
in universities, government laboratories and other publicly supported 
organizations.  By 1998, this figure had risen to 80%.

- Patents cite papers published in the most prestigious scientific journals

- Examining the funding acknowledgements on the papers cited in patents 
established that they were overwhelmingly supported by leading federal 
agencies such as NSF and NIH.

- As the New York Times reported, this study found that "Publicly Financed 
Science Is a Pillar of Industry" (Tuesday, May 13, 1997, p. C10).  The data 
make it clear that public science plays an overwhelming role in the science 
base of U.S. industry. 

- Recent work has shown that the papers cited by patents are selectively 
chosen from the most highly cited, highest quality U.S. research papers.  
More specifically, a U.S. paper amongst the top cited 1 percent of scientific 
papers is 9 times more likely to be cited by a patent as a randomly chosen U.S. 
paper.  We conclude that American technology draws on the best of American 
science. 

## Technological and scientific value are often aligned

**Share of 1993-95 US papers cited in US invented, USPTO patents issued in 1997**

## U.S. Companies Preferentially Cite In-state Public Sector Science in Their Patents

- Expected # of citations = "patent state's" # of citations to all states multiplied by "paper state's" share of cites received from all states

- Citations from Industry Patents to Public Sector Papers

**Notes:**
- Here we examine the relationship between corporate technology and 
in-state public sector science.  In this plot, the patents are owned by companies 
and the papers were authored by universities or other public sector institutions 
(thus there is no self-citation). 

- We compare the actual number of in-state citations with the number of citations 
we might expect if each state's references were distributed according to the amount 
of industrially relevant public sector R&D in each cited state.  Total number of 
citations received from patents was used to indicate the amount of industrially 
relevant public sector R&D in each state.  Thus, the expected number of citations 
was calculated by multiplying the total number of patent references from a citing 
state by the cited state's share of all paper citations received.  

- All 25 states cite in-state science more than expected.  The median of the in-state 
actual/expected ratios is 4.2 compared to 0.9 for the entire set of state-state pairs.  
The preferential citing of in-state research is displayed above with the 
in-state ratios highlighted in gray. See appendix for state-by-state table.

- Scientific research seems easy to get - by attending conferences, reading journal 
articles or searching the Internet, and companies should be able to use science 
from anywhere. But complex knowledge contains tacit elements that must be exchanged 
in person, so personal relationships are important.  Thus we find evidence that technological 
development has strong links to local scientific research. 

## Does an agency’s research support technological innovation?

**Includes only papers with explicit support acknowledgements**

**Funding agencies acknowledged on biomedical papers cited by patents**

- 1993-94 patents citing US authored biomedical papers published 1981-1991

## DOE science supports US innovators

- US patents 1985-2000 citing DOE papers.
- Inventor addresses used to identify nationalities.
- A 10 year, 2 year lagged citation window is used.
- Citations fractionally counted

## DOE science has value for a range of technologies

- Number of citations from patents to technology area.
- Patents 1985-2000 citing papers in 10 year, 2 year lagged window
- Technology-science combinations with >200 citations are shown

## L. Bito and the Technology of Prostaglandins in the Treatment of Glaucoma

## Quadrant Model of Research Value

*Research has value for:*

- Technology (patents)

|  | No | Yes |
| --- | --- | --- |
| Yes | Nuclear & Particle
Physics | Can be found in: 
 Science, 
Nature,
 Cell, PNAS |
| No |  | Ultrasonic Imaging
AT&T Technology Review
Non-science linked patents |

- Science
- (Scientific Literature)

**After Stokes**

## Knowledge value is very unevenly distributed

**People are key, especially for prospective studies**

## Summary

- Bibliometrics, properly constructed, are useful in assessing knowledge value.
- The value of research for future scientific research and technological innovation can be examined.
- People are crucial.