# Presentation: 426575

## Scientific Publication Information Retrieval & Evaluation System(SPIRES)

- Linking NLM PubMed Data to NIH Extramural Grant Data
- Presented by Paul Jordan, Systems Analyst
- NIH OD OER ORIS - eRA Project
- November 2007

## What is SPIRES?

- A “translation” of NLM publication data into a relational database that can be linked *directly* to eRA grant data.
- Oracle Text word indexes of various publication and project data elements.
- Third party journal impact factor data and (in the future) citation information.
- SPIRES application provides tools for analysis of publication and project data.
- SPIRES database also serves as a data provider to other NIH systems.

## What is SPIRES?How SPIRES Searches PubMed for NIH Publications

- SPIRES extracts all publications from PubMed that meet these search criteria:
- Where the publication year is >= 1995.
- Where the GRANT tag in the citation contains data AND
- Where the GRANT *string* contains an NIH organization code (2 letter phs_org_code or full abbreviation [ie-CA or NCI])

## SPIRES Database Limitations

- Not all publications related to a grant are necessarily cited in PubMed. For example, a grant that is part of a Center may cite the publication whereas the center grant may not.
- Not all journals in which grantees publish are cited in PubMed.
- Not all journals allow the author to acknowledge grant support.

## SPIRES History

- Project originated at NIEHS in 2001 by Dr. Ben Van Houten of DERT.
- Primary goal was to reliably match scientific publications from PubMed with their corresponding NIH extramural grant numbers, for purposes of program analysis.
- Version 1 of SPIRES completed in Spring 2002 by Paul Jordan and Ying Gao at NIEHS.
- Included data from 1995 to Present for NIEHS, NICHD, NIA and FIC.
- Data was downloaded and updated MANUALLY each month.

## Goals for NIH SPIRES Project NIH SPIRES initial development funded by a grant from NIEHS director in fall 2005.Development started in December 2005

- Expand database to include all NIH ICs.
- Further refine/improve grant number matching process.
- Improve data query and presentation tools.
*Major Goal:* _Create an __*automated*__ process that updates the publication database daily and reliably_.

## NIH SPIRES: Database Snapshot(Database statistics as of November 25, 2007)

- 24 NIH ICs represented
- Publication data for 1995 – Present
- 716,487  Total Publications
- 1,609,318 Project to Pub Matches
- Approx. 5000 Journals
- ISI Journal impact factor data for 1997 – 2005 publication years

## Why is SPIRES Different from Other Publication Search Tools?

- SPIRES query tool provides a TWO WAY search option, *in one system**:*
- Query publication data and view all related project data from eRA tables.
- Query eRA project data and view all related publication data.

## SPIRES Application Features

- Query publication data and view all related project data from eRA tables, including project history.
- Query eRA project data and view all related NLM publication data.
- Perform Oracle Text searches on publication and project data elements.
- Generate a variety of standard and custom reports.

## IC’s w/Pubs, Represented in SPIRESIncluded in initial data load and daily update

| CSR | NIA | NIDCR |
| --- | --- | --- |
| FIC | NIAAA | NIDDK |
| NCCAM | NIAID | NIEHS |
| NCI | NIAMS | NIGMS |
| NCRR | NIBIB | NIMH |
| NEI | NICHD | NINDS |
| NHGRI | NIDA | NINR |
| NHLBI | NIDCD | NLM |

## Publication Count by IC 1995 – PresentCurrent as of November 25, 2007

| NCI | 122683  | NICHD | 44923  | NIDCR | 14848  |
| --- | --- | --- | --- | --- | --- |
| NIGMS | 105013  | NIA | 40384  | NHGRI | 5336  |
| NHLBI | 100898  | NIDA | 27846  | NIBIB | 5061  |
| NIDDK | 78250  | NEI | 26836  | FIC | 4683  |
| NIAID | 77123  | NIEHS | 25015  | NINR | 4527  |
| NINDS | 62982  | NIAMS | 24620  | NLM | 3076  |
| NCRR | 51345  | NIAAA | 15516  | NCCAM | 1977  |
| NIMH | 50246  | NIDCD | 14869  | CSR | 5  |

## SPIRES Publication Count by Pub Year (11/25/07)

|  1995 | 47206 |
| --- | --- |
|  1996 | 47077 |
|  1997 | 47047 |
|  1998 | 47845 |
|  1999 | 50841 |
|  2000 | 52647 |
|  2001 | 53527 |
|  2002 | 55439 |
|  2003 | 59126 |
|  2004 | 63496 |
|  2005 | 66041 |
|  2006 | 68560 |
|  2007 | 55231 |

## Pub. to Project Number Matches1995 – PresentCurrent as of November 25, 2007

| NCI | 228059 | NICHD | 72557 | NIDCR | 23784 |
| --- | --- | --- | --- | --- | --- |
| NHLBI | 183420 | NIA | 70324 | NHGRI | 7842 |
| NIGMS | 178603 | NIDA | 58001 | FIC | 7169 |
| NIAID | 141203 | NEI | 55513 | NIBIB | 6206 |
| NIDDK | 127582 | NIEHS | 46168 | NINR | 5411 |
| NINDS | 100908 | NIAMS | 35925 | NLM | 4409 |
| NIMH | 100031 | NIAAA | 30180 | NCCAM | 2428 |
| NCRR | 81695 | NIDCD | 28336 | CSR | 6 |

## SPIRES Oracle Text Searches

*Publication Data that is text indexed:*
- Publication Titles
- Publication Abstracts
- Publication MESH terms
*Project Data that is text indexed:*
- Project Title
- Project Abstracts
- Project CRISP terms
*Can be searched simultaneously or separately.*
*Note: text searches are database intensive!*

## SPIRES Daily Update Process - 1

- The original NIEHS SPIRES system could only be updated once monthly, due to the labor intensive nature of downloading and refreshing the data.
- NIH SPIRES uses new technology to search PubMed automatically, each day, for new and revised publications.
*The daily update process handles all aspects of the load process, from raw data to finished database records that are linked to projects.*

## SPIRES Daily Update Process - 2

*The daily update process looks for*
- New publications not in SPIRES, using the standard criteria applied to the GRANT tag.
- Publications where the last revised date has changed.
- Publications where the MeSH terms indexing date has changed.

## SPIRES Daily Update ProcessHow it works - 1

- Step I: Pubmed Download
- Search PubMed for the following data types for designated NIH IC codes and download data:
- New Records, not in SPIRES
- Revised Records (existing)
- MeSH terms updated (existing)
- Step 2: IRDB Project Table Update
- Builds a project number lookup table optimized for speed in matching pubs to projects.

## SPIRES Daily Update ProcessHow it works - 2

- Step III: Unpack Publications
- Complex, multi-step process for translating and decoding publication data into usable format for SPIRES.
- Makes extensive use of XML parsing techniques.
- Extraction of data elements makes possible Oracle text indexing of pub data as well as optimizing data structures for querying.

## SPIRES Daily Update ProcessHow it works - 3

- Step IV: Decode Grant Strings
- Complex, multi-step process using pattern matching algorithms.
- Extensive recursive processing for best possible result.
- PL-SQL procedures and other techniques.

## SPIRES Daily Update ProcessHow it works - 4

- Step V: Match Grant Numbers
- Match decoded project numbers with eRA project numbers.
- Assign score to each match based on quality of match.
- Step VI: Complete Load Table Build
- Step VII: Build PUB_SUMMARY_T
- Step VIII: Push to Production Tables.

## Decoding NIH Grant Numbers

- Project Number data is available for a majority of the publication records downloaded from PUBMED.
- Each publication record may reference several NIH grant numbers.
- Grant numbers in PubMed are NOT stored in eRA recognized format.
- Many different string formats have been used in the system over the years.

## Decoding NIH Grant Numbers(continued)

- A few examples of the variations in grant number string formats:
- 1-R03-TW01490-01A/TW/FIC
- 5 R03 TW00857/TW/FIC
- GM-14312/GM/NIGMS
- P41RR-04293/RR/NCRR
- TW1064/TW/FIC
- R03 TW1064/TW/FIC
- U01-TW01015-01/TW/FIC
- AI 47053/AI/NIAID
- HD-24870/HD/NICHD
- This is only a sampling. There are many more!

## Decoding and Matching Grant Numbers

- SPIRES only matches on the base grant number: *Activity Code+IC+Serial Number*.
- Grant number string components are decoded from raw data using pattern matching algorithms that have been refined over the last 6 years.
- Decoded components are then matched to eRA project numbers in IRDB.
- Resulting matches are scored based on the quality of match.

## SPIRES Match Case Score SystemRates the “match quality” for each project number

## Matching Process: Summary Stats

|  | Total Number of records with match case of  5 | 426,982 |
| --- | --- | --- |
|  | % of  matched records with match case of  5 | 26.53 % |
|  | Total Number of records with match case of  4 | 628,868  |
|  | % of  matched records with match case of  4 | 39.08 % |
|  | Total Number of records with match case of  3 | 545,134 |
|  | % of  matched records with match case of  3 | 33.87 % |
|  | Total Number of records with match case of  2 | 1,581 |
|  | % of  matched records with match case of  2 | 0.10% |
|  | Total Number of records with match case of  1 | 6,753 |
|  | % of  matched records with match case of  1 | 0.42% |

*Statistics as of November 25, 2007*

## Overall Success for Decoding/MatchingSPIRES Pub to Project Match Success Rate: 98.48% (projects matched at some level)

| a | Total Publications Downloaded to Date | 716,487 |
| --- | --- | --- |
| b | Total Number of Rows in Match Table
(possible matches) | 1,609,318 |
| c | Total Number of DISTINCT Pubs Matched | 705,629 |
| d | % of Publications Decoded and Matched at Some Level ((c / a)*100) | 98.48% |

*Statistics as of November 25, 2007*

## Third Party Data in SPIRES

- Journal Impact Factors
- Impact factor data has been loaded for 1997 through 2005.
- Publication Citation Data
- Citation data is a valuable tool in analyzing publication data.
- Citation data must be purchased from ISI.
- Adding citation data would greatly expand the potential of the SPIRES data.

## Current Project Status

- The initial database load for NIH SPIRES was completed in February 2006.
- The daily update process for database maintenance was completed in March 2006. The system is currently in beta mode.
- Feb. 2007: Data analysis project for NIH Director’s Office.
- Fall 2007 Work in progress: development of web services, loading pubs for 1985 to 1994, development of a new PI/author matching algorithm.
- Partnership with QVR: NEW Bibliography Report in beta!

## New: QVR Bibliography Report (beta)

- Obtain a SPIRES bibliography report from selected projects resulting from QVR searches.
- Report is available in Program section of standard reports listing in QVR.
- Report is downloaded to your local machine in Excel pivot table format.
- Two formats offered: project summary and standard bibliography.

## Analysis Projects Using SPIRES Data

- Graph of R01s Showing correlation between number of grants per PI and number of papers published over time.
- Top 20 journals publishing NIH supported research 2001 – 2006.
- NIEHS T32 supported papers clustered by MeSH terms.

## Slide 31

## Slide 32

## OmniViz galaxy view of T32 supported papers 2000-2006, N = 943, clustered into 15 groups of papers, mesh terms included, many terms demoted

- neurotoxicity

- Cancer biology

- Genetic toxicology

- Epidemiology  Analysis of SNP’s

- Cancer biology 2

- Genetic toxicology 2

- Air pollution

-  chemical exposure/risk assessment

- Liver injury and pro-inflammatory

- Exposure & occupational health

- Patient oriented

- toxicology

- Apoptosis and cell death

- Reactive metabolites

- Receptor biology

## SPIRES Sample Queries

- Publication Search Screenshots
- Project Search Screenshots
- Example Query 1: Publication Search
- Example Query 2: Project Search

## SPIRES Publication Search Screen (1)

## SPIRES Publication Search Screen (2)

## SPIRES Project Search Screen (1)

## SPIRES Project Search Screen (2)

## SPIRES Project Search Screen (3)

## SPIRES Sample QueriesExample 1- Publication Search

*On Publication Search Page, enter:*
- IC = NIEHS
- Pub Date= 1997 Jan to 2000 Dec
- Author Name = VAN HOUTEN B
*Search Results in 15 Publications*
*Further limit search, enter:*
- Publication Text Search:
- Search in Title, Abstract, MeSH Terms:
- DNA and REPAIR
*New Search Results in 11 Publications*

## Example 1- Publication Search:Publication Search - Basic Hit List

- Click **Title** to bring up summary screen for that publication

## Example 1- Publication Search:Publication Summary Detail Page

- Features: links to project history, searched words highlighted

## Example 1- Publication Search:Publication Summary Detail Page

*Publication Detail Screen Features:*
- All basic publication information
- Publication abstract
- All searched words highlighted
- NIH supported projects cited by author
- Project numbers linked to project history
- Match score value indicates quality of grant number match.

## Example 1- Publication Search:Publication Summary Detail Page

- Click Project History Link for More Information

## Example 1- Publication Search:Project History Summary Page

*Project History page features:*
- Download spreadsheet of all publications related to this project
- Download current page project data into spreadsheet
*Coming Soon:* Project Title links to summary statement

## Example 1- Publication Search:Publication Search - Basic Hit List (2)

- Click **Project Count** to bring up project list for a publication

## Example 1- Publication Search:Project Support List for Publication

- Click **Project Count** to bring up pub project support list.
- Click **Project Number** to bring up Project History.

## SPIRES Sample QueriesExample 2- Project Search

*On Project Search Page, enter:*
- IC = NIEHS
- FY = 1997, 1998, 1999, 2000
- Project Text Search:
- Search in Project Title for:
- CHEMICAL and MIXTURES
*Search Results in 25 project records, listed by individual support year.*

## Example 2- Project Search: Project Search - Basic Hit List

- Click Project Number to bring up project summary statement
- Click Publication Count to bring up all pubs related to this project
- Click HISTORY icon to bring up project history hit list
*CHECK** projects you wish to add to a new publication search query*

## Example 2- Project Search: List of All Pubs for Selected Project

- Click publication title to bring up publication summary detail page.

## Example 2- Project Search: Select Projects for Publication SearchBuild a hit list of all pubs related to a group of projects

*1. Select projects for next pub search via check boxes*
- 2. Click Add to Scratch Pad

## Example 2- Project Search: Select Projects for Publication SearchBuild a hit list of all pubs related to a group of projects

- 1. Select projects for pub search via check boxes
*2. Click Add to Scratch Pad(1), then press Pub Search(2)*

**1**

**2**

## Example 2- Project Search:Selected Projects Automatically Transferred to Pub Search Screen Using Scratch Pad

- --Project List was automatically transferred to pub search from scratch pad
- --Simply press Search Pubs to get pub hit list for these projects.

## Example 2- Project Search:Publication Search Result Page (After Using Scratch Pad)

- On any pub search results screen, select projects for inclusion in a spreadsheet download.