# eRA Project Team Meeting

**Date:** June 25, 2002

**Time:** 9:00 am

**Location:** 6700 B Rockledge, Room 1205

**Chair:** John McGowan

[**Next Meeting:** July 23, 2002, 6700 B Rockledge, Room 1205]{.mark}

## Action Item

(OCO) Write article on SPIRES system for *Inside eRA*.

## Agenda Items

1.  **Feedback from the Steering Committee**

John McGowan (JJ) reported on his meeting with the eRA Steering
Committee on June 20. Paul Sieving replaces Marvin Cassman as chair. Joe
Ellis and Louise Ramm are two new members; Marvin Kalt has left the
committee. Judy Duff, NEI Executive Officer (XO), will oversee the
coordination of Steering Committee activities. She replaces Martha Pine
in this capacity. Lyn Albrecht will record minutes at Steering Committee
meetings.

At the June 20 meeting, JJ presented the advantages of J2EE
architecture; Donna Frahm detailed the costs of the migration. JJ then
explained eRA's vital need for knowledge discovery software by 2005.
External organizations already are analyzing NIH data with these
sophisticated tools, a situation which places us in a vulnerable
position. Richard Morris and the National Institute of Standards and
Technology (NIST) are working on a white paper to show what we can
accomplish for \$100-150K. A pilot project would be the most convincing
way to illustrate the value of knowledge management (KM). Belinda Seto
commented that Dr. Zerhouni is an advocate of KM; we need to get the XOs
on board. In response to Belinda's question about possible applications,
JJ suggested using Collexis to identify potential reviewers and possible
conflicts of interest. He also mentioned Tracker, software used by the
FBI to extract information from multiple sources (awards, newspapers,
Internet) to produce a daily report for the president.

Although JJ appealed to the Steering Committee for financial relief, we
have to operate as if no additional funds are forthcoming. This means
cutting costs and delaying schedules. The details are to be determined.

2.  **Small Business Innovation Research (SBIR) Update**

Jerry Stuck reported that summary statements for the 16 applications are
available in IMPAC II. Based on the scoring, Jerry expects that three to
five grants will be awarded soon. He meets this Thursday with NCRR, the
IC that will administer the grants.

3.  **National Research Service Awards (NRSA) and the Loan Repayment
    Program (LRP)**

Wally Schaffer gave a
[presentation](http://era.nih.gov/Docs/Loan_Repayment_Program-6-25-02.ppt)
on issues related to the Pediatric and the Clinical Research Loan
Repayment Programs. Considerable information about these programs is
available on the LRP website at <http://www.lrp.nih.gov/>. Recipients of
these awards must contractually agree to conduct research for a two-year
consecutive period in order to receive educational loan repayment of up
to \$35,000 annually, depending on total educational debt.

The LRP is administered by the Office of Loan Repayment and Scholarship,
directed by Marc Horowitz. Applicants use a web-based system to submit
applications to a stand-alone LRP system. JJ pointed out it would be a
more efficient use of NIH resources to roll the receipt, referral and
review components of LRP into the existing eRA infrastructure for access
by IC business processes. He stated that he would initiate a discussion
with Marc. After the meeting he will come back to the Project Team and
the Steering Committees to see if they would support an effort to bring
the issue of integrating the two systems to the NIH IT Board of
Governors (BOG). Marcia Hahn remarked that the "stovepipe" LRP system
was a real issue for grantees. JJ encouraged the Commons Working Group
(CWG) to forward their views to Marc Horowitz, who already has come to
eRA for assistance.

Wally informed the Project Team about a condition that requires NRSA
awards and the LRP to be closely coordinated. Postdoctoral NRSA
recipients in their *first or third year of support* were eligible for
LRP support in FY 2002. The second year of postdoc NRSA involves
repayment of a service obligation incurred during the first year of NRSA
support, and in 2002, eliminated the possibility of concurrent
participation in the LRP because recipients can't pay back two
obligations simultaneously. Existing provisions permit NRSA recipients
to extend the time to begin payback. Wally indicated the need for a
marker for LRP awards in payback files to prevent the second year of
NRSA support from serving as a payback year.

Michael Cox suggested that, for the short term, we manually enter an LRP
indicator in IMPAC II. Belinda pointed out that unlike other grants, LRP
establishes a relationship with an individual and not an institution. JJ
believes that we should refuse to take action until there is a coherent
plan and funding.

4.  **Scientific Publications Information Retrieval System (SPIRES)**

Thor Fjellstedt introduced Paul Jordan and Ben Van Houten who gave a
[presentation](http://era.nih.gov/Docs/SPIRES_System_6-25-02.ppt) on
SPIRES, a new web-based tool for retrieving and analyzing publication
data for NIEHS grantees. In developing the system, Ben and Paul linked
grant data in the IMPAC II IRDB with publications data in the National
Library of Medicine (NLM) flat files. The SPIRES database currently is
populated with records for 9,615 publications issued from 1995-August
2001.

The SPIRES query screen enables searching by publication parameters or
by project (grant) parameters. The resulting hitlist can be exported to
an Excel file for further manipulation. Clicking on the "detail" button
links to a publication summary record containing UI record code, title,
author, journal, grant number(s), abstract and keywords. It is also
possible to access PubMed to view or print the full article.

The publication summary also displays an impact factor, computed by
dividing the number of times the publication has been cited by the
number of articles published that year. The assumption is that the
higher the frequency, the more significant the science. It is also
possible to determine the number of articles published by a PI during a
given period as one measure of productivity.

The SPIRES reporting facility contains numerous "canned" reports in four
main categories: publications/organization; publications/author;
projects/organization; and projects/PI. The system can report on where
PIs are publishing and how many of their articles appear in the top
twenty journals. In addition to reporting on PI output and identifying
high-impact papers, SPIRES facilitates the evaluation of NIEHS grant
programs such as arsenic and oxidative stress.

As part of data quality validation, Jerry Phelps compared SPIRES
publication data to information contained in progress reports and
competitive renewals stored in the grant file. Overall, the accuracy
rate was 80% for "R" awards and 47% for "P" awards. Discrepancies are
due, in large part, to journals not abstracted by NLM and publications
that do not cite supporting grants. Based on these validation studies,
Paul Jordan and Jerry Nehis have made improvements in SPIRES' ability to
capture papers from PubMed.

Ben concluded with future directions. In the next phase, developers will
incorporate data for NIEHS cofunders. Ben is actively seeking other ICs
to partner with NIEHS. JJ suggested use of an evaluation proposal to
support SPIRES development by having interested ICs pay for use of the
system. Other plans include integrating SPIRES into the QVR component of
ECB and launching a website for grantees.

After the presentation, JJ asked about the pros and cons of SPIRES'
quantifiable methods. Ben replied that there has been pressure from GPRA
to measure productivity. The long-term goal, however, is to assess
productivity through a hybrid of quantitative and qualitative analysis.
Thor agreed and said SPIRES was a metric, which should be one part of a
total decision-support system. Eileen Bradley commented that SPIRES
places too much emphasis on numbers; she fears misinterpretation and
misuse, especially in the area of clinical research. Carlos Caban added
that the system is biased because studies with negative findings aren't
always published. JJ reiterated his concern that NIH doesn't have KM
tools and is not linking databases for a comprehensive picture.
Foundations currently are extracting and analyzing NIH data. Stephen
Hughes pointed out that, in a similar way, NIEHS has been downloading
NLM data for its own use.

## Attendees

Albrecht, Lyn

Bradley, Eileen

Caban, Carlos

Cain, Jim

Christian, Vonetta

Copeland, Zoe-Ann

Cox, Michael

Erickson, Bud

Fitzgerald, Steve

Fjellstedt, Thor

Frahm, Donna

Gibbs, Cassandra

Greenleaf, Andy

Hahn, Marcia

Hall, Dan

Hann, Della

Hausman, Steve

Hodgkins, Earl

Hughes, Stephen

Jordan, Paul

Maurer, JJ

Martin, Carol

McGowan, John

Miller, Maeve

Monheit, Madeline

Moore, Bob

Panniers, Richard

Patel, Kalpesh

Pearson, Johnnie

Schaffer, Wally

Seppala, Sandy

Seto, Belinda

Silver, Sara

Silverman, Jay

Soto, Tracy

Spitzberg, Bobbi

Stiggers, Bridget

Stuck, Jerry

Tucker, Jim

Twomey, Tim

Van Brunt, Virginia

Van Houten, Ben

Walker, Catherine

Zucker, Sherry