# Presentation: 425282

## Lessons Learned From Peer Reviews of Other National Programs

- John W. Radin
- National Program Leader
- Plant Physiology and Cotton

## Plant Biological and Molecular Processes

**ARS National Program 302**
**John W. Radin and Kay W. Simmons, Co-Leaders**

**Hypoallergenic soybeans:**
**Good for humans and pigs**

**Late blight-resistant potatoes**
**through genetic engineering**

**$33.9 Million Annual Research Budget**
**99 Scientists (FTEs)**
**37 Locations**

## Plant Biological and Molecular ProcessesLaboratory Locations

## Plant Biological and Molecular ProcessesTwo-Year Initial Timeline for Program Establishment

**1999**
**June:****  ****Customer Workshop I**
**Sept.:****  ****Customer Workshop II**
**Dec.:****   Draft Action Plan Written and Made Available**

**2000**
**Feb.:****  ****Customer Feedback Incorporated into Action Plan**
**Sept.:**** ****Project Plans Submitted for Peer Review**
**Nov.:****  ****Plans Reviewed by External Peer Panels**** **

**2001**
**Mar.:****   ****Revised Plans Submitted as Needed**
**June:****  ****Plans Approved and Activated**

## Plant Biological and Molecular Processes Peer Review Results 

**Results before revisions, Nov. 2000 (36 projects)**

**Results after revisions, June 2001 (35 projects)**

## Plant Biological and Molecular ProcessesPeer Review Follow-up

- Of 36 initial projects, 13 were rated “Needs Major Revision” and 1 was rated “Unfeasible.”  Both categories require resubmission to the panel.
- One sub-standard project was discontinued, and the personnel and resources were reassigned.  The others were extensively revised with much closer oversight from the Area Directors and NPS.
- On second review, 1 project was rated “Needs Major Revision.”  It underwent a second round of restructuring, with even closer oversight.
- The last project was approved in Jan. 2002, 14 months after the peer panel met.

**Moral: Take the process seriously**

## The Top 5 Causes of Failure

**Contingencies are addressed carelessly if considered at all.**  Few research programs progress exactly as planned.  What will you do if the research plan does not work?

- _Example of poor contingency plan_
- “If we are unable to demonstrate potential for effective biological control with this predator species, we will work with other species.”

- _Needed_
- (1) Which new insect species will be evaluated? (2) Why are they expected to be useful? (3) What will be the sources for the new species? (4) Do new handling and rearing procedures need to be developed?

## The Top 5 Causes of Failure

- 4	**Cooperators are almost always needed because they bring in specialized skills, but the plans ignore or underplay that need.**  Panels look for specific documentation of collaborations, i.e., letters with a description of what the cooperator will actually do.

- _Example of poorly documented cooperation_
- “Allellopathic effects of cover crops will be evaluated in the field by Dr. John Smith.”

- _Needed_
- Details of Dr. Smith’s contribution: (1) What crops/rotations? (2) What cover crops? (3) Which weeds?  Dr. Smith must also supply a letter describing his input.

## The Top 5 Causes of Failure

- 3	**Plans are unfocused, or objectives are unrelated to each other.**  Panels look for objectives that clearly build upon each other. Do not combine unrelated objectives just because they seem interesting or convenient for management purposes.

- Reviewers evaluate whether the planned work will reach the stated goal.  This includes (1) technical quality and (2) whether the information acquired will be interpretable and applicable.  The latter usually requires a multi-faceted coordinated approach.  Both are equally important.

## The Top 5 Causes of Failure

- 2	**Too many objectives dilute resources and minimize the likelihood of success.  **Describe only what you expect to accomplish during the 5-year life of the project.  Provide preliminary data, if you have it, to show feasibility.

- _Example of over-written plan_
- “We will genetically transform insects with a conditional lethal gene, evaluate its stability, inheritance, expression, and phenotypic effects, obtain permits for release of the transgenic insects, and monitor their effectiveness in field trials.”

- _Needed_
- A better grip on reality!

## The Number 1 Cause of Failure

**Plans are written without acknowledging parallel work elsewhere in ARS, and without building multi-laboratory teams to take advantage of additional resources. **This is a major reason for the workshop.  Identify and make those connections while you are here.

- The panels have a unique broad view of research across ARS.  Most have been greatly distressed when they see failure to communicate, cooperate, and coordinate.  If other scientists are collecting relevant data, plan and work jointly with them.

## What Has Peer Review Accomplished So Far? 

- ARS research is given much higher credibility than previously (both by groups evaluating the process and by peer scientists who participate in panels).
- Panels do not move or redirect funds, which Congress has appropriated for specific locations and objectives.  Instead, they identify the poorest projects for special attention and upgrading (unlike a competitive grants panel, which concentrates on only the best proposals).  Without question, this has been good for ARS without altering priorities.
- The best projects have little trouble with peer review. (See Top-5 list of problems).
- The system is working as intended.