+--------------------------------+-------------------------------------+
| #  **De                        | # **DRAFT VHA HANDBOOK 1204.1**     |
| partment of Veterans Affairs** |                                     |
+--------------------------------+-------------------------------------+
| ###                            | ###                                 |
| Veterans Health Administration |                                     |
+--------------------------------+-------------------------------------+
| **Washington, DC 20420**       | **(Date)**                          |
+--------------------------------+-------------------------------------+

**SCIENTIFIC RESEARCH AND DEVELOPMENT PROJECTS**

**1. REASON FOR ISSUE.** This Veterans Health Administration (VHA)
Handbook provides procedures on application for support and scientific
merit review of research and development projects.

**2. SUMMARY OF MAJOR CHANGES.** The principal changes in this document
clarify and incorporate additional information on funding programs
within Health Services Research and Development Service (HSR&D).
Specific new information includes:

a\. General guidelines for Concept Papers for submission to HSR&D; for
submission of proposals for support through the HSR&D; for the merit
review process as implemented by HSR&D; and for funding for
developmental projects through HSR&D.

b\. The common procedures regarding investigator requirements,
application procedures and review policies for HSR&D support of
scientific research and development projects.

**3.** **RELATED DIRECTIVES.** VHA Directive 1204, VHA Handbook 1200.5,
VHA Handbook 1200.9, and VHA Handbook 1200.18.

**4. RESPONSIBLE OFFICE.** The Health Services Research and Development
Service (124) is responsible for the contents of this VHA Handbook.
Questions may be referred to (202) 254-0207, or by facsimile at (202)
254-0461.

**5. RESCISSION.** VHA Directive 1204.1, dated May 15, 2002, is
rescinded.

**6. RECERTIFICATION.** This VHA Handbook is scheduled for
recertification on or before July 30, 2010.

  ------------------------------- ---------------------------------------
                                  

                                  Jonathan B. Perlin, MD, PhD, MSHA, FACP

                                  Under Secretary for Health
  ------------------------------- ---------------------------------------

  --------------- ------- ------------------------------------------------
  DISTRIBUTION:   CO:     E-mailed 7/ /05

                  FLD:    VISN, MA, DO, OC, OCRO, and 200 -- E-mailed
  --------------- ------- ------------------------------------------------

**\`**

**CONTENTS**

**SCIENTIFIC RESEARCH AND DEVELOPMENT PROJECTS**

**PARAGRAPH PAGE**

1\. Purpose 1

2\. Project Types 1

a\. Investigator-initiated Research (IIR) 1

b\. Service-directed Research (SDR) 1

c\. Nursing Research Initiative (NRI) 2

3\. Application Requirements 2

a\. Eligibility 2

b\. Merit Review Application 2

4\. Concept Papers 3

a\. Scope 3

b\. Rationale 3

c\. General Requirements 3

d\. Review Schedule 3

e\. Review Results and Next Steps 4

f\. Inquiries and Additional Information 4

#### 5. Merit Review Applications 4

a\. Requirements for Principal Investigators (PIs) 4

b\. Required Approvals 4

c\. General Instructions 5

6\. Merit Review 6

a\. Scope 6

b\. Implementation 6

c\. General Review Criteria 7

d\. Reviewer Recommendations and Priority Scores 10

e\. Post-review Notification of Review Results 10

f\. Appeals 11

**CONTENTS** Continued...

**PARAGRAPH PAGE**

7\. HSR&D Funding For "Development" 12

a\. Scope 12

b\. Appropriate HSR&D IIR Support 12

c\. Non-appropriate HSR&D IIR Support 13

d\. Exceptions 13

#### **APPENDIXES**

### A Health Services Research and Development Scoring Guideline A-1

**SCIENTIFIC RESEARCH AND DEVELOPMENT PROJECTS**

**1. PURPOSE**

This Veterans Health Administration (VHA) Handbook presents the common
procedures regarding investigator requirements, application procedures,
and review policies. ***NOTE:** Unless the solicitation identifies an
exception, all procedures presented in this Handbook are applicable.
Exceptions and any requirements that are unique to each type of project
are specified in the appendixes.*

**2. PROJECT TYPES**

Project support from the Health Services Research and Development
Service (HSR&D) is based on scientific merit review and program
relevance. The same basic principles apply to all types of projects that
HSR&D considers for funding: Investigator-initiated Research (IIR),
Service-directed Research (SDR), and Nursing Research Initiative (NRI)
Projects. HSR&D project support is available through three funding
mechanisms, described as follows:

a\. **[Investigator-initiated Research (IIR)]{.underline}**

\(1\) The IIR program enables eligible Department of Veterans Affairs
(VA) clinicians and social scientists to pursue their research goals
that advance HSR&D research priorities and contribute to the quality,
effectiveness, and efficiency of VA health care. The IIR Program spans
the traditional areas of health services research (cost, quality, and
access) as well as emerging areas and current topics (e.g., patient
safety). Most projects are multi-disciplinary in approach, involving a
team of researchers with expertise in a variety of clinical specialties
and academic disciplines. Many of these studies involve data collection
at multiple sites to enhance generalizability and the eventual
translation of the findings into practice. .. ***NOTE:** Solicitations
for particular categories of IIRs may impose limits on duration and
total cost.*

# 

#  (2) Periodically, HSR&D publishes program announcements, or other types of research solicitations to inform the field regarding particular research priorities and opportunities within the IIR Program. Collectively, HSR&D refers to these announcements as "special solicitations." Special solicitations are highlighted in electronic announcements and are posted on HSR&D's website (<http://www.hsrd.research.va.gov/>). The announcement's expected "lifespan," submission deadlines and review dates, any special requirements, and expected investment are specified. Review may be carried out by the Scientific Merit Review Board as part of its regular deliberations, or by an ad hoc review subcommittee with more specialized expertise. Unless the solicitation identifies an exception, all policies and procedures presented in this Handbook are applicable. 

b\. **[Service-directed Research (SDR).]{.underline}** Periodically,
HSR&D invites submission of proposals that address a specific research
or development need identified in VA Central Office. (Depending on the
purpose of the research and the timeframe for completion, eligibility to
apply may be restricted (e.g., to investigators at established HSR&D
Centers) or there may be special requirements (e.g., matching funds).
Review of required SDR Concept Papers serves as a screen to identify the
most competitive applicants, who are then invited to submit full
proposals.

***NOTE:*** *IIR and SDR provide mechanisms for funding projects that
may be programmatically part of another activity. For example, some IIR
and SDR projects are part of the Quality Enhancement Research Initiative
(QuERI). As such, they may be funded through separate accounts; however,
all follow the basic policies and procedures set forth in this Handbook.
Any exceptions, for example, in funding limits, due dates, or who may
apply, are specified in the applicable solicitation.*

c\. **[Nursing Research Initiative (NRI)]{.underline}**

\(1\) NRI is a mentored research and capacity-building program that
HSR&D manages for the Office of Research and Development (ORD). Program
goals include developing nurses' research skills, encouraging nursing
research career opportunities, and developing independent nurse
investigators within VA. The Principal Investigator (PI) for all NRI
projects must be a clinically-active nurse and the PI must have a
mentor. In other respects, NRI adheres to the same general application
and review policies and procedures as HSR&D's IIR Program.

\(2\) The program announcement for NRI is reissued periodically to
incorporate any changes in research priorities or administrative
requirements. Eligible nurse investigators interested in applying need
to refer to the current NRI announcement, available on HSR&D's website
([http://www.hsrd.research.va.gov](http://www.hsrd.research.va.gov/)).
NRI proposals are reviewed at least twice each year, by a subcommittee
of the Scientific Merit Review Board. ***NOTE:** The applicant needs to
indicate whether the proposed research is most pertinent to the
interests of HSR&D, Biomedical Laboratory Research and Development
(BLR&D), Clinical Science Research and Development (CSR&D), or
Rehabilitation Research & Development (RR&D).* *Approved projects will
be funded according to the respective Service funding limits.*

**3. APPLICATION REQUIREMENTS**

a\. **[Eligibility.]{.underline}** Only "eligible" individuals may serve
as the PI or Co-Principal Investigator (Co-PI) on a VA-funded research
project (see VHA Handbook 1200.15). A prospective PI who is not
currently eligible may submit a proposal for consideration; however,
eligibility must be established before funding for an approved proposal
is initiated.

b\. **[Merit Review Application.]{.underline}** The specific forms that
must be included (and guidance for completing them), as well as page
limits for the narrative sections, and deadlines can be found on HSR&D's
website at:
[http://www.hsrd.research.va.gov](http://www.hsrd.research.va.gov/) .
Merit Review applications for all HSR&D projects must be submitted
through formal channels, consistent with all current instructions, and
received by the specified due dates. **Strict attention must be paid to
all application requirements and deadlines or Merit Review Applications
will not be reviewed.**

# 

# 

# 

# **4. CONCEPT PAPERS**

a\. **[Scope]{.underline}**

\(1\) Merit Review applications outside of the IIR program may require a
concept paper. These brief, preliminary papers are reviewed in order to
determine relevance to program goals and soundness of the research plan.
Approval of the Concept Paper is a prerequisite to submitting a full
research proposal to HSR&D.

\(2\) The Concept Paper is to be submitted by the PI through the local
Research and Development (R&D) office. The required signature of the
Associate Chief of Staff (ACOS) for R&D signifies local review and
ensures, at a minimum, local support and conformance to current VA
Central Office guidelines. ***NOTE:*** *Applicants are strongly
encouraged to obtain local assistance or review regarding scientific and
technical issues.*

###  b. [Rationale.]{.underline} HSR&D's requirement that investigators gain approval of a Concept Paper prior to preparation of a full proposal is designed to assist investigators as well as program administrators. Specifically, this preliminary review serves to:

\(1\) Identify and, when indicated, redirect proposed research that is
not appropriate to the goals of HSR&D or VA.

\(2\) Support HSR&D program planning by alerting administrators to
research projects that are under development.

###  c. [General Requirements.]{.underline} All Concept Papers must be prepared and submitted in accordance with current instructions and received by the applicable due date. 

d\. [Review Schedule.]{.underline} Merit review is conducted by an ad
hoc panel of peer reviewers by a pre-specified deadline.

e\. **[Review Results and Next Steps]{.underline}**

\(1\) **Notification.** Results of Concept Paper review will be provided
within six weeks.

\(2\) **Revision of Concept Paper.**

\(a\) Revision and resubmission of a Concept Paper is generally not an
option.

\(3\) **Proposal Submission.** Once notified of Concept Paper approval,
applicants are encouraged to submit a full application package by a
pre-specified deadline.

f\. **[Inquiries and Additional Information.]{.underline}** The local VA
facility R&D Office needs to be the investigator's first point of
contact for information about Concept Papers. The Administrative Officer
of the local R&D office may direct inquiries to the individual specified
as the HSR&D contact in the solicitation.

## 5. MERIT REVIEW APPLICATIONS

This paragraph provides general guidance regarding submission of Merit
Review Applications for support through HSR&D (see HSR&D's website:
[http://www.hsrd.research.va.gov](http://www.hsrd.research.va.gov/) for
Detailed Instructions).

a\. **[Requirements for PIs]{.underline}**

\(1\) **Eligibility**. The PI and any Co-PI of a proposed research study
must meet VA eligibility criteria before funding is initiated (see VHA
Handbook 1200.15).

\(2\) **Good Standing**. Investigators must fulfill their obligation to
complete a final report for any previous HSR&D-funded project and have
followed all requirements regarding properly reporting publications
before a new proposal will be reviewed.

\(3\) **Co-PIs.** HSR&D requires designation of a single PI as the point
of contact (corresponding PI). The same requirements and
responsibilities apply equally to the PI and any Co-PI.

\(4\) **Human Subjects Protection Training**. All individuals applying
for VA research project funding as PI or Co-PI are required to complete
an approved course in human subjects protection. Documentation
concerning this training is to be submitted along with any research
proposal (see VHA Handbook 1200.5). Once a project has been approved for
funding, the ACOS/R&D must ensure that all study personnel listed on the
project have received Human Subjects Protection training. It is the
responsibility of the ACOS/R&D to maintain the original training
certificates locally and to ensure that all annual training requirements
are met.

b\. **[Required Approvals.]{.underline}** All proposals submitted to
HSR&D must be approved by the R&D Committee at the VA facility.
Institutional Review Board (IRB) approval of each site involved in the
study is required before funds will be distributed to that site.

\(1\) **Research and Development Committee**. See VHA Handbook 1200.1.

\(2\) **IRB**. Most HSR&D studies involve human subjects or the use of
personal data. To ensure proper protections, proposals for all such
studies must be approved by the IRB at each designated project site
before funds will be distributed to that site. It is the PI's
responsibility to renew IRB approval annually for active projects. Every
site included in the proposed research must hold a current Assurance of
Compliance with provisions of the Federal Common Rule.

c\. **[General Instructions]{.underline}**

[**(1)** Intent to Submit. HSR&D may require notification of an
investigator's intent to submit a proposal for merit
review.]{.underline} The responsibility for a complete, properly
formatted, and timely submission of HSR&D's Intent to Submit information
and a proposal abstract lies with the R&D Office at the originating VA
facility. The Intent to Submit and Abstract must be submitted by the
designated deadline in order for a proposal to be reviewed.  **Proposals
that have not complied with this requirement will not be accepted for
review.** Current information as to the correct format and current
submission deadlines can be found at HSR&D's website
([http://www.hsrd.research.va.gov](http://www.hsrd.research.va.gov/)).

\(2\) **Receipt Dates**. Application deadlines for review by HSR&D's
Scientific Merit Review Board (SMRB) are posted on HSR&D's website
([http://www.hsrd.research.va.gov](http://www.hsrd.research.va.gov/)).
The same receipt dates apply for new and revised applications.

\(3\) **Proposal Limit**. A proposed PI may submit more than one
application to HSR&D per review cycle; however, an application that is
submitted to HSR&D may not be submitted to any other component of VA's
ORD (i.e., BLR&D, CSR&D, or RR&D)**.**

\(4\) **Revised Proposals**. Proposals that are approved by HSR&D's SMRB
(or one of its subcommittees), but are not funded, may be revised and
submitted for a new review. A revised proposal is expected to address
explicitly the issues, highlighted in the Summary Statement, which were
raised by reviewers of the previous proposal. HSR&D allows a total of
three proposal submissions: the original submission and two
resubmissions. All resubmissions need to be received within two years of
the original submission date (5 biennial merit review cycles). If the
proposal has not been funded within two years of the original submission
date, the project will be removed from further consideration.

\(5\) **Withdrawal.** Withdrawal of an application once Intent to Submit
information has been submitted requires formal notification by the
ACOS/R&D to CO (an e-mail from the ACOS/R&D is acceptable). The contact
person for this communication is the Merit Review Program Manager
(124R).

\(6\) **Corresponding PI.** The corresponding PI, or \"applicant,\" is
the individual who will have principal responsibility for the scientific
and technical direction of and the completion of the research. The
corresponding PI is responsible for the appropriate expenditure of
project funds and for ensuring that the project and staff complies with
all governing regulations.

\(7\) **Proposal Content and Format.** Proposals are to be prepared
using all current instructions and required Merit Review forms. Detailed
information is provided at HSR&D's website
([http://www.hsrd.research.va.gov](http://www.hsrd.research.va.gov/)).
Applications must be complete upon arrival in VA Central Office. Once
received, additional or replacement information will not be accepted
unless requested by HSR&D. The responsibility for a complete and timely
submission lies with the R&D Office at the originating VA facility.
***NOTE:** An incomplete application may be returned without review.*

\(8\) **Local Approvals.** All required forms, approvals, and
endorsements must be submitted by the PI's VA facility. If a PI
transfers to another VA facility after the application has been
submitted, new approvals and endorsements must be obtained. ***NOTE:**
The PI, through the R&D office, needs to notify the assigned HSR&D
portfolio manager about an expected transfer, and recognize that it may
delay review of the application, or start of the project.*

\(9\) **Off-site Research.** An investigator who plans to perform
research outside of a VA medical center, VA-owned or VA-leased space,
must request a waiver to perform the research off-site (see Handbook
1200.16).

\(10\) **Intellectual Property, i.e., Inventions and Transfer of New
Scientific Discoveries.** Refer to VHA Handbook 1200.18.

\(11\) **Inquiries.** Questions about administrative issues on the
application process need to be directed to the ACOS for R&D or
Coordinator for R&D at the applicant's facility. The ACOS for R&D may
communicate with the Scientific Merit Review Program Manager if
clarification or additional information is required. Questions regarding
scientific issues may be directed to the appropriate Portfolio Manager.

## 6. MERIT REVIEW

This paragraph provides guidance on the merit review process as managed
by HSR&D.

a\. **[Scope]{.underline}**

2.  

\(1\) HSR&D employs a system of rigorous scientific review to ensure the
scientific and technical merit of individual research projects and the
integrity of its programs. Each application is evaluated by a
multidisciplinary group of experts, from inside and outside VA, who
constitute the SMRB or one of its subcommittees. The recommendations of
the SMRB, the priority scores for approved projects, and reviewers'
specific comments guide the decisions of VA research administrators
regarding which projects to fund. In addition, VA research
administrators consider VA priorities, responsiveness of the proposed
work to solicitations, and the significance and importance of the
research to veterans and/or veterans\' healthcare.

\(2\) The scientific review process is essential to funding the best
science. Reviewers' assessments and suggestions are communicated to
applicants to help them understand the board's recommendation, to
improve already strong projects, and to assist applicants who may wish
to revise and resubmit their application.

##  b. [Implementation]{.underline}

## 

##  (1) Scientific Merit Review Board (SMRB) and Subcommittees

\(a\) HSR&D merit review is carried out by the SMRB and several
subcommittees. SMRB consists of a multidisciplinary panel of experts.
Each member is appointed for a 3 or 4-year term. Each subcommittee has a
chairperson. Members are researchers and clinicians from within VA and
external to VA. If additional expertise is required beyond that readily
available on the SMRB, ad hoc reviewer(s) with appropriate expertise are
recruited to supplement the review.

\(b\) Each subcommittee of SMRB is chaired by a current member of SMRB
and populated by VA and non-VA researchers with expertise appropriate to
the review group. Review of proposals responding to special HSR&D
solicitations may also be conducted by an ad hoc panel with expertise
appropriate to the topic.

## 

\(c\) SMRB is a chartered VA committee that is subject to rules of the
Federal Advisory Committee Act (FACA). In accordance with FACA
requirements, HSR&D announces each review meeting in the [Federal
Register]{.underline}*,* and the public is invited to attend the opening
announcements and instructions. During review of the research proposals,
deliberations are confidential, and the meeting is closed to the public.
As a learning opportunity, HSR&D may permit VA researchers to observe
portions of the review session that are closed to the general public.

\(2\) **Review Schedule**

\(a\) SMRB reviews IIR proposals at least twice each year.

\(b\) SDR and IIR proposals with special receipt dates are reviewed as
specified in the relevant solicitation.

\(3\) **Reviewer Responsibilities**

\(a\) Each proposal is assigned to reviewers with appropriate expertise
to review the scientific merit of the proposal, with one member
designated as the primary reviewer, one as secondary reviewer, and one
as tertiary reviewer. Care is taken to avoid reviewers having a real or
perceived conflict of interest (and, if a conflict is identified,
assignments are changed). All non-conflicted reviewers are expected to
participate in the review of every application, whether or not it is
specifically assigned to them and to vote on recommendations regarding
approval or disapproval.

#  (b) Prior to each review meeting, each reviewer independently prepares a written critique for each proposal to which the reviewer is assigned as primary, secondary, or tertiary reviewer. These critiques address the general review criteria listed (see subpar. 6c) as well as any special criteria that may be included in a particular research solicitation. These critiques (with reviewer identifiers removed) are sent to the applicant, along with notification of the review outcome and a summary of the discussion at the review meeting written by HSR&D staff. 

**(**c) **[General Review Criteria]{.underline}**

\(1\) **Adequacy of Response to Previou*s* Feedback Provided by HSR&D
Regarding the Proposed Study.** If the proposal is a re-submission, the
applicant will have received detailed comments on the previous proposal.
Any subsequent proposal is expected to highlight changes made in
response to such feedback or to defend the earlier plan.

\(2\) **Responsiveness to Research Priorities Solicitations.** HSR&D may
give special funding consideration to proposals that are responsive to
HSR&D or ORD solicitations for research. Investigators must indicate if
their proposal is responsive to a particular solicitation. Reviewers
will evaluate whether the justification provided by the investigator
adequately supports identifying the proposal as responsive to a
particular solicitation.

**(3) Scientific Significance** **and Originality.** Reviewers assess
the scientific significance, theoretical foundation, and originality of
the stated goals, objectives, and specific research questions and/or
hypotheses. Reviewers consider the proposed research in relation to
information and/or pilot data that the investigator provides regarding
prior work (by self and others), as well as information from other
sources that relates to the scientific significance and likely
contribution of the proposed work.

## 

\(4\) **Methods.** Reviewers assess the appropriateness of the research
design and specific methods proposed for conducting the research. The
following list contains some of the elements that reviewers consider, as
applicable to the particular project, and in accordance with their
particular expertise:

> \(a\) Study design (e.g., retrospective versus prospective,
> experimental, quasi-experimental, etc.).
>
> \(b\) Analytical approach (quantitative, qualitative, mixed methods).
>
> \(c\) Theoretical model and conceptualization of key components.
>
> \(d\) Population and sample, sampling plan, and/or comparison groups.
>
> \(e\) Statistical power.
>
> \(f\) Key variables and their measurement.
>
> \(g\) Data analysis plan.
>
> \(h\) Data collection issues, including respondent burden.
>
> \(i\) Definition and feasibility of any intervention.

\(5\) **Adequacy of Data.** Reviewers address the adequacy of data for
the proposed study. For primary data, reviewers consider the adequacy of
the proposed data collection instrument(s) or the plan for developing
and testing new instruments, as well as the feasibility and
appropriateness of data collection procedures. Secondary data issues to
be considered include: appropriateness, availability, accuracy, and
completeness. Applicants proposing to use existing databases need to
provide evidence of familiarity with these, and an awareness of the
idiosyncrasies and limitations of the data*.* For all types of data,
reliability, validity, and adequacy of quality control procedures are
important issues.

\(6\) **Project Organization and Management.** Reviewers consider the:

\(a\) Distribution of roles and responsibilities across project staff;

\(b\) Justification of Full-time Equivalent (FTE) employee allocations
for each project year;

\(c\) Plans for coordinating multiple participants, tasks, or sites;

\(d\) Reasonableness of the timeline showing important benchmarks and
products; and

\(e\) General feasibility of the management plan.

\(7\) **Investigator Qualifications.** Reviewers assess the expertise of
each investigator and each major consultant, including professional
credentials, institutional position, role in the project, expertise
(especially as reflected in publications), and relevant experience. All
reviewers assess the combined strength of the team in relation to the
objectives of the project and determine whether it encompasses all
needed skills and competencies.

\(8\) **Human Subjects.** Reviewers consider the risk/benefit ratio of
the study, analyzing whether the study places human subjects at risk of
physical or psychological harm and evaluating the adequacy of provisions
to minimize risk, protect participants' privacy and the confidentiality
of their records or responses, ensure informed consent, and minimize
respondent burden. ***NOTE:*** *In considering human subjects issues,
reviewers may question the decision of an IRB and may impose a stricter
standard (see VHA Handbook 1200.5).*

\(9\) **Inclusion of Women and Minorities.** Review of each proposal's
compliance with VA policy regarding the inclusion of minorities and
women in the study population is the responsibility of the R&D Committee
at each VA facility and VA human studies subcommittees. The HSR&D
reviewers are also responsible for considering the adequacy of
representation, and they do not need to concur with a decision by the
R&D Committee.

\(10\) **Facilities and Resources.** Reviewers evaluate the adequacy of
facilities and resources to carry out the proposed study. The proposal
must include evidence of support from the applicant\'s VA facility,
support from any additional study site(s), and documentation of any
agreements with consultants, or commitment of non-VA resources to the
study.

\(11\) **Budget.** Project budgets need to be appropriate to the
proposed work, sufficiently detailed, and well-justified. Reviewers
assess the reasonableness of the project timeline and costs allocated to
major budget categories. Personnel costs, and whether projects are
staffed appropriately, are key considerations. Items that appear to be
outliers, line items that change markedly from one year to another,
identical total annual requests, and large amounts for equipment,
travel, or subcontracts are scrutinized. ***NOTE:*** *Prior to any
funding decisions, all projects under consideration must undergo
administrative review of budgets by HSR&D staff. This review ensures
that VA research funds are not used for any inappropriate purposes, such
as patient care, salaries of Title 38 employees, and development
projects that lack a strong evaluation component. []{.mark}*

\(12\) **Importance of the Problem Addressed.** Reviewers assess the
importance of the problem or question that the proposed research seeks
to address, in terms of its prevalence, severity, urgency, cost, etc.,
for VA and the general public. ***NOTE:** Importance of the problem is
assessed independently of the investigator's approach.*

\(13\) **Contribution to VHA.** Reviewers consider the expected
contribution of findings of the proposed research to improving the
quality, effectiveness, or efficiency of health care in VA, or its
potential to improve the health status of veterans. This includes
consideration of the adequacy of the investigator's plans for
translating findings into practice.

##  d. [Reviewer Recommendations and Priority Scores]{.underline}

\(1\) At the conclusion of discussion on each proposal, reviewers make a
motion to recommend approval, conditional approval, or disapproval, and
then vote on the motion. The vote of the majority carries. For all
approved and conditionally approved proposals, individual reviewers then
assign a priority score, ranging from 10 (strongest) to 50 (weakest).
The committee's recommendation for each proposal and the mean priority
score, are critical elements in funding decisions made by the Director,
HSR&D. ***NOTE:** The definitions reviewers use in making
recommendations and assigning priority scores appear in Appendix A.*

\(2\) Each merit review session is independent. In the case of a
proposal that has been revised and resubmitted, it is possible that
reviewers will raise different or new issues concerning the proposed
research, and this may result in a less favorable recommendation than in
a previous review.

##  e. [Post-review Notification of Review Results]{.underline}

\(1\) **Preliminary Notification.** Following each review meeting, the
HSR&D review staff contacts the ACOS for Research and Development (R&D)
at each VA facility that submitted one or more proposal(s) to
communicate the review committee's priority score for each proposal from
that facility. ***NOTE:** Priority scores should not be construed as
funding decisions.*

\(2\) **Written Notification of Review Results**

\(a\) Written notification of the results of merit review generally is
sent to the facility Director within 6 weeks after each review meeting.
The notification includes the review committee's recommendation (i.e.,
approval or disapproval), priority score, and funding decision (unless
the proposal received conditional approval). Copies of this letter are
sent to the corresponding PI and the ACOS for R&D at the facility;
copies of the letter are sent to the Director of the Center of
Excellence (CoE), Research Enhancement Award Program (REAP), or Targeted
Research Enhancement Program (TREP), if applicable. With the
notification letter, the facility Director, the ACOS for R&D, and the PI
receive a summary statement that outlines the main points of the
reviewers' discussion and any administrative concerns. The PI and ACOS
for R&D also receive a redacted copy of all written critiques with
identifiers removed.

\(3\) **Questions about Reviews and/or Conditional Approvals.** HSR&D's
assigned Scientific Review Administrator is available to discuss with
the PI any questions about the individual critiques, the summary
statement, or a conditional approval.

f\. **[Appeals]{.underline}**

\(1\) In limited circumstances, the PI for a project that is either
disapproved or approved but not funded after three proposal reviews may
appeal the recommendation of the review board and request a new review
of the current proposal. An appeal may be appropriate when, in the
opinion of the investigator, the board did not understand the research,
missed relevant points, or was biased. A discrepancy between the
conclusions of previous and current review boards, unless due to an
error or oversight by reviewers, is not grounds for an appeal.
***NOTE:** The appeals process is to highlight potential procedural
errors, not to resolve differences on scientific points of view between
the applicant and the reviewers.*

\(2\) The appellant needs to prepare a formal letter that identifies the
specific points of possible misunderstanding or misinterpretations of
the proposal, or bias on the part of the scientific reviewers. The
summary statement provided to the applicant is the [only
document]{.underline} [acceptable]{.underline} as the basis for an
appeal. All information contained in the appeal must have been part of
the original proposal. ***NOTE:** Data obtained since the original
review or any additional information [can not]{.underline} be included.*

\(3\) The appeal document must be submitted through the local R&D
Committee and the ACOS for R&D, together with a supporting letter from
the facility Director, to the Director, HSR&D. Facility directors need
to be copied on the appeal. Any appeal needs to be received by VA
Central Office HSR&D within 6 weeks of written notification of the
review results. The original and two copies of the appeal must be sent
to the Director, HSR&D (124), VA Central Office, 810 Vermont Avenue, NW,
Washington, DC, 20420.

\(4\) If HSR&D determines that the appeal is appropriate, staff will
arrange for a new review by individuals with the relevant expertise, who
were not involved in the disputed review. The review is based on the
proposal as provided to the review board. Additional information and
clarification, including the PI's rebuttal letter, are not shared with
the ad hoc reviewers. This ad hoc review group makes a new
recommendation regarding approval or disapproval to the Director, HSR&D,
and assigns a new priority score if the proposal is approved. This
recommendation, priority score, and HSR&D Director's decision will be
promptly communicated to the facility Director, ACOS for R&D, and PI.

**7. HSR&D FUNDING FOR "DEVELOPMENT"**

a\. **[Scope.]{.underline}** The mission of the HSR&D Service includes
the support of scientifically meritorious and VA-relevant research and
development. Scientific activity that yields new knowledge (research)
and work resulting in new products (development) are often
interdependent; and HSR&D receives proposals that include elements of
both. This following clarifies the nature and extent of "developmental"
work that may be supported with HSR&D funds.

\(1\) HSR&D-IIR funding support is appropriate either for developing new
methods and tools for conducting research or for evaluating existing
methods and tools where efficacy has not been determined. Funding for
development is limited to methods and tools, not efficacy trials of
clinical interventions.

\(2\) All development work supported through HSR&D's IIR program is
expected to:

\(a\) Meet established standards of scientific peer review and
applicable review criteria.

\(b\) Be submitted with a well-developed evaluation plan.

b\. **[Appropriate HSR&D IIR support for development.]{.underline}**
HSR&D IIR support is appropriate for, but not limited to, development of
the following:

\(1\) Measures of quality of care,

\(2\) Measures of functional status,

\(3\) Measures of cognitive status,

\(4\) Methods for risk adjustment,

\(5\) Methods for measuring or estimating costs, and

# 

#  (6) Methods to elicit patient preferences. 

c\. **[Non-appropriate HSR&D IIR support for development.]{.underline}**
HSR&D-IIR support is not appropriate for development of the following:

\(1\) Clinical practice guidelines,

#  (2) Computer algorithms or reminder systems,

> \(3\) Databases or registries,
>
> \(4\) Computer software,
>
> \(5\) Clinical or surgical techniques,
>
> \(6\) Diagnostic tests,
>
> \(7\) Drugs,
>
> \(8\) Educational materials (for patients or providers),

####  (9) Equipment, and

######  (10) Medical devices.

**NOTE:** Products such as those listed in the preceding become
appropriate subjects of HSR&D-IIR research once they are developed and
there is some evidence of their efficacy or validity. For example, IIR
research might focus on implementation of clinical practice guidelines,
evaluation of outcomes related to a new drug, or adaptation of a
computerized reminder system for use in VA.

d\. **[Exceptions]{.underline}**

\(1\) Development that normally would not be funded but constitutes a
relatively small portion of the total time and budget requested for the
entire project may also be appropriate for HSR&D funding (i.e., minor
development with the primary focus on evaluation).

\(2\) Developmental work that is not appropriate under HSR&D's IIR
program may be appropriate for support by HSR&D through the SDR Program
or when the Director, HSR&D, identifies a specific need.

# **HEALTH SERVICES RESEARCH AND DEVELOPMENT SCORING GUIDE**

**1. APPROVAL.** Approval indicates that the proposed study has
potential to produce original, valid, and useful findings regarding one
or more important research question(s*)*. The priority score, ranging
from 10 to 50, indicates reviewers' overall enthusiasm based on their
combined assessment of the study's technical and/or scientific merit,
innovation, and importance. When assessing the scientific merit of a
proposal, the following factors should be considered:

GUIDELINES FOR SCORING OF PROPOSALS

**Scientific Merit Review Criteria Guidelines**

a\. **[Significance.]{.underline}** The proposal supports and/or
advances the health and health care of veterans and the research field
in general; addresses important scientific question and/or area;
potential contribution to scientific literature.

b\. **[Approach.]{.underline}** The proposal incorporates current
scientific and/or theoretical bases; hypothesis-driven; use of
appropriate research design/methods for addressing hypothesis;
feasibility of methods are clear.

c\. **[Innovation.]{.underline}** The proposal addresses new concepts
and/or gaps in the research area; potential for impact of findings on
existing field of research and/or treatment paradigms.

d\. **[Environment.]{.underline}** The proposal involves appropriate
knowledge and/or background and resources (e.g., equipment, staff) to
ensure completion of project; Institutional Review Board (IRB) IUCAC
Biosafety committee oversight sufficient.

#  e. **[Scientific Merit Parameters.]{.underline}** Using the preceding factors, the scientific merit of a proposal is assigned a score corresponding to the following: 

  ----------- -------------------------------------------------------------
  10 -- 15    The proposed research addresses important scientific area
  Excellent   that currently lacks needed knowledge base. Each hypothesis
              and/or research question is stated clearly and research
              design and/or methodology is appropriate. Research is
              innovative, representing state-of-the-art science. Potential
              findings may have a vital role in advancing the health and
              healthcare of veterans, and the scientific field in general.
              Resources listed suggest a very high probability of the
              project's completion.

  16 -- 22    The proposed research addresses important scientific area.
  Very Good   Each hypothesis and/or research question is stated clearly
              and the research design and/or methodology is appropriate,
              with a few minor exceptions. Potential findings may have an
              important role to the health and healthcare of veterans, and
              the research field in general. Resources listed suggest a
              high probability of the project's completion.

  23 -- 28    The proposed research addresses a valid area of
  Good        investigation. Each hypothesis and/or research question is
              stated clearly, but research design and/or methodology
              contain key flaws that should be corrected. Potential
              findings may contribute to the health and healthcare of
              veterans, and the field in general. Resources listed suggest
              the project could be completed.

  29 -- 34    The proposed research requires further preliminary data to
  Fair        warrant investigation as a viable area of research.
              Hypothesis(es) and/or research question(s) is(are) not clear
              and research design and/or methods contain significant flaws.
              It is not clear how potential findings would contribute to
              the health and health care of veterans, and the field in
              general. It is unclear whether the resources listed are
              sufficient to ensure project completion.

  35 -- 50    The proposed research does not appear to address an important
  Poor        scientific question/area. Hypothesis(es) and/or research
              question(s) is(are) not stated clearly and/or research design
              and/or methodology is inappropriate or contains uncorrectable
              flaws. Design and/or methodological limitations hinder any
              significant conclusions that would contribute to the health
              and health care of veterans, and/or the field in general.
              Resources listed do not suggest that the project will be
              completed.
  ----------- -------------------------------------------------------------

**2. CONDITIONAL APPROVAL**

a\. The importance of the research question(s) is \"high\" or
\"exceptional\" and the plan is solid, but specific technical or
scientific issue(s) diminish reviewers\' enthusiasm for the project. The
required modification(s) are discrete and limited, and addressing them
would not involve fundamental redesign of the study. Further, the
limited number and relative simplicity of the required modifications
make it reasonable for the investigator to provide the requested
response within 4 weeks of receiving the review notification.

b\. When reviewers recommend Conditional Approval, they assign a score
to represent their overall enthusiasm for the study as if the
investigator has made appropriate changes to address the required
modifications or to justify the original approach.

c\. The investigator\'s response to the required modification(s) is
reviewed by Health Services Research and Development (HSR&D) staff and
may be reviewed by one or more reviewers when additional expertise is
needed. The individual(s) conducting this review makes a recommendation
as to whether the investigator\'s response is satisfactory.

d\. If the investigator cannot respond within 4 weeks, or if the
response does not satisfy reviewers, a new proposal must be submitted.

**3. TRIAGE**

a.  The Scientific Merit Review Board may decide that a proposal is not
    ready for review and may remove the proposal from review
    consideration (triage).

b.  Generally, if the primary, secondary, and tertiary review scores are
    30 or greater, the proposal will not be reviewed by the full panel.
    If only two of the reviewers score the proposal greater than 30, the
    third reviewer will have the option to agree with the majority and
    recommend that the study not be reviewed by the full panel.

c.  The PI will not be given a summary score or summary bullets and will
    be advised that

1\) the proposal was not reviewed by the full panel (not scored) and

2\) any resubmission needs to address the key issues raised in the
written critiques.

**4. DISAPPROVAL.** Disapproval is warranted if the proposed study is
unethical or is radically flawed with respect to significance, approach,
or feasibility. Proposals that are disapproved are not given a numerical
score and may not be resubmitted.