Internet Assisted Review

Scope Document

Version 1.09 05/16/02 6:00 PM

Revision History

  ---------- ------------- ------------------------------------- ------------------
  **Date**   **Version**   **Description of Change**             **Author**

  2/24/02    1.0           Initial Draft                         Tracy Soto

  3/7/02     1.01          Modified per IAR Focus Group comments Tracy Soto
                           (3/4/02)                              

  3/18/02    1.02          Modified per IAR Focus Group comments Tracy Soto
                           (3/14/02)                             

  3/22/02    1.03          Modified per IAR Focus Group comments Tracy Soto
                           (3/18/02)                             

  4/1/02     1.04          Modified per IAR Focus Group comments Tracy Soto
                           (3/25/02) and security                
                           recommendations for registration and  
                           account creation (3/28/02)            

  4/4/02     1.05          Modified per IAR Focus Group comments Tracy Soto
                           (4/1/02) and security recommendations 

  4/10/02    1.06          Modified per IAR Focus Group comments Tracy Soto
                           (4/8/02)                              

  4/11/02    1.061         Modified per clarification of         Tracy Soto
                           security recommendations for          
                           registration and login process        
                           (4/11/02)                             

  4/17/02    1.07          Modified per IAR Focus Group comments Tracy Soto
                           (4/15/02)                             

  4/26/02    1.08          Modified per IAR Focus Group comments Tracy Soto
                           (4/15/02)                             

  5/3/02     1.08          Added Appendix A, B, C for            Tracy Soto
                           registration email, PIN letter and    
                           user agreement language.              

  5/10/02    1.09          Modified per IAR Focus Group minutes  Tracy Soto
                           (5/6/02)                              

  5/16/02    1.09          Modified per new streamlining         Tracy Soto
                           requirements and draft screens.       

                                                                 

                                                                 
  ---------- ------------- ------------------------------------- ------------------

Table of Contents

# [1. Introduction 6](#introduction) {#introduction-6 .TOC-2}

## [1.1 Background 6](#background) {#background-6 .TOC-3}

## [1.2 Purpose 6](#purpose) {#purpose-6 .TOC-3}

## [1.3 Definitions, Acronyms, and Abbreviations 6](#definitions-acronyms-and-abbreviations) {#definitions-acronyms-and-abbreviations-6 .TOC-3}

## [1.4 References 7](#references) {#references-7 .TOC-3}

# [2. Positioning 7](#positioning) {#positioning-7 .TOC-2}

## [2.1 Business Opportunity/Scope 7](#business-opportunityscope) {#business-opportunityscope-7 .TOC-3}

# [3. Stakeholder and User Descriptions 8](#stakeholder-and-user-descriptions) {#stakeholder-and-user-descriptions-8 .TOC-2}

## [3.1 User Environment 8](#user-environment) {#user-environment-8 .TOC-3}

## [3.2 Stakeholder Profiles 8](#stakeholder-profiles) {#stakeholder-profiles-8 .TOC-3}

## [3.3 User Profiles 10](#user-profiles) {#user-profiles-10 .TOC-3}

# [4. Product Overview 12](#product-overview) {#product-overview-12 .TOC-2}

## [4.1 Product Perspective 12](#product-perspective) {#product-perspective-12 .TOC-3}

## [4.2 Summary of Capabilities 14](#summary-of-capabilities) {#summary-of-capabilities-14 .TOC-3}

## [4.3 Assumptions and Dependencies 14](#assumptions-and-dependencies) {#assumptions-and-dependencies-14 .TOC-3}

## [4.4 Cost and Pricing 14](#cost-and-pricing) {#cost-and-pricing-14 .TOC-3}

## [4.5 Licensing and Installation 15](#licensing-and-installation) {#licensing-and-installation-15 .TOC-3}

# [5. Product Features 15](#product-features) {#product-features-15 .TOC-2}

## [5.1 System (SYS) Feature 16](#system-sys-feature) {#system-sys-feature-16 .TOC-3}

#### [5.1.1 Error Handling 16](#error-handling) {#error-handling-16 .TOC-4}

#### [5.1.2 Usage Reporting 16](#usage-reporting) {#usage-reporting-16 .TOC-4}

#### [5.1.3 System Control 16](#system-control) {#system-control-16 .TOC-4}

#### [5.1.4 Browser Interface 16](#browser-interface) {#browser-interface-16 .TOC-4}

#### [5.1.5 Interface Conventions 16](#interface-conventions) {#interface-conventions-16 .TOC-4}

#### [5.1.6 On-line help 16](#on-line-help) {#on-line-help-16 .TOC-4}

#### [5.1.7 Release Notes 16](#release-notes) {#release-notes-16 .TOC-4}

#### [5.1.8 Bug Status 16](#bug-status) {#bug-status-16 .TOC-4}

#### [5.1.9 Availability 16](#availability) {#availability-16 .TOC-4}

#### [5.1.10 Performance 16](#performance) {#performance-16 .TOC-4}

#### [5.1.11 Auditing 16](#auditing) {#auditing-16 .TOC-4}

#### [5.1.12 Exception Reporting 16](#exception-reporting) {#exception-reporting-16 .TOC-4}

#### [5.1.13 External Interfaces 16](#external-interfaces) {#external-interfaces-16 .TOC-4}

## [5.2 Release Meeting to IAR / IAR Control Center 18](#release-meeting-to-iar-iar-control-center) {#release-meeting-to-iar-iar-control-center-18 .TOC-3}

## [5.3 Pre-Registration System Process 21](#pre-registration-system-process) {#pre-registration-system-process-21 .TOC-3}

## [5.4 User Registration 21](#user-registration) {#user-registration-21 .TOC-3}

## [5.5 Log on to IAR 23](#log-on-to-iar) {#log-on-to-iar-23 .TOC-3}

## [5.6 Password Replace 23](#password-replace) {#password-replace-23 .TOC-3}

## [5.7 Account Access and Expiration 24](#account-access-and-expiration) {#account-access-and-expiration-24 .TOC-3}

## [5.8 Changing Passwords 25](#changing-passwords) {#changing-passwords-25 .TOC-3}

## [5.9 Submit Phase 25](#submit-phase) {#submit-phase-25 .TOC-3}

## [5.10 Read Phase 31](#read-phase) {#read-phase-31 .TOC-3}

## [5.11 Streamlining 33](#streamlining) {#streamlining-33 .TOC-3}

## [5.12 Edit Phase 34](#edit-phase) {#edit-phase-34 .TOC-3}

## [5.13 IC Program Officer Access 34](#ic-program-officer-access) {#ic-program-officer-access-34 .TOC-3}

## [5.14 Reports 35](#reports) {#reports-35 .TOC-3}

## [5.15 Summary Statement Assembly 36](#summary-statement-assembly) {#summary-statement-assembly-36 .TOC-3}

## [5.16 Purging/Meeting Closure 38](#purgingmeeting-closure) {#purgingmeeting-closure-38 .TOC-3}

## [5.17 Other General Features 39](#other-general-features) {#other-general-features-39 .TOC-3}

# [6. Constraints 39](#constraints) {#constraints-39 .TOC-2}

# [7. Quality Ranges 39](#quality-ranges) {#quality-ranges-39 .TOC-2}

# [8. Precedence and Priority 40](#precedence-and-priority) {#precedence-and-priority-40 .TOC-2}

# [9. Other Product Requirements 40](#other-product-requirements) {#other-product-requirements-40 .TOC-2}

## [9.1 Applicable Standards 40](#applicable-standards) {#applicable-standards-40 .TOC-3}

## [9.2 System Requirements 40](#system-requirements) {#system-requirements-40 .TOC-3}

## [9.3 Performance Requirements 40](#performance-requirements) {#performance-requirements-40 .TOC-3}

## [9.4 Environmental Requirements 40](#environmental-requirements) {#environmental-requirements-40 .TOC-3}

## [9.5 Security Recommendations 40](#security-recommendations) {#security-recommendations-40 .TOC-3}

# [10. Documentation Requirements 40](#documentation-requirements) {#documentation-requirements-40 .TOC-2}

## [10.1 User Manual 41](#user-manual) {#user-manual-41 .TOC-3}

## [10.2 On-line Help 41](#on-line-help-1) {#on-line-help-41 .TOC-3}

## [10.3 Installation Guides, Configuration, Read Me File 41](#installation-guides-configuration-read-me-file) {#installation-guides-configuration-read-me-file-41 .TOC-3}

## [10.4 Labeling and Packaging 41](#labeling-and-packaging) {#labeling-and-packaging-41 .TOC-3}

## [10.5 Appendix A: Registration Email to Reviewer 42](#appendix-a-registration-email-to-reviewer) {#appendix-a-registration-email-to-reviewer-42 .TOC-3}

## [10.6 Appendix B: Letter with PIN to Reviewer 43](#appendix-b-letter-with-pin-to-reviewer) {#appendix-b-letter-with-pin-to-reviewer-43 .TOC-3}

## [10.7 Appendix C: IAR User Agreement 44](#appendix-c-iar-user-agreement) {#appendix-c-iar-user-agreement-44 .TOC-3}

####   {#section .TOC-4}

# Introduction

## Background

A Web-based system to manage the process of electronic submission of
critiques by Reviewers was developed by the National Institute of
Allergy and Infectious Diseases (NIAID). This system, Electronic Review
(ER), has been successfully implemented at several ICs and has provided
proof of concept for this electronic process. The NIAID system will be
used as a model for development of an eRA system. An eRA
Internet-Assisted Review system will expedite the scientific review of
grant applications by standardizing the current process of critique and
initial priority scores submissions by reviewers via the Internet.
Currently, Reviewers usually do not submit their critiques before the
actual meeting and they do not have the opportunity to see others'
critiques before the meeting. When critiques are finally submitted, they
may not all be in the same format. Since critiques are used to build the
summary statement body text, this method poses problems for staff. An
Internet Assisted Review system would improve this process. Review
meetings would contain more informed discussions because reviewers would
be able to read the evaluations entered by others prior to the review
meeting (except where there is a conflict of interest). The system will
also serve to facilitate the generation of summary statements since all
critiques would be submitted in the same electronic format and be stored
centrally.

## Purpose

The purpose of this document is to define the scope and high-level
business requirements of the Internet Assisted Review System. The
structure and content of this document is based on the Rational Unified
Process (RUP). It focuses on the capabilities and features needed by the
stakeholders and the target users. The detailed requirements that are
derived from these features are specified in the Software Requirements
document (which will include Use Cases) and the Supplementary
Specifications document.

The Internet Assisted Review System will be developed in multiple
releases. This document will be a living document. Initially it will
focus on functionality to be delivered in releases 1 and 2. As
additional upgrades are planned in the future, this document will evolve
to capture the capabilities and features for those future releases.

## Definitions, Acronyms, and Abbreviations

SRA Scientific Review Administrator

GTA Grants Technical Assistant

PI Principal Investigator

ER Electronic Review (NIAID system)

RUG Review Users Group

IAR Internet Assisted Review

COI Conflict of Interest

## References

-   *IMPAC II Peer Review User Guide*, Version 2.1.1.0

-   Information provided at NIAID's ER Info site
    <http://grants.niaid.nih.gov/review/staff/SRA_Erhelp_cvrpg-grants.htm>

**Documents Submitted 2/11/02:**

-   *Business Requirements for an IMPACII Internet Assisted Peer Review
    System* From a Sub-Committee of the CSR Information Resources
    Advisory Committee (contact: Richard Panniers)

-   Attachment A: Dr. Everett Sinnett's *Design Issues For Electronic
    Critique System (ECS)*

-   Attachment B: Dr. Thomas Tatham\'s request for ER enhancements

**Proposed Focus Group Meetings Scheduled for:**

March 4, 2002; March 14, 2002; March 18, 2002; March 25, 2002; April 1,
2002; April 8, 2002; April 15, 2002; April 25, 2002; May 6, 2002; May
13, 2002; May 20, 2002; June 3, 2002.

# Positioning

## Business Opportunity/Scope

An eRA-developed Internet-Assisted Review system will help expedite the
scientific review of grant applications by providing a standard process
for Reviewers to submit their critiques and initial priority scores via
the Internet. Currently, for staff not using the NIAID ER system,
reviewers might submit their critiques and initial priority score using
several methods including paper copies, diskettes, or email attachments,
but usually not before the actual meeting. Since critiques are used to
build the summary statement body text, each of these current methods
pose problems for staff. Data provided in paper copies must be manually
entered or scanned, electronic documents provided by email or diskette
may have been written in different, incompatible word processing
formats, and all must be combined into a single document. Electronic
documents submitted have the potential of containing a computer virus.
Another major flaw in the current business model is that reviewers do
not have the opportunity to review other critiques before the meeting.

An eRA-developed Internet-Assisted Review system would eliminate many
problems evident in the current method. All critiques would be submitted
in the same electronic format (Word \*.doc or WordPerfect \*.wpd). After
the deadline for submission has passed, reviewers would be able to read
the evaluations entered by others prior to the review meeting (except
where there is a conflict of interest). This pre-meeting review of
critiques would provide for more informed discussions at the review
meeting. The SRA/GTA would also have the ability to generate a
preliminary report of upper and lower scores. Subsequent to the meeting,
reviewers would be permitted to update their critiques; the system could
also serve to facilitate the generation of summary statements.

Providing this system to the eRA user community would offer several
major benefits. First, critiques would be available immediately after
the review meeting. This greatly expedites the creation of summary
statements. Staff who currently use the NIAID system report that this
method has saved them 1--3 weeks time. Second, this method greatly
reduces human errors associated with manipulation of the documents and
problems encountered using computer diskettes. Third, the system would
expedite the approval/funding process via easier, more efficient
administration of reviews (e.g., summary statement preparation by NIH
staff). Fourth, it would improve the overall quality of reviews and
permit more efficient and effective use of reviewers' time at meetings.

# Stakeholder and User Descriptions

## User Environment

Since IAR essentially facilitates the exchange of information between
NIH Scientific Review Administrators (SRAs), Grants Technical Assistants
(GTAs) and Reviewers, these are the major users groups and stakeholders
of the system. While each IC may hold their own Reviews, a majority of
Reviews are held by the Center for Scientific Review (CSR). SRAs may be
considered primary users; GTAs also work closely with the SRAs and may
perform similar functions in the system.

## Stakeholder Profiles

This section describes the stakeholders' profiles, in terms of their
roles, responsibilities, success criteria, and involvement in this
development effort. The following terms will help define a stakeholders'
profile:

> **Representative**---the stakeholders' envoy to the project, this will
> either be the name (or names) of individual(s), or a specific body of
> people
>
> **Description**---a brief explanation of the stakeholder type
>
> **Type**---the stakeholders' expertise, technical background, and
> degree of sophistication
>
> **Responsibilities**---The stakeholders' key tasks in this effort;
> that is, their interest as a stakeholder. Examples might be "captures
> details," "produces reports," or "coordinates work."
>
> **Success Criteria**---the stakeholders' definition of accomplishment
> of this project
>
> **Involvement**---the stakeholders' role in this project, if any
>
> **Deliverables**---any documents or resultant products the stakeholder
> produces, and for whom
>
> **Comments/Issues**---problems that interfere with success and any
> other relevant information (these would include trends that make the
> stakeholders' job easier or harder)

+---------+------------+------------+--------------------------------+
| Stak    | Repre      | Profile    |                                |
| eholder | sentatives |            |                                |
+=========+============+============+================================+
| NIH IT  | J. J.      | D          | IT Project Management          |
| Man     | McGowan,   | escription |                                |
| agement |            |            |                                |
|         | James Cain |            |                                |
+---------+------------+------------+--------------------------------+
|         |            | Type       | Manages IT finances and        |
|         |            |            | priorities                     |
+---------+------------+------------+--------------------------------+
|         |            | Respon     | Responsible for all aspects of |
|         |            | sibilities | the eRA project in general,    |
|         |            |            | and all aspects of IMPAC II    |
|         |            |            | project in particular. Formal  |
|         |            |            | management reviews, as defined |
|         |            |            | in the "eRA J2EE Project       |
|         |            |            | Management Plan."              |
+---------+------------+------------+--------------------------------+
|         |            | Success    | Success is completion of the   |
|         |            | Criteria   | project within approved        |
|         |            |            | budget, and fulfillment of     |
|         |            |            | user needs in a timely manner. |
+---------+------------+------------+--------------------------------+
|         |            | I          | Project guidance and review    |
|         |            | nvolvement |                                |
+---------+------------+------------+--------------------------------+
|         |            | De         | Responsible for delivering the |
|         |            | liverables | Internet Assisted Review       |
|         |            |            | system to the user community.  |
+---------+------------+------------+--------------------------------+
|         |            | Comments/  | None.                          |
|         |            | Issues     |                                |
+---------+------------+------------+--------------------------------+
| Group   | Eileen     | D          | Communicates needs of NIH      |
|         | Bradley    | escription | community to the development   |
| A       |            |            | team.                          |
| dvocate |            |            |                                |
+---------+------------+------------+--------------------------------+
|         |            | Type       | Possesses strong communication |
|         |            |            | and facilitation skills, along |
|         |            |            | with good domain expertise.    |
+---------+------------+------------+--------------------------------+
|         |            | Respon     | Manages expectations of the    |
|         |            | sibilities | users. Approves requirements   |
|         |            |            | documents. Works with the      |
|         |            |            | analyst and development team   |
|         |            |            | to set priorities.             |
+---------+------------+------------+--------------------------------+
|         |            | Success    | Project meets the              |
|         |            | Criteria   | specifications that have been  |
|         |            |            | agreed upon with the NIH       |
|         |            |            | community in a timely manner.  |
+---------+------------+------------+--------------------------------+
|         |            | I          | Project guidance and review    |
|         |            | nvolvement |                                |
+---------+------------+------------+--------------------------------+
|         |            | De         | Responsible for delivering the |
|         |            | liverables | system to the user community.  |
+---------+------------+------------+--------------------------------+
|         |            | Comments/  | None                           |
|         |            | Isssues    |                                |
+---------+------------+------------+--------------------------------+

##  User Profiles 

This section describes in detail the profile of each system user, in
terms of their roles, responsibilities, success criteria, and
involvement in the development effort. The same aspects will be used to
define a user's profile as were used to define the stakeholders'
profiles, above.

+---------+------------+------------+--------------------------------+
| User    | Repre      | Profile    |                                |
|         | sentatives |            |                                |
+=========+============+============+================================+
| SRA     | IAR Focus  | D          | Scientific Review              |
|         | Group      | escription | Administrator                  |
+---------+------------+------------+--------------------------------+
|         |            | Type       | Possesses strong communication |
|         |            |            | skills, along with good domain |
|         |            |            | expertise.                     |
+---------+------------+------------+--------------------------------+
|         |            | Respon     | 1.  Assigns Reviewers          |
|         |            | sibilities |                                |
|         |            |            | 2.  Identifies any COI         |
|         |            |            |                                |
|         |            |            | 3.  Determine deadlines for    |
|         |            |            |     critique submission,       |
|         |            |            |     read-only phase,           |
|         |            |            |     post-meeting edit phase.   |
|         |            |            |                                |
|         |            |            | 4.  Reviews critiques and      |
|         |            |            |     scores                     |
|         |            |            |                                |
|         |            |            | 5.  May submit critiques for   |
|         |            |            |     Reviewers                  |
|         |            |            |                                |
|         |            |            | 6.  Attends review meeting     |
|         |            |            |                                |
|         |            |            | 7.  Writes summary statements  |
+---------+------------+------------+--------------------------------+
|         |            | Success    | Project meets the              |
|         |            | Criteria   | specifications that have been  |
|         |            |            | agreed upon with the NIH       |
|         |            |            | community.                     |
+---------+------------+------------+--------------------------------+
|         |            | I          | Product reviewer, beta tester. |
|         |            | nvolvement | Attends and participates in    |
|         |            |            | meetings. Validates business   |
|         |            |            | rules.                         |
+---------+------------+------------+--------------------------------+
|         |            | De         | Any documents necessary to     |
|         |            | liverables | collect requirements. Examples |
|         |            |            | of reports or sample screen    |
|         |            |            | shots as requested by Analyst  |
|         |            |            | and Development team members.  |
+---------+------------+------------+--------------------------------+
|         |            | Comments/  |                                |
|         |            | Issues     |                                |
+---------+------------+------------+--------------------------------+
| GTA     | IAR Focus  | D          | Grants Technical Assistant     |
|         | Group      | escription |                                |
+---------+------------+------------+--------------------------------+
|         |            | Type       | Possesses strong communication |
|         |            |            | skills, along with good domain |
|         |            |            | expertise.                     |
+---------+------------+------------+--------------------------------+
|         |            | Respon     | 1.  Assists SRA                |
|         |            | sibilities |                                |
|         |            |            | 2.  May perform some tasks     |
|         |            |            |     listed above in SRA        |
|         |            |            |     Responsibilities           |
+---------+------------+------------+--------------------------------+
|         |            | Success    | Project meets the              |
|         |            | Criteria   | specifications that have been  |
|         |            |            | agreed upon with the NIH       |
|         |            |            | community.                     |
+---------+------------+------------+--------------------------------+
|         |            | I          | Product reviewer, beta tester. |
|         |            | nvolvement | Attends and participates in    |
|         |            |            | meetings. Validates business   |
|         |            |            | rules.                         |
+---------+------------+------------+--------------------------------+
|         |            | De         | Any documents necessary to     |
|         |            | liverables | collect requirements. Examples |
|         |            |            | of reports or sample screen    |
|         |            |            | shots as requested by Analyst  |
|         |            |            | and Development team members.  |
+---------+------------+------------+--------------------------------+
|         |            | Comments/  |                                |
|         |            | Issues     |                                |
+---------+------------+------------+--------------------------------+
| R       | IAR Focus  | D          | Reviewer (Non-NIH)             |
| eviewer | Group      | escription |                                |
+---------+------------+------------+--------------------------------+
|         |            | Type       | For the purpose of gathering   |
|         |            |            | requirements, this user will   |
|         |            |            | be represented by the SRAs and |
|         |            |            | GTAs.                          |
+---------+------------+------------+--------------------------------+
|         |            | Respon     | 1.  Reviews grant applications |
|         |            | sibilities |     for scientific merit.      |
|         |            |            |                                |
|         |            |            | 2.  Writes and submits         |
|         |            |            |     critiques and scores for   |
|         |            |            |     assigned applications      |
|         |            |            |                                |
|         |            |            | 3.  Reviews critiques          |
|         |            |            |     submitted by others in     |
|         |            |            |     preparation for meeting.   |
|         |            |            |                                |
|         |            |            | 4.  Attends and participates   |
|         |            |            |     in review meeting.         |
+---------+------------+------------+--------------------------------+
|         |            | Success    | Project meets the              |
|         |            | Criteria   | specifications that have been  |
|         |            |            | agreed upon with the NIH       |
|         |            |            | community.                     |
+---------+------------+------------+--------------------------------+
|         |            | I          | Beta tester.                   |
|         |            | nvolvement |                                |
+---------+------------+------------+--------------------------------+
|         |            | De         | None.                          |
|         |            | liverables |                                |
+---------+------------+------------+--------------------------------+
|         |            | Comments/  |                                |
|         |            | Issues     |                                |
+---------+------------+------------+--------------------------------+

# Product Overview

This section provides a high level view of the system capabilities,
interfaces, etc.

## Product Perspective

The system will be Web-based and developed using J2EE technology. The
basic logic and functionality of the NIAID ER system will be used as an
example to build this system. Other current NIH business practices for
electronic critique submission may also be reviewed to aid in collecting
requirements. Processes already integrated into the Peer Review module
such as Assigning Reviewers and defining COI will not be duplicated in
IAR. Since Reviewers are key system users, IMPAC II data items relevant
to IAR will be replicated to the Commons database to ensure data
security and prevent unauthorized access to the IMPAC II database. IAR
will be an external interface to the Commons database. The following
diagram shows how key processes of Internet Assisted Review fit into the
existing Peer Review business process

.

![](media/image1.png){width="7.95in" height="5.447916666666667in"}

## Summary of Capabilities

The system will comprise several primary functions: user registration
and management; critique/priority score submission and modification;
streamline voting; reports; and, critique combination/merging to create
summary statement draft. Within these functions, the following
capabilities will be included:

-   The IMPAC II Peer Review module will allow the SRA/GTA to finalize
    assignments to be created and made accessible in the Internet
    Assisted Review (IAR) system via the Commons database.

-   The system will allow the SRA/GTA to provide deadlines for critique
    submission and other events.

-   The system will manage accounts, passwords and registration for
    Reviewers.

-   The system will accept critiques of a Word (\*.doc) or WordPerfect
    (\*.wpd) format (version TBD).

-   The system will allow the SRA/GTA to modify the critique submission
    deadline and block certain reviewers from accessing the critiques of
    others.

-   The system should allow reviewers to post streamlining votes and
    allow for the production of related reports.

-   The system should provide reports that show the status of reviewer's
    critique submissions.

-   The system will facilitate summary statement creation by merging the
    critiques into one document to be used as the draft summary
    statement body text and allowing the SRA/GTA to export this
    document.

## Assumptions and Dependencies

The system will have dependencies with the IMPAC II Peer Review module
and will use data entered into this module. It will be mandatory for
Scientific Review Administrators (SRA/GTAs) to assign reviewers to
applications, check conflict of interest, and organize the meeting,
etc., in IMPAC II Peer Review to collect critiques in this system. When
the SRA/GTA has "finalized" their assignments, the Peer Review module
should error check for simultaneous conflicts and assignments.

## Cost and Pricing

IAR development efforts will meet the cost guidelines established by the
NIH eRA oversight boards and committees. The following preliminary
budget for Version 1 of IAR was submitted with the FY2002 Business Plan
and may be baselined after the Critical Design Review.

Estimated Cost: Non-Government Staff

<table>
<colgroup>
<col style="width: 14%" />
<col style="width: 6%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 6%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 10%" />
</colgroup>
<tbody>
<tr class="odd">
<td><p>Remaining</p>
<p>Work</p>
<p>To be done/ hrs by role &amp;</p>
<p>SDLC</p></td>
<td><blockquote>
<p>req.anal/ JADs</p>
</blockquote></td>
<td><blockquote>
<p>prelim. design &amp; prototype</p>
</blockquote></td>
<td><blockquote>
<p>Detailed design &amp; CDR</p>
</blockquote></td>
<td><blockquote>
<p>Hrs. build</p>
</blockquote></td>
<td><blockquote>
<p>Hrs. test</p>
</blockquote></td>
<td><blockquote>
<p>pilot</p>
</blockquote></td>
<td><blockquote>
<p>doc/ train.</p>
</blockquote></td>
<td><blockquote>
<p>Maint.. and enhancements</p>
</blockquote></td>
<td><blockquote>
<p>Tot hrs.</p>
</blockquote></td>
<td><blockquote>
<p>$/hr</p>
</blockquote></td>
<td><blockquote>
<p>Tot$</p>
</blockquote></td>
</tr>
<tr class="even">
<td>Total hrs non gov’t</td>
<td>300</td>
<td>550</td>
<td>150</td>
<td>1200</td>
<td>160</td>
<td>240</td>
<td>200</td>
<td>420</td>
<td>3220</td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>--Designer</td>
<td>300</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>300</td>
<td>$95</td>
<td>$28,500</td>
</tr>
<tr class="even">
<td>--Pgmr.</td>
<td></td>
<td>550</td>
<td>150</td>
<td>1200</td>
<td>80</td>
<td>160</td>
<td></td>
<td>300</td>
<td>2440</td>
<td>$83</td>
<td>$202,520</td>
</tr>
<tr class="odd">
<td>--Test/QA</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>80</td>
<td>80</td>
<td></td>
<td>80</td>
<td>240</td>
<td>$83</td>
<td>$19,920</td>
</tr>
<tr class="even">
<td>--Writer</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>200</td>
<td>40</td>
<td>240</td>
<td>$41</td>
<td>$9,840</td>
</tr>
<tr class="odd">
<td colspan="11"><p>Hrs. Hrs. Hrs. Hrs. Hrs. Hrs. Hrs. Hrs.</p>
<p>Total $ required to complete</p></td>
<td>$260,780</td>
</tr>
</tbody>
</table>

**Government Staff**

Tracy Soto (OD) 30% of total time

Sherry Zucker, (OD) 10% of total time

Scarlett Gibb (OD) 10% of total time

## Licensing and Installation

There are no licensing requirements for IAR. Since the IAR product is
Web-based, installation of the product is not necessary. Only a Web
browser will be required for users.

# Product Features

This section defines and describes the proposed features of this
application. Where applicable, information regarding Version and
Priority are provided.

Key to "Version" in tables:

-   Ver.1: Must be in the initial version

-   Ver.2: Must be in the second version

-   Ver.3: Can be added at any point

Proposed features are prioritized using MuSCoW: "Must", "Should",
"Could", or "Won't".

-   Must---Version can't be deployed without function; requirement is
    basic functionality.

-   Should---Requirement should be deployed in version. Version can be
    deployed before function is implemented.

-   Could---'Nice to have' enhancement.

-   Won't---Requirement should not be included in functionality.

## System (SYS) Feature

### Error Handling

The system shall graciously handle and log all errors encountered.

### Usage Reporting

The system shall provide reports on general system use and exceptional
behavior.

### System Control 

The system shall provide convenient mechanisms for startup, shutdown,
and recovery of individual subsystems. 

### Browser Interface

The system shall provide a user interface through a thin, browser-based
client. 

### Interface Conventions

The user interface shall follow standard interface conventions based on
acceptable industry standards. 

### On-line help

The user interface shall include on-line help features.

### Release Notes

The user interface shall include links to the release notes.

### Bug Status

The user interface shall include links to the status of reported bugs.

### Availability

The system shall be generally available for use on a 24x7 basis with
limited downtime acceptable for system upgrades and unexpected
conditions.

### Performance

The system shall provide performance and response times generally
consistent with industry standards for Internet applications.

### Auditing

The system shall provide configurable auditing capabilities.

### Exception Reporting

The system shall report exceptional conditions to an administrator via
e-mail.

### External Interfaces

The system shall provide an interface to the Review community.

**Action Item/Research Needed:**

1.  Tracy: Discuss with Dan Hall, Sara Silver: ability to store and
    modify secondary email address for a reviewer, copy corrected email
    to profile, mark profile for cleanup

2.  Daniel: ability for Reviewer to see times for their time zone, virus
    check on critique when uploaded

3.  Scarlett: survey Review users for list of valid preliminary scores
    (alpha codes), check to see if it would be allowable to list a
    secondary SRA on the List of Meetings.

4.  ~~Brian: revise sample text for special consideration reminder on
    critique upload screen per discussion.~~

5.  Focus Group Members: provide list of document types, documents you
    wish to include in IAR and specify whether they are already publicly
    available on the Internet.

6.  Security Advisors: ~~provide security recommendations, provide
    sample text for User Agreement,~~ supply Daniel Fox with the
    security requirements for importing files.

7.  ~~Neal -- draft sample letter to Reviewers (to be faxed or mailed)
    to provide them with their person_id for registration.~~

8.  ~~Roy -- draft sample email to Reviewers to provide them with
    registration URL, instructions, SRA contact information and alert
    them to locate other correspondence for person_id. Subject line
    could be "NIH IAR Registration".~~

\*\* Items that have Version and MuSCoW have been discussed and
prioritized by the IAR Focus Group. Items missing this information will
be discussed at future Focus Group meetings.

## Release Meeting to IAR / IAR Control Center

Before Reviewers can access a meeting in IAR, the SRA/GTA must enable
the meeting to be used by IAR. There are several functions that the
SRA/GTA will need to perform. Below are the requirements for the "IAR
Control Center" which will be available in Peer Review module.

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | The Peer Review module should allow SRA/GTA to   | 1    | M      |
| . | navigate from the Peer Review banner screen to   |      |        |
|   | the "IAR Control Center" module with default     |      |        |
|   | meeting as selected on the banner screen.        |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Control Center must have an ability to   | 1    | M      |
| . | easily enable the entire list of reviewers for   |      |        |
|   | the meeting for IAR.                             |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The IAR Control Center should have the ability   | 2    | S      |
| . | to easily enable selected Reviewers for IAR.     |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | To Enable the Reviewers for IAR, Reviewers in    | 1    | M      |
| . | the meeting must be on the Committee Management  |      |        |
|   | Meeting Roster. SRA/GTA will be alerted to any   |      |        |
|   | discrepancies (if Reviewer isn't on Roster).     |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The IAR Control Center must prevent selection of | 1    | M      |
| . | Reviewers for IAR if it detects the absence of   |      |        |
|   | the MLG role e-mail address and must alert       |      |        |
|   | SRA/GTA.                                         |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | Provide a "Roster Reconciliation" screen to      | 2    | C      |
| . | allow the SRA/GTA to link people from the CM     |      |        |
|   | roster to people listed on the Reviewer List. No |      |        |
|   | person search would be required, and the process |      |        |
|   | would be straightforward and quick for the user. |      |        |
|   | IAR could then \"transfer\" the assignments to   |      |        |
|   | the CM identified reviewers.                     |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | The IAR Control Center must allow SRA/GTA to     | 1    | M      |
| . | view and edit the MLG email address (on role)    |      |        |
|   | for each reviewer.                               |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 8 | When SRA/GTA edits the MLG role email address,   | 1    | M      |
| . | the IAR Control Center must give SRA/GTA the     |      | (cont  |
|   | option of copying that email address to the      |      | ingent |
|   | profile record.                                  |      | on     |
|   |                                                  |      | action |
|   |                                                  |      | item   |
|   |                                                  |      | #1)    |
+---+--------------------------------------------------+------+--------+
| 9 | To Enable at least one reviewer for IAR, the     | 1    | M      |
| . | Critique Submission Due Date and Time (ET) must  |      |        |
|   | be entered.                                      |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | To Enable at least one reviewer for IAR, the     | 1    | M      |
| 0 | Read Phase End Date and Time (ET) must be        |      |        |
| . | entered.                                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | To Enable at least one reviewer for IAR, the     | 1    | M      |
| 1 | Start Date and Time (Eastern Time) for the Post  |      |        |
| . | Mtg. Edit Phase must be an optional field (If    |      |        |
|   | the date is not entered, the meeting will stay   |      |        |
|   | in the Read Phase until the Read phase end date; |      |        |
|   | then Reviewers will not be able to do anything   |      |        |
|   | in IAR).                                         |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | To Enable the meeting for IAR, if the Start Date | 1    | M      |
| 2 | and Time (ET) for the Post Mtg. Edit Phase has   |      |        |
| . | been entered, the End date for the Post Mtg.     |      |        |
|   | Edit Phase must be entered.                      |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | To Enable the meeting for IAR, the Closure date  | 1    | M      |
| 3 | for the Meeting must be entered.                 |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Control Center must verify that dates    | 1    | M      |
| 4 | and time entered for the phases are sequential   |      |        |
| . | (1. Critique Submission Due Date, 2. Read Phase  |      |        |
|   | End Date, 3. Edit Phase Start and End Date, 4.   |      |        |
|   | Closure Date)                                    |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Control Center must allow SRA/GTA to     | 1    | M      |
| 5 | disable (block) [individual]{.underline}         |      |        |
| . | Reviewers from reading critiques on applications |      |        |
|   | where they are assigned but did not post their   |      |        |
|   | own critique.                                    |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Control Center should allow SRA/GTA to   | 1    | S      |
| 6 | easily disable (block) [all]{.underline}         |      |        |
| . | Reviewers from reading critiques on applications |      |        |
|   | where they are assigned but did not post their   |      |        |
|   | own critique.                                    |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Control Center should allow SRA/GTA to   | 1    | M      |
| 7 | toggle show/hide preliminary scores from all     |      |        |
| . | Reviewers in IAR. If Scores are hidden, Reviewer |      |        |
|   | would only see scores they've entered.           |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | **Revisit: The IAR Control Center should allow   | *    | **S**  |
| 8 | SRA/GTA to toggle the ability to show only       | *2** |        |
| . | average scores or raw scores but show the entire |      |        |
|   | matrix.**                                        |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Control Center should allow SRA/GTA to   | 1    | S      |
| 9 | toggle ability for reviewers to enter critiques  |      |        |
| . | or comments for applications where they are not  |      |        |
|   | assigned.                                        |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | **Revisit: is this necessary given proposed      | *    | **M**  |
| 0 | screens? The IAR Control Center must allow       | *1** |        |
| . | SRA/GTA to run a printable report, which will    |      |        |
|   | list all reviewers in the meeting, their MLG     |      |        |
|   | role email address, username, answer and PIN     |      |        |
|   | (person_id). This report could provide SRA/GTA   |      |        |
|   | information on whether or not a Reviewer has     |      |        |
|   | registered to use IAR.**                         |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Control Center must allow SRA/GTA to     | 1    | M      |
| 1 | extend read permissions on critiques to          |      |        |
| . | Telephone Reviewers for entire meeting instead   |      |        |
|   | of the default view of only their assigned       |      |        |
|   | applications.                                    |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | IAR Control Center must allow SRA/GTA to send    | 2    | S      |
| 2 | custom batch emails to ALL reviewers in the      |      |        |
| . | meeting.                                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | IAR Control Center must allow SRA/GTA to specify | 2    | S      |
| 3 | the "From" addressee in the custom batch email   |      |        |
| . | to all Reviewers.                                |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | IAR Control Center must send the Carbon Copy     | 2    | S      |
| 4 | emails to SRA/GTA when batch emails are sent to  |      |        |
| . | all Reviewers.                                   |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | IAR Control Center must return undeliverable     | 2    | S      |
| 5 | emails to the SRA/GTA when batch emails are sent |      |        |
| . | to Reviewers.                                    |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Control Center must allow SRA/GTA to     | 1    | M      |
| 6 | trigger an email with a customized registration  |      |        |
| . | URL (embedded with unique reviewer               |      |        |
|   | identifier/system generated random number) to    |      |        |
|   | individual reviewers.                            |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Control Center should include the        | 1    | S      |
| 7 | ability for the SRA/GTA to identify hyperlinks   |      |        |
| . | to documents for display within their meeting in |      |        |
|   | IAR: Reviewer Guidelines (for specific           |      |        |
|   | mechanisms, human subjects, etc.), grant         |      |        |
|   | application, prior summary statements, program   |      |        |
|   | announcements, meeting roster, blank COI form,   |      |        |
|   | master list of applications/order of review      |      |        |
|   | report, travel instructions (hotel, airline,     |      |        |
|   | etc.), meeting agenda. Specific document types   |      |        |
|   | to be determined.                                |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | For documents not available on the web, the IAR  | 2    | S      |
| 8 | Control Center should allow the SRA/GTA to       |      |        |
| . | upload documents for display within IAR: cover   |      |        |
|   | letter (specific to each SRA), travel            |      |        |
|   | instructions (hotel, airline, etc). Specific     |      |        |
|   | document types to be determined.                 |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Control Center should allow SRA/GTA to   | 1    | S      |
| 9 | choose which document or document hyperlinks to  |      |        |
| . | include in their meeting.                        |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | Changes in assignments, COI, Roster in Peer      | 1    | M      |
| 0 | Review must be immediately available in IAR.     |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | When changes occur (assignments, COI,            | 2    | S      |
| 1 | application added or withdrawn), affected        |      |        |
| . | Reviewers should receive email notification of   |      |        |
|   | the changes. This requirement may be met by      |      |        |
|   | another upcoming eRA system -eNotification.      |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The IAR Control Center must display a Reviewer's | 1    | M      |
| 2 | IAR username and answer to their personal        |      |        |
| . | question to the SRA/GTA.                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The IAR Control Center must provide SRA/GTA with | 1    | M      |
| 3 | a printable report/letters to facilitate user    |      |        |
| . | registration. The PIN (person_id) and other      |      |        |
|   | relevant registration information will be on the |      |        |
|   | letter. Group will provide language for letter.  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The IAR Control Center must have an ability to   | 1    | M      |
| 4 | Disable the meeting for IAR (this does not purge |      |        |
| . | the meeting).                                    |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+

## Pre-Registration System Process

Once the meeting is released to IAR via "IAR Control Center," the
process of inviting Reviewers to access IAR and enabling the system to
allow Reviewer accounts to be created for IAR is activated. Reviewers
with Commons account will use those accounts. Reviewers who do not have
Commons accounts will be sent an email containing an URL with a unique
identifier. The requirement for this process described below:

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | If Reviewer has a Commons account, the meeting   | 1    | M      |
| . | release to IAR will enable the Commons account   |      |        |
|   | to be used in IAR.                               |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | If Reviewer has a Commons account, system will   | 1    | M      |
| . | email the reviewer an invitation to the IAR      |      |        |
|   | website, requesting him/her to use existing      |      |        |
|   | Commons Account log on information to enter IAR  |      |        |
|   | (exact text to be determined).                   |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | If Reviewer does not have a Commons account,     | 1    | M      |
| . | email the reviewer invitation to the IAR         |      |        |
|   | website, and include detailed instructions on    |      |        |
|   | how to register (exact text to be determined) to |      |        |
|   | obtain an IAR account. This email will not       |      |        |
|   | include a PIN. Provide a hyperlink containing an |      |        |
|   | embedded token identifier for the reviewer.      |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | If Reviewer does not have a Commons account,     | 1    | M      |
| . | reviewer profile in\                             |      |        |
|   | IMPAC II will be marked for cleanup.             |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | Undeliverable registration invitation emails     | 1    | M      |
| . | should be sent to SRA/GTA. (GTA must be on       |      |        |
|   | roster to get the email.)                        |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+

## User Registration

Once the registration invitation email is sent, Reviewers that do not
have Commons accounts are asked to register. To access the registration
form, Reviewers will click on the customized URL in the email. Reviewers
will also need a PIN to register. The SRA/GTA will provide the PIN to
the Reviewer via phone, fax, postal mail or Federal Express. Below are
requirements for the registration form.

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | The IAR Registration module must be able to      | 1    | M      |
| . | accept an identifier (person_id) from the        |      |        |
|   | Reviewer to register.                            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | If the identifier is not associated with the     | 1    | M      |
| . | Reviewer, the IAR Registration module will not   |      |        |
|   | allow Reviewer to create account and a message   |      |        |
|   | to try again or contact SRA/GTA will be          |      |        |
|   | displayed.                                       |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The Reviewer must have 3 attempts to enter a     | 1    | M      |
| . | valid identifier.                                |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | If the Reviewer fails to enter a valid PIN 3     | 1    | M      |
| . | times, the system will expire the original       |      |        |
|   | registration invitation URL. The Reviewer will   |      |        |
|   | be instructed to contact their SRA or the eRA    |      |        |
|   | Helpdesk for their PIN and a new registration    |      |        |
|   | invitation url.                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | If Reviewer's identifier was accepted by the     | 1    | M      |
| . | system, the IAR Registration module will display |      |        |
|   | Person Name and Institution.                     |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The IAR Registration module should allow         | 1    | M      |
| . | Reviewer to indicate that the name displayed is  |      |        |
|   | not their name and should alert the Reviewer to  |      |        |
|   | contact their SRA/GTA or eRA Helpdesk.           |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | If Reviewer indicates displayed name is not      | 1    | M      |
| . | their name, system should inactivate Reviewer's  |      |        |
|   | account.                                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 8 | The IAR Registration module must accept a User   | 1    | M      |
| . | Name of Reviewer's choice (must adhere to        |      |        |
|   | reasonable security rules).                      |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 9 | The IAR Registration module must accept a        | 1    | M      |
| . | password of Reviewer's choice (must adhere to    |      |        |
|   | reasonable security rules)                       |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Registration module must allow Reviewer  | 1    | M      |
| 0 | to retype the password (since the password field |      |        |
| . | is masked with (\*) asterisks.                   |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Registration module must allow Reviewer  | 1    | M      |
| 1 | to choose a question and enter a corresponding   |      |        |
| . | answer to be used in the future if they forget   |      |        |
|   | their password. Suggested questions:             |      |        |
|   |                                                  |      |        |
|   | -   What is your city of birth?                  |      |        |
|   |                                                  |      |        |
|   | -   What is your mother's maiden name?           |      |        |
|   |                                                  |      |        |
|   | -   What is your pet's name?                     |      |        |
|   |                                                  |      |        |
|   | -   What is your spouse's name?                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The system should allow Reviewer to enter        |      | Cont   |
| 2 | additional email addresses where they may be     |      | ingent |
| . | reached.                                         |      | on     |
|   |                                                  |      | Action |
|   |                                                  |      | item   |
|   |                                                  |      | #1     |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Registration module must notify the      | 1    | M      |
| 3 | Reviewer that registration has completed         |      |        |
| . | successfully and provide the link to the IAR     |      |        |
|   | Banner screen (where they can click to Log On).  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | All system generated email communication with    | 1    | S      |
| 4 | the Reviewer should include the IAR Banner       |      |        |
| . | screen URL.                                      |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | After the Reviewer enters a correct PIN and      | 1    | M      |
| 5 | successfully registers (creates user name,       |      |        |
| . | password and answers question) the original      |      |        |
|   | registration invitation URL will expire.         |      |        |
+---+--------------------------------------------------+------+--------+

## Log on to IAR

Once users have a valid account, they can log into the IAR via the Log
on screen. Below are the requirements for a log on screen.

+---+--------------------------------------------------+------+--------+
|   |                                                  | Ver  | MuSCoW |
|   |                                                  | sion |        |
+---+--------------------------------------------------+------+--------+
| 1 | The IAR Log on module must accept Reviewer user  | 1    | M      |
| . | name and password to log in.                     |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Log on module should accept existing     | 1    | M      |
| . | IMPACII user name and password for SRA/GTA       |      |        |
|   | users.                                           |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The IAR Log on module must display an error      | 1    | M      |
| . | message if log in fails.                         |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The IAR Log on module must provide a method for  | 1    | M      |
| . | Reviewers to reset their forgotten password.     |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | When user enters correct User Name and Password, | 1    | M      |
| . | User must be logged in to IAR.                   |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+

## Password Replace

*If a Reviewer doesn't know their User Name they must contact the
SRA/GTA*. If Reviewer has forgotten their password, they can create a
new one via the Password Replace Form. This feature will be accessible
from the Log on screen.

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | The IAR Password Reset module must allow         | 1    | M      |
| . | Reviewer to enter their User name.               |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The IAR Password Reset module must validate user | 1    | M      |
| . | name.                                            |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | If user name is not valid, the IAR Password      | 1    | M      |
| . | Reset module must display a message indicating   |      |        |
|   | invalid username and prompt Reviewer to try      |      |        |
|   | again or contact SRA/GTA.                        |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The IAR Password Reset module must allow         | 1    | M      |
| . | Reviewer 3 attempts to provide a valid user      |      |        |
|   | name.                                            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | If user name is valid, the IAR Password Reset    | 1    | M      |
| . | module must display the question entered by the  |      |        |
|   | Reviewer at time of account creation.            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The IAR Password Reset module must allow         | 1    | M      |
| . | Reviewer to enter the answer to the question.    |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | The IAR Password Reset module must validate the  | 1    | M      |
| . | answer to the question.                          |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 8 | If answer is not valid, the IAR Password module  | 1    | M      |
| . | must display a message indicating invalid answer |      |        |
|   | and prompt Reviewer to try again or contact      |      |        |
|   | SRA/GTA.                                         |      |        |
+---+--------------------------------------------------+------+--------+
| 9 | The IAR Password reset module must allow         | 1    | M      |
| . | Reviewer 3 attempts to provide a valid answer.   |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If all 3 attempts to provide valid username      | 1    | M      |
| 0 | fail, Reviewer will be alerted to contact their  |      |        |
| . | SRA/GTA or eRA Helpdesk.                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If all 3 attempts to provide valid answer fail,  | 1    | M      |
| 1 | Reviewer will be alerted to contact their        |      |        |
| . | SRA/GTA or eRA Helpdesk.                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If answer is valid, system must require Reviewer | 1    | M      |
| 2 | to create a new password.                        |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The module must accept a password of Reviewer's  | 1    | M      |
| 3 | choice (must adhere to reasonable security       |      |        |
| . | rules)                                           |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The module must allow Reviewer to retype the     | 1    | M      |
| 4 | password (since the password field is masked     |      |        |
| . | with (\*) asterisks.                             |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | After password is successfully reset, system     | 1    | M      |
| 5 | will present Reviewer with IAR log on screen.    |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+

## Account Access and Expiration

Reviewers who have a registered Commons account will have different
access and account expiration than Reviewers with an IAR-only account.
These differences are described below.

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | Once in the IAR system, the Common Account       | 1    | M      |
| . | holders will be able to access all active        |      |        |
|   | meetings that they participate in as Reviewers.  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | Once in the IAR system, the IAR-only account     | 1    | M      |
| . | holders will have access to the active meeting   |      |        |
|   | only.                                            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | SRA/GTA must have access to all their active     | 1    | M      |
| . | (prior to purge) IAR meetings. Events to trigger |      |        |
|   | purge will be discussed later in this document.  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The list of Meetings shall display the meeting   | 1    | M      |
| . | identifiers, SRA of the meeting, meeting start   |      |        |
|   | date, meeting phase (SUBMIT, READ, EDIT).        |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The accounts for the Reviewers with IAR-only     | 1    | M      |
| . | accounts will expire after the Meeting Closure   |      |        |
|   | Date.                                            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The Commons Account holders will lose their      | 1    | M      |
| . | access to the IAR system after 1 year of IAR     |      |        |
|   | inactivity.                                      |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+

## Changing Passwords

Changing of passwords is a standard eRA website utility available to all
eRA web development initiatives. This system will adhere to the
standards set forth by Commons 2.

## Submit Phase

The Submit Phase is the first phase of the IAR workflow. During the
Submit phase, Reviewers post and edit their critiques. They will only
have access to their assigned applications and can only view critiques
they post. They cannot view critiques posted by other Reviewers. All
functions needed in the Submit phase will be accessible from the
Reviewer list of applications screen. From Reviewer List of Applications
screen, Reviewers may:

a)  See a list of Meetings he/she is assigned to (if they have a Commons
    account).

b)  See all applications assigned to them.

c)  Submit the Critique and/or Score.

d)  Re-Submit Critique and/or Score.

e)  See critiques and scores they have submitted.

SRA/GTAs may:

a)  See all applications in the meeting.

b)  See critiques and scores for the applications one at a time.

c)  See critiques and scores batched by application into one file.

d)  See critiques and scores batched by reviewer into one file.

e)  See ALL critiques and scores for the meeting batched into one file.

f)  Submit the Critique and/or score for the Reviewer.

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
|   | REVIEWER'S VIEW                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If Reviewer has a Commons account, after         | 1    | M      |
| . | successful login the system will display a List  |      |        |
|   | of Meetings.                                     |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | If a user has an IAR only account, after         | 1    | M      |
| . | successful login the system will display the     |      |        |
|   | reviewer's assigned List of Applications.        |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The list of Meeting screen should display        | 1    | M      |
| . | meeting identifier, meeting title, meeting       |      |        |
|   | dates, phase (Submit, Read, Edit), SRA name and  |      |        |
|   | contact information, Critique due date and time, |      |        |
|   | edit phase start and end date and time.          |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The system must allow Reviewer to easily         | 1    | M      |
| . | navigate from List of Meetings to their assigned |      |        |
|   | List of Applications for a specified meeting.    |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | If meeting is telephone conference, system       | 1    | S      |
| . | should display "Teleconference" with meeting     |      |        |
|   | title.                                           |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The List of Applications screen must allow       | 1    | M      |
| . | Reviewer to easily navigate to screens for       |      |        |
|   | Submitting and viewing their critiques and       |      |        |
|   | scores.                                          |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | The List of Applications screen for Reviewers    | 1    | M      |
| . | should provide a way to easily identify New      |      |        |
|   | Investigators.                                   |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 8 | The List of Applications screen for the Reviewer | 1    | M      |
| . | should display the Reviewer's name (Last, First  |      |        |
|   | and M.I.), meeting identifier, meeting title,    |      |        |
|   | meeting phase, meeting dates, critique due date  |      |        |
|   | and time and a list of their assigned            |      |        |
|   | applications.                                    |      |        |
+---+--------------------------------------------------+------+--------+
| 9 | The List of Applications screen for the Reviewer | 1    | M      |
| . | should display a message instructing them to     |      |        |
|   | contact their SRA/GTA if they identify conflicts |      |        |
|   | or assignment discrepancies in IAR.              |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The List of Applications screen for the Reviewer | 1    | S      |
| 0 | should provide easy access to SRA contact        |      |        |
| . | information (name, email and phone from WRK      |      |        |
|   | addr)                                            |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The List of Applications screen for the Reviewer | 1    | M      |
| 1 | must display the following for each application: |      |        |
| . | application number, PI name, new PI indicator,   |      |        |
|   | project title, assignment role, score and        |      |        |
|   | critique submitted date and time.                |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If a critique has been submitted for an          | 1    | M      |
| 2 | application, the List of Applications should     |      |        |
| . | display these action items: View, Submit,        |      |        |
|   | Delete.                                          |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If a critique has not been submitted for an      | 1    | M      |
| 3 | application, the List of Applications should     |      |        |
| . | display the Submit action item.                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The List of Applications screen for Reviewer's   | 1    | M      |
| 4 | should have a default sort order of PI name with |      |        |
| . | a secondary sort on Activity Code/IC/Serial      |      |        |
|   | Number.                                          |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The List of Applications screen for Reviewer's   | 1    | S      |
| 5 | should allow reviewer to sort their list of      |      |        |
| . | applications by these column headings:           |      |        |
|   | application number (activity code/IC/serial), PI |      |        |
|   | name (secondary sort on application number),     |      |        |
|   | assignment role (secondary sort on PI name),     |      |        |
|   | score, and critique submitted date (show blanks  |      |        |
|   | on top and do secondary sort on application      |      |        |
|   | number).                                         |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The default List of Applications screen for the  | 1    | S      |
| 6 | Reviewer should show only applications assigned  |      |        |
| . | to the Reviewer but provide access to show "All  |      |        |
|   | Applications," if the SRA has opened the meeting |      |        |
|   | for unassigned critiques or comments to be       |      |        |
|   | posted                                           |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The All Applications list should show all        | 1    | M      |
| 7 | applications for the meeting, including those    |      |        |
| . | with conflicts.                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | On the All Applications list, if a Reviewer is   | 1    | M      |
| 8 | in conflict with an application, the assignment  |      |        |
| . | role should show "COI" and the Reviewer should   |      |        |
|   | not be able to perform any action (Submit, View, |      |        |
|   | Delete).                                         |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | At the time of posting of each critique, the     | 1    | W      |
| 9 | system should remind the reviewer whether there  |      |        |
| . | are any special considerations involved. These   |      |        |
|   | special considerations should be flagged:        |      |        |
|   |                                                  |      |        |
|   | a\. The involvement of human subjects can be     |      |        |
|   | flagged (this could be tied to a optional text   |      |        |
|   | entry screen where the text can be auto-appended |      |        |
|   | to the reviewer's posted critique or the         |      |        |
|   | reviewer may wish to complete their critique and |      |        |
|   | post it later)                                   |      |        |
|   |                                                  |      |        |
|   | b\. Vertebrate animal subjects involved.         |      |        |
|   |                                                  |      |        |
|   | c\. New R01 investigator reminder with a simple  |      |        |
|   | flag.                                            |      |        |
|   |                                                  |      |        |
|   | d\. Reminder with a simple flag that an extra    |      |        |
|   | section is required when the application is      |      |        |
|   | submitted by a foreign organization. (Again, an  |      |        |
|   | optional text box might be tied to the flag.)    |      |        |
|   |                                                  |      |        |
|   | 3/25/02 Focus Group felt this was not necessary  |      |        |
|   | and instead added the next requirement.          |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen should display a      | 1    | S      |
| 0 | reminder about special considerations. 4/8/02    |      |        |
| . | Group agreed on this language:                   |      |        |
|   |                                                  |      |        |
|   | REMINDERS  \-- Human Subjects are part of the    |      |        |
|   | review criteria and need to be assessed by the   |      |        |
|   | assigned reviewers. This includes:               |      |        |
|   |                                                  |      |        |
|   |   protection of human subjects from research     |      |        |
|   | risks; data and safety monitoring; inclusion of  |      |        |
|   | women; inclusion of minorities; inclusion of     |      |        |
|   | children. NIH policy also requires the review    |      |        |
|   | panel to consider the following items:           |      |        |
|   |                                                  |      |        |
|   |   animal welfare;                                |      |        |
|   |                                                  |      |        |
|   | biohazards; budgetary overlap.                   |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen for Reviewers should  | 1    | M      |
| 1 | show the application number, project title, pi   |      |        |
| . | name, reviewer type and score (if already        |      |        |
|   | entered).                                        |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen must allow Reviewers  | 1    | M      |
| 2 | to browse their computer to find and select a    |      |        |
| . | file to upload to IAR.                           |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen for Reviewers should  | 1    | M      |
| 3 | accept critiques in Word or WordPerfect format.  |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | Critiques should be stored in their native       | 1    | M      |
| 4 | format (Word or WordPerfect).                    |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | Critiques should be displayed in Adobe PDF       | 1    | M      |
| 5 | format (to be word processor independent).       |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen must allow Reviewers  | 1    | M      |
| 6 | to submit numeric preliminary scores or choose   |      |        |
| . | either DF (deferred) or NR (not recommended).    |      |        |
|   | Only one is permitted.                           |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen should allow          | 2    | C      |
| 7 | Reviewers to submit alphanumeric preliminary     |      |        |
| . | scores.                                          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen should only accept a  | 1    | M      |
| 8 | score of one or two digits or two digits with    |      |        |
| . | decimal point (acceptable range is 1.0--5.0)     |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The Critique Upload screen should convert a      | 1    | M      |
| 9 | 2-digit score to 2-digit with decimal (example   |      |        |
| . | 20 would be converted to 2.0) and convert        |      |        |
|   | 1-digit to 2-digit with decimal (example 3 would |      |        |
|   | be converted to 3.0).                            |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | If a Reviewer submits an alphanumeric score, the | 2    | S      |
| 0 | Critique Upload screen should limit the entry to |      |        |
| . | 3 characters.                                    |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The Critique Upload Screen should verify that    | 2    | S      |
| 1 | the alphanumeric score submitted by the Reviewer |      |        |
| . | exists on the score list of values (acceptable   |      |        |
|   | values need to be determined by group).          |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The Critique Upload screen must allow Reviewers  | 1    | M      |
| 2 | to resubmit (edit) their critiques and scores.   |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | After successful upload of Critique and/or       | 1    | M      |
| 3 | Score, system must provide a confirmation        |      |        |
| . | screen.                                          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The system might allow the Reviewer to input of  |      | W      |
| 4 | a score mark as conditional with an explanation. |      |        |
| . | Often reviewers indicate their score is          |      |        |
|   | conditional on outcome of discussion. 3/25/02    |      |        |
|   | Group felt this was not necessary since scores   |      |        |
|   | are preliminary and therefore conditional.       |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The system must allow the Reviewer to view       | 1    | M      |
| 5 | critiques and preliminary scores they have       |      |        |
| . | posted for individual applications.              |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The system must allow the Reviewer to generate a | 1    | S      |
| 6 | printable report of critiques and preliminary    |      |        |
| . | scores they have posted for all applications.    |      |        |
|   | Critiques will be assembled by PI name.          |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The printable report of critiques and            | 1    | S      |
| 7 | preliminary scores posted by a Reviewer should   |      |        |
| . | be assembled by PI name.                         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The printable report of critiques and            | 1    | S      |
| 8 | preliminary scores posted by a Reviewer must     |      |        |
| . | have a page break between each critique.         |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | The printable report of critiques and            | 1    | S      |
| 9 | preliminary scores posted by a Reviewer must     |      |        |
| . | show the following: PI Name, Grant Number,       |      |        |
|   | Critique Submitted Date, Score (if entered),     |      |        |
|   | Assignment Role, Critique Text.                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The system must track date and time the critique | 1    | M      |
| 0 | was posted.                                      |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The system could allow Reviewers to view         | 3    |        |
| 1 | abstracts.                                       |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | If specified by the SRA/GTA within the IAR       | 1    | S      |
| 2 | Control Center, the system should allow          |      |        |
| . | Reviewers to enter critiques/comments for        |      |        |
|   | applications where they are not assigned.        |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The score entry for the unassigned application   | 1    | M      |
| 3 | is NOT allowed.                                  |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
|   | SRA/GTA'S VIEW                                   |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The List of Applications screen must provide     | 1    | M      |
| 4 | SRA/GTAs easy access to corresponding screens    |      |        |
| . | for posting and viewing critiques and scores.    |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The List of Applications screen should provide   | 2    | S      |
| 5 | SRA/GTAs with the ability to toggle the ability  |      |        |
| . | to show/hide Discussants, Mail Reviewers and     |      |        |
|   | Readers.                                         |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The List of Applications screen for SRA/GTAs     | 1    | M      |
| 6 | should provide a way to easily identify New      |      |        |
| . | Investigators (\* - New PI).                     |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The List of Applications screen for the SRA/GTA  | 1    | M      |
| 7 | should display the SRA's name, meeting           |      |        |
| . | identifier, meeting phase, critique due date and |      |        |
|   | all applications for the meeting.                |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The List of Applications screen for the SRA/GTA  | 1    | M      |
| 8 | must display the following for each application: |      |        |
| . | application number, pi name, project title,      |      |        |
|   | reviewer name, assignment role, score and        |      |        |
|   | critique submitted date.                         |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | The List of Applications screen for SRA/GTAs     | 1    | S      |
| 9 | should have a default sort order of PI name with |      |        |
| . | a secondary sort on Activity Code/IC/Serial      |      |        |
|   | Number.                                          |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The List of Applications screen should allow     | 1    | S      |
| 0 | SRA/GTA to sort their list of applications by    |      |        |
| . | these column headings: application number        |      |        |
|   | (Activity Code/IC/Serial Number), pi name,       |      |        |
|   | reviewer name, assignment role, score and        |      |        |
|   | critique submitted date.                         |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | On the list of Applications screen for SRA/GTAs  | 1    | S      |
| 1 | a sort on critique submitted date should show    |      |        |
| . | blank dates at the top of the list.              |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The Critique Upload screen for SRA/GTAs should   | 1    | M      |
| 2 | show the application number, project title, pi   |      |        |
| . | name, and score (if already entered).            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The Critique Upload screen for SRA/GTAs must     | 1    | M      |
| 3 | allow SRA/GTA to choose the Reviewer Name that   |      |        |
| . | should be associated with the critique they need |      |        |
|   | to upload.                                       |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The Critique Upload screen must allow SRA/GTAs   | 1    | M      |
| 4 | to browse their computer to find and select a    |      |        |
| . | file to upload to IAR.                           |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The Critique Upload screen for the SRA/GTAs      | 1    | M      |
| 5 | should accept critiques in Word or Word Perfect  |      |        |
| . | format.                                          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | Critiques should be stored in their native       | 1    | M      |
| 6 | format (Word or WordPerfect).                    |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | Critiques should be displayed in Adobe PDF       | 1    | M      |
| 7 | format (to be word processor independent).       |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The Critique Upload screen must allow SRA/GTAs   | 1    | M      |
| 8 | to submit numeric preliminary scores or choose   |      |        |
| . | either DF (deferred) or NR (not recommended).    |      |        |
|   | Only one is permitted.                           |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | The Critique Upload screen should allow          | 2    | C      |
| 9 | Reviewers to submit alphanumeric preliminary     |      |        |
| . | scores.                                          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The Critique Upload screen should only accept a  | 1    | S      |
| 0 | score of two digits or two digits with decimal   |      |        |
| . | point (acceptable range is 1.0--5.0)             |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The Critique Upload screen should convert a      | 1    | S      |
| 1 | 2-digit score to 2-digit with decimal (example   |      |        |
| . | 20 would be converted to 2.0) and convert        |      |        |
|   | 1-digit to 2-digit with decimal (example 3 would |      |        |
|   | be converted to 3.0).                            |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | If an SRA/GTA submits an alphanumeric score, the | 2    | C      |
| 2 | Critique Upload screen should limit the entry to |      |        |
| . | 3 characters.                                    |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The Critique Upload Screen should verify that    | 2    | C      |
| 3 | the alphanumeric score submitted by the SRA/GTA  |      |        |
| . | exists on the score list of values (acceptable   |      |        |
|   | values need to be determined by group).          |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The Critique Upload screen must allow the        | 1    | M      |
| 4 | SRA/GTAs to resubmit (edit) critiques and scores |      |        |
| . | for Reviewers.                                   |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The system must allow the SRA/GTA to view        | 1    | M      |
| 5 | critiques and preliminary scores.                |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The system must allow the SRA/GTA to generate a  | 1    | M      |
| 6 | printable report of all meeting critiques and    |      |        |
| . | preliminary scores grouped by application.       |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The system must allow the SRA/GTA to generate a  | 2    | S      |
| 7 | printable report of all meeting critiques and    |      |        |
| . | preliminary scores grouped by reviewer.          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The printable reports of all meeting critiques   | 1    | M      |
| 8 | should separate each critique with a page break. |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The printable reports should be available in two | 1    | S      |
| 9 | formats: MS Word documents, Adobe PDF documents. |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | The system must track date and time of critique  | 1    | M      |
| 0 | posting.                                         |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | The system could allow the SRAs/GTAs to post the | 3    |        |
| 1 | application abstract themselves (for reviewers   |      |        |
| . | to read or that can be used to auto assemble the |      |        |
|   | summary statement).                              |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | The system could include online, completely      | 3    |        |
| 2 | electronic, conflict of interest forms.          |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | Another option for deadline dates for posting    |      | W      |
| 3 | critiques can be considered. Once a reviewer has |      |        |
| . | posted their finalized critique, they can be     |      |        |
|   | blocked from editing it and will then            |      |        |
|   | immediately be able to see other posted          |      |        |
|   | critiques for that application. 3/25/02 After    |      |        |
|   | discussion, Group decided not to include this    |      |        |
|   | feature in the system. Most critiques aren't     |      |        |
|   | submitted until 48 hours before the Read phase   |      |        |
|   | so this would have little benefit and may        |      |        |
|   | introduce problems since Reviewers often need to |      |        |
|   | modify a critique during the Post phase.         |      |        |
+---+--------------------------------------------------+------+--------+

## Read Phase

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | The system should not allow critiques to be      | 1    | M      |
| . | modified by Reviewers during the read phase.     |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The system must never allow Reviewers to see     | 1    | M      |
| . | critiques or scores for applications on which    |      |        |
|   | they are in conflict.                            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | Unless blocked by the SRA/GTA or in conflict,    | 1    | M      |
| . | the system should allow all                      |      |        |
|   | [Regular]{.underline} Reviewers to view          |      |        |
|   | critiques and scores for all applications.       |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | Unless blocked by the SRA/GTA for not submitting | 1    | M      |
| . | or in conflict, the system should allow all      |      |        |
|   | [Telephone]{.underline} Reviewers to view all    |      |        |
|   | critiques and scores for their assigned          |      |        |
|   | applications (this is the default view).         |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | If specified by the SRA/GTA in the IAR control   | 1    | M      |
| . | center, the Telephone reviewers may view all     |      |        |
|   | meeting critiques (unless in conflict or blocked |      |        |
|   | by the SRA/GTA for not submitting)               |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | The system should not allow Mail-in Reviewers to | 1    | M      |
| . | view critiques submitted by others.              |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | For the Reviewer, the application number, pi     | 1    | M      |
| . | name, assignment type, average score, score and  |      |        |
|   | the date and time of posting need to be          |      |        |
|   | displayed in the critique header.                |      |        |
+---+--------------------------------------------------+------+--------+
| 8 | For the SRA/GTA, the application number, PI      | 1    | M      |
| . | name, Reviewer name, assignment type, average    |      |        |
|   | score, score and the date and time of posting    |      |        |
|   | need to be displayed in the critique header.     |      |        |
+---+--------------------------------------------------+------+--------+
| 9 | If the SRA/GTA designates in IAR Control Center  | 1    | M      |
| . | to hide scores, they should not display on       |      |        |
|   | Critiques.                                       |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | Critiques or comments from unassigned Reviewers  | 1    | M      |
| 0 | should be marked as "Unassigned" when viewing    |      |        |
| . | the list of critiques.                           |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The system should allow a blocked (by the        | 1    | M      |
| 1 | SRA/GTA) Reviewer to post their critique.        |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | After a blocked Reviewer posts a late critique   | 1    | M      |
| 2 | during the read only phase his block must be     |      |        |
| . | lifted for that application and he will be able  |      |        |
|   | to read the critiques submitted by other         |      |        |
|   | Reviewers.                                       |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | Default sort for the SRA/GTA "View by Reviewer"  | 1    | M      |
| 3 | should be first by Reviewer Name, second by Role |      |        |
| . | and third by PI name.                            |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | A sort on Application number should have a       | 1    | M      |
| 4 | secondary sort on PI Name.                       |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | A sort on PI name should have a secondary sort   | 1    | M      |
| 5 | on Application number.                           |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | A sort by Average Score should have a secondary  | 1    | M      |
| 6 | sort on PI name.                                 |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | Some SRA/GTAs read critiques as they are added   | 2    | S      |
| 7 | to the ER Web site allowing them to be better    |      |        |
| . | prepared for meeting and to spot potential       |      |        |
|   | problems. A useful feature would be the ability  |      |        |
|   | to mark an application as read and approved by   |      |        |
|   | the SRA/GTA to help streamline the assembly of   |      |        |
|   | triaged summary statements in particular. If a   |      |        |
|   | critique is updated then the check mark will be  |      |        |
|   | removed automatically.                           |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | Allow reviewers to submit comments after the     |      | W      |
| 8 | submission deadline \[these comments would be    |      |        |
| . | marked as after the deadline and changes to the  |      |        |
|   | pre-deadline critiques should not be allowed     |      |        |
|   | until after the meeting\]. This might be limited |      |        |
|   | to the unassigned or might also include assigned |      |        |
|   | reviewers, or comments could be mailed to        |      |        |
|   | SRA/GTA who will have ability to post for the    |      |        |
|   | unassigned or assigned. Development of such a    |      |        |
|   | function will require more policy input.         |      |        |
|   | Clearly, the timing the peer review process      |      |        |
|   | would have to be substantially changed for       |      |        |
|   | reviewers to utilize it optimally. Careful       |      |        |
|   | thought has to be given to such a step and clear |      |        |
|   | policies developed. 4/25/02 discussion, group    |      |        |
|   | decided that during the read phase, no one       |      |        |
|   | should be permitted to submit critiques for      |      |        |
|   | unassigned applications. This feature was not    |      |        |
|   | recommended.                                     |      |        |
+---+--------------------------------------------------+------+--------+

## Streamlining 

Streamlining is the process for identifying applications that will not
be reviewed at the meeting. To accomplish this task, both SRAs and
Reviewers must agree on the list of applications (usually with high
scores or NR) that will not be reviewed. To assist in identifying the
applications, there is a need for score matrix screen that will list all
applications in the meeting with scores. The screen will allow SRAs to
identify streamlined applications.

+---+--------------------------------------------------+------+--------+
|   | **Requirement**                                  | **V  | **Mu   |
|   |                                                  | ersi | SCoW** |
|   |                                                  | on** |        |
+===+==================================================+======+========+
| 1 | The system shall allow users to navigate to the  |      |        |
| . | score matrix screen from the list of             |      |        |
|   | applications screen.                             |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 2 | The system shall allow users to navigate to the  |      |        |
| . | list of applications from the score matrix       |      |        |
|   | screen.                                          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 3 | System shall provide a score matrix view of      |      |        |
| . | applications in the meeting for both SRA/GTAs    |      |        |
|   | and Reviewers.                                   |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 4 | System shall allow reviewers to access score     |      |        |
| . | matrix in Read phase.                            |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 5 | System shall allow SRA/GTAs to access score      |      |        |
| . | matrix in both Submit and Read phases.           |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 6 | Score matrix screen should show application      |      |        |
| . | number, PI name, average score, individual       |      |        |
|   | scores, and UN (unscored check box).             |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 7 | Average score is computed only if every score    |      |        |
| . | for the application is numeric.                  |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 8 | SRA/GTA must be able to edit the UN checkbox to  |      |        |
| . | identify an application as streamlined.          |      |        |
|   |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 9 | SRA/GTA and Reviewers must be able to toggle     |      |        |
| . | between the complete list of applications and    |      |        |
|   | the streamlined only applications list.          |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | Reviewers cannot edit anything on score matrix   |      |        |
| 0 | screen.                                          |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | System shall filter out applications where       | [    |        |
| 1 | Reviewers are in conflict.                       | ]{.m |        |
| . |                                                  | ark} |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | If Reviewer is blocked from reading critiques    |      |        |
| 2 | (if they didn't submit) they cannot see the      |      |        |
| . | score matrix for that application.               |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | Score matrix should allow sort by                |      |        |
| 3 | activity/IC/serial number, IC/Serial number,     |      |        |
| . | activity/PI name, PI name, Average score and ??? |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The system shall prevent Telephone reviewers     |      |        |
| 4 | from seeing applications for which they are not  |      |        |
| . | assigned.                                        |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+
| 1 | The system shall prevent Mail-in reviewers from  |      |        |
| 5 | seeing the score matrix???                       |      |        |
| . |                                                  |      |        |
|   |                                                  |      |        |
+---+--------------------------------------------------+------+--------+

## Edit Phase

The SRA may wish to have an after meeting Edit phase during which time
Reviewers can modify their critiques.

+---+----------------------------------------------+------+----------+
|   | **Requirement**                              | **V  | **       |
|   |                                              | ersi | MuSCoW** |
|   |                                              | on** |          |
+===+==============================================+======+==========+
| 1 | Unassigned members should be allowed to      |      |          |
| . | submit post-meeting comments or to edit      |      |          |
|   | their earlier comments.                      |      |          |
|   |                                              |      |          |
+---+----------------------------------------------+------+----------+
| 2 | The date stamp in ER that allows SRAs/GTAs   |      |          |
| . | to track when a critique was updated is      |      |          |
|   | important. This date can be used as a filter |      |          |
|   | in IAR to retrieve the subset of critiques   |      |          |
|   | posted since a particular date. Color-coding |      |          |
|   | could be used to aid tracking.               |      |          |
+---+----------------------------------------------+------+----------+

## IC Program Officer Access

+---+-------------------------------------------------+------+--------+
|   | **Requirement**                                 | **V  | **Mu   |
|   |                                                 | ersi | SCoW** |
|   |                                                 | on** |        |
+===+=================================================+======+========+
| 1 | SRA/GTA should have the option of making the    |      |        |
| . | lower half list public to Program Officers.     |      |        |
|   | This allows the PO to optimize their travels    |      |        |
|   | between SRG meetings.                           |      |        |
+---+-------------------------------------------------+------+--------+
| 2 | The system could place critique text directly   |      |        |
| . | into draft summary statements. In IMPAC II,     |      |        |
|   | there is already a Preview mode for SRA/GTAs to |      |        |
|   | share summary statements with POs. If critiques |      |        |
|   | can be directly loaded as draft summary         |      |        |
|   | statements then the preview feature will allow  |      |        |
|   | quick IC access if necessary.                   |      |        |
+---+-------------------------------------------------+------+--------+

## Reports

+---+-------------------------------------------------+------+--------+
|   | **Requirement**                                 | **V  | **Mu   |
|   |                                                 | ersi | SCoW** |
|   |                                                 | on** |        |
+===+=================================================+======+========+
| 1 | A useful additional feature would be to allow   |      |        |
| . | the reviewers to print their own copies of      |      |        |
|   | their assignment lists. The format would need   |      |        |
|   | to be SRA/GTA controlled---either "Assignment   |      |        |
|   | List and Conflicts by Reviewer" (full           |      |        |
|   | assignment information on only those            |      |        |
|   | applications assigned to the reviewer) or       |      |        |
|   | "Assignment List and Conflicts by Reviewer      |      |        |
|   | (Restricted Version)" (no information on        |      |        |
|   | co-reviewers). The New Investigator asterisk    |      |        |
|   | should print.                                   |      |        |
+---+-------------------------------------------------+------+--------+
| 2 | SRA/GTA should have a Meeting Report 1---A      |      |        |
| . | two-column report displaying the PI name and    |      |        |
|   | application number along with the current       |      |        |
|   | information in the Results field in the PSR     |      |        |
|   | screen. Another column would print a C next to  |      |        |
|   | those applications for which there is a         |      |        |
|   | conflict in the system, since those cannot be   |      |        |
|   | discussed at all when streamlining is confirmed |      |        |
|   | at the beginning of the meeting. Numbering is   |      |        |
|   | needed to correspond with reviewer vote sheets. |      |        |
+---+-------------------------------------------------+------+--------+
| 3 | SRA/GTA should have a Meeting Report 2---For    |      |        |
| . | reference, a copy of the master assignment list |      |        |
|   | with reviewers who voted to streamline a        |      |        |
|   | particular application printing in bold.        |      |        |
+---+-------------------------------------------------+------+--------+
| 4 | SRA/GTA should have a Meeting Report            |      |        |
| . | 3---Preliminary Score Matrix. The score matrix  |      |        |
|   | should allow multilevel sorts. For example, it  |      |        |
|   | would be very useful to be able to sort by      |      |        |
|   | mechanism then PI name.                         |      |        |
+---+-------------------------------------------------+------+--------+
| 5 | All the current tools for SRA/GTAs in ER to     |      |        |
| . | view and print score and critiques are useful.  |      |        |
|   | The ability to sort outputs can be made more    |      |        |
|   | flexible with the addition of secondary sorts   |      |        |
|   | where applicable. Further, reports can be       |      |        |
|   | enhanced with functions to batch print selected |      |        |
|   | critiques.                                      |      |        |
+---+-------------------------------------------------+------+--------+
| 6 | SRA/GTA needs the ability to print all reviews, |      |        |
| . | sorted by PI, with each PI beginning on a new   |      |        |
|   | page. The requested feature would allow all     |      |        |
|   | SRA/GTAs to bring a collated copy of each       |      |        |
|   | \"proto summary statement\" to the meeting and  |      |        |
|   | be able to refer to them during the meeting.    |      |        |
|   | This allows SRA/GTAs to ensure that key points  |      |        |
|   | made at the meeting are actually written into   |      |        |
|   | the critiques.                                  |      |        |
+---+-------------------------------------------------+------+--------+
| 7 | SRA/GTA will need a printable report of the     |      |        |
| . | lower half list (to take as the official list   |      |        |
|   | to the meeting) and their associated combined   |      |        |
|   | critiques.                                      |      |        |
+---+-------------------------------------------------+------+--------+
| 8 | SRA/GTA will need a printable report of a list  |      |        |
| . | of applications that have been nominated by one |      |        |
|   | reviewer for streamlining.                      |      |        |
|   |                                                 |      |        |
+---+-------------------------------------------------+------+--------+
| 9 | SRA/GTA will need a Critique Posting Status     |      |        |
| . | Report similar to report current in ER (example |      |        |
|   | provided by Cindy Lassnoff 4/25/02)             |      |        |
|   |                                                 |      |        |
+---+-------------------------------------------------+------+--------+
| 1 | SRA/GTA will need a printable report of the     |      |        |
| 0 | significant difference list and their           |      |        |
| . | associated combined critiques.                  |      |        |
|   |                                                 |      |        |
+---+-------------------------------------------------+------+--------+

## Summary Statement Assembly

+---+-------------------------------------------------+------+--------+
|   | **Requirement**                                 | **V  | **Mu   |
|   |                                                 | ersi | SCoW** |
|   |                                                 | on** |        |
+===+=================================================+======+========+
| 1 | The main post-meeting report is the assembled   |      |        |
| . | critiques in a pre-summary statement draft.     |      |        |
|   |                                                 |      |        |
|   | Two versions are required:                      |      |        |
|   |                                                 |      |        |
|   | -   One similar to the current ER draft that    |      |        |
|   |     shows application, reviewer role (not name) |      |        |
|   |     and score                                   |      |        |
|   |                                                 |      |        |
|   | -   A second Word output that is closer to a    |      |        |
|   |     true summary statement draft                |      |        |
|   |                                                 |      |        |
|   | \[RTF output will provide compatibility with    |      |        |
|   | all major word processors.\]                    |      |        |
|   |                                                 |      |        |
|   | Production of this second output must take the  |      |        |
|   | opportunity to maximize autoassembly of draft   |      |        |
|   | summary statement text. Critiques would begin   |      |        |
|   | with the heading "Critique" (a nice touch would |      |        |
|   | allow SRA/GTAs to rearrange the order of        |      |        |
|   | critiques; the default order should be by       |      |        |
|   | role). \[Although many reviewers add the        |      |        |
|   | heading "critique," they can be asked not to do |      |        |
|   | this.\] The description would be added if       |      |        |
|   | available. A further nice touch would create an |      |        |
|   | output with as many template headings as        |      |        |
|   | possible. So, for example, if there are human   |      |        |
|   | subjects codes, the appropriate headings can be |      |        |
|   | created in the output. The bolded statement     |      |        |
|   | proposed by OER for separating reviewer and     |      |        |
|   | SRA/GTA remarks can be added. If biohazard of   |      |        |
|   | foreign are checked, these headings can be      |      |        |
|   | added, etc. If such an option is provided, it   |      |        |
|   | will be important to be able to toggle off the  |      |        |
|   | template.                                       |      |        |
+---+-------------------------------------------------+------+--------+
| 2 | One possibility---the system will facilitate    |      |        |
| . | summary statement creation by merging the       |      |        |
|   | critiques into one document to be used as the   |      |        |
|   | draft summary statement body text and allowing  |      |        |
|   | the SRA/GTA to export this document.            |      |        |
+---+-------------------------------------------------+------+--------+
| 3 | Export to Summary Statement Module. This option |      |        |
| . | would formally associate each file for the      |      |        |
|   | designated application to allow access through  |      |        |
|   | the summary statement module. Until the button  |      |        |
|   | is pushed, the files should remain in a         |      |        |
|   | temporary file. There would need to be an       |      |        |
|   | "update" button that would bring in the most    |      |        |
|   | recent posting, and there should be a warning   |      |        |
|   | when a newer version has been posted. The       |      |        |
|   | advantage of this scheme would be in knowing    |      |        |
|   | which version you are working with so that an   |      |        |
|   | update would not be posted without your         |      |        |
|   | knowing.                                        |      |        |
+---+-------------------------------------------------+------+--------+
| 4 | Direct Storage in the Summary Statement Module. |      |        |
| . | Posted critiques would be available to the      |      |        |
|   | SRA/GTA through the IMPAC II Peer Review        |      |        |
|   | Summary Statement Module as soon as posted. The |      |        |
|   | difficulty would be in keeping track of when a  |      |        |
|   | review has been modified. A log could show the  |      |        |
|   | SRA/GTA when updates have been posted, but it   |      |        |
|   | might be difficult to keep track of those       |      |        |
|   | changes when working offline on a draft in Word |      |        |
|   | or WordPerfect.                                 |      |        |
+---+-------------------------------------------------+------+--------+
| 5 | Automated Assembly. The IAR and/or the summary  |      |        |
| . | statement module should have a display of which |      |        |
|   | reviews are in and which are missing. When all  |      |        |
|   | expected reviews are there, an Export Raw       |      |        |
|   | Reviews button should assemble the reviews in a |      |        |
|   | prescribed order (e.g., Primary, Secondary,     |      |        |
|   | Tertiary, Mail, Discussant) and allow the       |      |        |
|   | SRA/GTA to save the assembled document on the   |      |        |
|   | c: drive with the prescribed file name format   |      |        |
|   | needed for later upload. PROBLEM ---How to deal |      |        |
|   | with files created in different word processing |      |        |
|   | programs. As noted above, we'd like to retain   |      |        |
|   | special characters. If the SRA/GTA specifies    |      |        |
|   | that the downloaded document should be in Word, |      |        |
|   | for instance, are there conversion programs to  |      |        |
|   | handle a WordPerfect document on the fly?       |      |        |
+---+-------------------------------------------------+------+--------+
| 6 | When exporting critiques, care should be taken  |      |        |
| . | regarding retention of paragraph breaks,        |      |        |
|   | deletion of superfluous hard returns. One       |      |        |
|   | problem with the current NIAID system is that   |      |        |
|   | some types of output result in the loss of      |      |        |
|   | paragraph breaks while others put a hard return |      |        |
|   | at the end of every line. Macros can often take |      |        |
|   | care of these problems, but the system should   |      |        |
|   | provide clean output if possible                |      |        |
+---+-------------------------------------------------+------+--------+
| 7 | The summary statement contains a "Description"  |      |        |
| . | submitted on the grant application. Since       |      |        |
|   | applications are scanned and bookmarked, this   |      |        |
|   | "Description" section should be evaluated for   |      |        |
|   | feasibility of automatically incorporating it   |      |        |
|   | into the summary statement during               |      |        |
|   | generation/combination of critiques.            |      |        |
+---+-------------------------------------------------+------+--------+
| 8 | If SRAs/GTAs have the capacity to download all  |      |        |
| . | the summary statements (collected critiques     |      |        |
|   | under each grant number) with a separate file   |      |        |
|   | for each summary statement (named with the      |      |        |
|   | grant number or PI name), this could be         |      |        |
|   | incorporated into their post meeting processes  |      |        |
|   | as a one-time event and will prevent            |      |        |
|   | inadvertent loss of data. The files might be    |      |        |
|   | zipped together for the download in a procedure |      |        |
|   | similar to that now used in IMPAC II.           |      |        |
|   |                                                 |      |        |
|   | The critiques for each application should be    |      |        |
|   | ordered (e.g., Primary, Secondary, Tertiary,    |      |        |
|   | Mail, Discussant) and the SRA/GTA needs control |      |        |
|   | over the order (PI name vs. application         |      |        |
|   | number). The insertion of some special          |      |        |
|   | character string (e.g., page breaks) between    |      |        |
|   | applications would allow efficient separation   |      |        |
|   | for storage in individual files.                |      |        |
+---+-------------------------------------------------+------+--------+
| 9 | If a feature to allow reviewers to attach files |      |        |
| . | is created, SRA/GTAs will not want to download  |      |        |
|   | these files but rather handle the rich text     |      |        |
|   | only.                                           |      |        |
+---+-------------------------------------------------+------+--------+
| 1 | A feature can be provided to use the text to    |      |        |
| 0 | assemble the IMPAC II .PDF draft summary        |      |        |
| . | statement avoiding an intermediary Word file.   |      |        |
|   | Often streamlined summary statements will need  |      |        |
|   | no editing and they can be rapidly released.    |      |        |
|   | However, such a function should be built to     |      |        |
|   | avoid inadvertent release of unread critiques.  |      |        |
|   | It could be combined with a check box           |      |        |
|   | indicating that the SRA/GTA has approved the    |      |        |
|   | critique. The check box would only be visible   |      |        |
|   | on the SRA/GTA's screen similar to the private  |      |        |
|   | check box on the Review module 1500 screen.     |      |        |
+---+-------------------------------------------------+------+--------+

## Purging/Meeting Closure

+---+------------------------------------------------+-------+--------+
|   | **Requirement**                                | *     | **Mu   |
|   |                                                | *Vers | SCoW** |
|   |                                                | ion** |        |
+===+================================================+=======+========+
| 1 | Reviewer assignments should be purged after    |       |        |
| . | period of time.                                |       |        |
|   |                                                |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 2 | IAR only Reviewer accounts should be deleted   |       |        |
| . | on the meeting closure date.                   |       |        |
|   |                                                |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 3 | Reviewers must not be allowed to access the    |       |        |
| . | meeting in IAR after the meeting closure date. |       |        |
|   |                                                |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 4 | How does releasing the meeting in Peer Review  |       |        |
| . | relate to IAR?                                 |       |        |
|   |                                                |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 5 | How does getting the critiques from IAR relate |       |        |
| . | to purging/meeting closure.                    |       |        |
|   |                                                |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 6 | When should all data on a meeting in IAR be    |       |        |
| . | purged?                                        |       |        |
|   |                                                |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 7 | From 3/18 minutes:                             |       |        |
| . |                                                |       |        |
|   | There was also considerable discussion on the  |       |        |
|   | trigger for purging of assignment information. |       |        |
|   | In the end, it was agreed that this purge      |       |        |
|   | should remain linked to pressing the "Release  |       |        |
|   | Meeting" button in REV. However, if an Edit    |       |        |
|   | Phase End Date is in the IAR system and that   |       |        |
|   | End Date is later than the current date and    |       |        |
|   | time, the user should get a warning that       |       |        |
|   | continuing will delete the assignment          |       |        |
|   | information from IAR (in addition to the       |       |        |
|   | warning about running the Conflict report).    |       |        |
|   | Even if it is deleted, reviewers would still   |       |        |
|   | be able to post. However, they would have to   |       |        |
|   | select from the entire list of application (a  |       |        |
|   | screen needed to allow for unassigned posting  |       |        |
|   | anyway) rather than a list restricted to their |       |        |
|   | own assignments. Retention of conflict         |       |        |
|   | information would be very nice.                |       |        |
+---+------------------------------------------------+-------+--------+
| 8 | How does the meeting closure date related to   |       |        |
| . | purging? What event(s) should the meeting      |       |        |
|   | closure date trigger?                          |       |        |
|   |                                                |       |        |
+---+------------------------------------------------+-------+--------+
| 9 | Critiques and reviewer assignments should not  |       |        |
| . | be linked together. Unlinked assignments can   |       |        |
|   | be purged leaving critiques in place.          |       |        |
|   | \[Depending on timing, there is a potential    |       |        |
|   | clash of reviewer purge and post meeting       |       |        |
|   | editing, critique download/summary statement   |       |        |
|   | assembly.\]                                    |       |        |
+---+------------------------------------------------+-------+--------+

## Other General Features

+---+-------------------------------------------------+---+------+---+-------+
|   | **Requirement**                                 |   | **V  |   | **MuS |
|   |                                                 |   | ersi |   | CoW** |
|   |                                                 |   | on** |   |       |
+===+=================================================+===+======+===+=======+
| 1 | In all screens displaying application numbers,  |   |      |   |       |
| . | display the mechanism. It is very difficult to  |   |      |   |       |
|   | evaluate score distributions and perform other  |   |      |   |       |
|   | essential tasks if mechanisms are not           |   |      |   |       |
|   | displayed.                                      |   |      |   |       |
+---+-------------------------------------------------+---+------+---+-------+
| 2 | Privacy for the IMPAC II Review module is at    |   |      |   |       |
| . | the level of the IRG (or equivalent). The same  |   |      |   |       |
|   | level of privacy appears to be appropriate for  |   |      |   |       |
|   | IAR. However, the private check box on the      |   |      |   |       |
|   | Review assignment screen limiting access to the |   |      |   |       |
|   | SRA/GTA only would have to work in the same way |   |      |   |       |
|   | for IAR if this privacy mode is to be kept in   |   |      |   |       |
|   | the Review module.                              |   |      |   |       |
+---+-------------------------------------------------+---+------+---+-------+
| 3 | There may be an opportunity to handle Travel    |   |      |   |       |
| . | Voucher via IAR.                                |   |      |   |       |
|   |                                                 |   |      |   |       |
|   |                                                 |   |      |   |       |
+---+-------------------------------------------------+---+------+---+-------+

# Constraints

IAR will be developed using the new eRA development environment, J2EE.
Consequently, the technology used for IAR will differ from the current
technology of the Peer Review module. Due to technology and time
constraints, it is recommended that the current Peer Review module is
only modified where necessary to accommodate data or functions critical
to the IAR process. Data and functions accessible through IAR should not
be replicated in Peer Review. With the future redesign of Peer Review in
J2EE, integration of SRA/GTA IAR functions will be addressed and
integrated where feasible.

# Quality Ranges

This section defines the quality ranges for performance, robustness,
fault tolerance, usability, and similar characteristics for this
application. These characteristics will be discussed in more detail in
the Supplemental Specification document.

**Availability:** The System shall be available 24 hours a day, 7 days a
week.

**Usability**: The System shall allow the users to provide 'human
factors' feedback online.

**Usability:** The System shall include on-line help for the user. Users
should not require the use of a hardcopy Manual to use the System.

**Maintainability:** The system shall not hardcode system parameters.

# Precedence and Priority

This section provides some direction on the relative importance of the
proposed system features. Until the detailed requirements are fully
defined, it is difficult to estimate schedules and establish priorities.
As time progresses, this section will be filled in with a prioritized
list of features per release.

# Other Product Requirements

## Applicable Standards

The desktop user-interface shall run under the Netscape Navigator
Version TBD or greater or Internet Explorer Version TBD or greater.

## System Requirements

-   The system shall interface with the existing IMPAC II System.

-   The server component of the system shall operate on a Sun Solaris
    operating system, located at the NIH CIT.

-   The client component of the system shall operate on any personal
    computer with Netscape Navigator Version TBD or greater or Internet
    Explorer Version TBD or greater.

## Performance Requirements

Detailed performance requirements will be described in the Supplementary
Specification document.

## Environmental Requirements

None.

## Security Recommendations

Bobbi Spitzberg will provide recommendations.

# Documentation Requirements

This section describes the documentation requirements of the Internet
Assisted Review System.

## User Manual

The User Manual shall describe use of the System from users' viewpoint.
The User Manual shall include:

-   Minimum system requirements

-   Logging on

-   Logging off

-   All system features

-   Customer support information

-   System Administrators Manual

The User Manual shall be available as hardcopy and through online help.

## On-line Help

On-line Help shall be available to the user for each system function.
Each topic covered in the User Manual shall also be available through
the on-line help.

## Installation Guides, Configuration, Read Me File

Since this application will be a Web-based application, no specific user
installation will be required.

## Labeling and Packaging

The NIH eRA logo shall be prominent on the user documentation and splash
screens.

## Appendix A: Registration Email to Reviewer

> Subject: NIH IAR Review Registration
>
> Dear Reviewer:
>
> Thank you for agreeing to be a participant on a peer review panel. To
> submit your written reviews electronically, you will need to log on to
> the NIH eRA Internet Assisted Review (IAR) web site with a user name
> and password. To establish that user name and password, we have set up
> a special URL (address on the Internet) that is unique to you. To
> ensure confidentiality and security, we have also sent you a Personal
> Identifier Number (PIN) by separate means. These two unique pieces of
> identification are required to establish an NIH eRA IAR account for
> critique submission.
>
> To establish your account, you must:

1.  Wait for your PIN to arrive by separate means. If you do not receive
    a PIN within one week of this email, please contact
    <sra@institute.nih.gov>.

2.  Open your Web browser and go to the URL
    [https://xxx.xxx.xxx.xxx](https://xxx.xxx.xxx.xxx/). (You can copy
    and paste this address into the "Location" window of your browser,
    and press *Enter.*)

3.  Follow the instructions on the screen to enter your PIN and to
    select your user name, password, and password reminder question.

> If you have questions or problems setting up your account, please call
> the NIH eRA Helpdesk at 1-301-402-7469 or 1-866-504-9552.
>
> Thank you for your time and effort.
>
> Sincerely,
>
> SRA, Ph.D./MD
>
> Scientific Review Administrator
>
> Somewhere in the Bowels of the Beast
>
> Bethesda, MD 2089x-xxx
>
> 301-xxx-xxxx

## Appendix B: Letter with PIN to Reviewer

> Dear Reviewer:
>
> Thank you for agreeing to participate as a Reviewer for the (name of
> committee), which was the subject of an email you should have received
> this week, entitled, *NIH IAR Review Registration.* You will submit
> your critiques electronically through the Internet Assisted Review
> (IAR) module.
>
> To provide a secure system that ensures data integrity, we ask that
> you follow these steps the first time you log on to the IAR module:

1.  Go to the special URL, that was sent to you via email by your SRA.

2.  Type your unique Personal Identification Number (PIN) (see your
    PIN\* below).

3.  Create your user name, password, and a password reminder question
    and answer.

4.  Follow the instructions to proceed to the IAR module login screen.

> Keep your user name and password handy for logging on to the IAR
> system.
>
> To log on to the IAR system, follow these steps:

1.  Go to ([www.URL](http://www.URL/) address).

2.  Bookmark this page for future use.

3.  Type your user name and password at the prompt.

> If you have questions or problems setting up your account, please call
> the NIH eRA Helpdesk at 1-301-402-7469 or 1-866-504-9552.
>
> Thank you for your time and effort.
>
> Sincerely,
>
> SRA, Ph.D./MD
>
> Scientific Review Administrator
>
> Address Lines
>
> Bethesda, MD 2089x-xxx
>
> 301-xxx-xxxx
>
> \*Your Personal Identification Number (PIN): xxxxxxx

## Appendix C: IAR User Agreement

User Agreement

By acceptance of this User Name and Password, I agree to safeguard the
security of the Internet Assisted Review data. This information is
protected by the Privacy Act of 1974 (PL93-579). In addition, I agree to
the following:

1.  I will not disclose my User Name and Password to anyone.

2.  My User Name and Password are considered the equivalent of my legal
    signature.

3.  I will not attempt to learn another user's User Name and Password or
    access information in the system by using a User Name and Password
    other than my own.

4.  I will only access information for which I have a demonstrable need
    to know based on my official duties.

5.  If I have reason to believe that the confidentiality of my User Name
    and Password has been breached, I will contact the eRA Helpdesk at
    1-866-504-9552 immediately so that the suspect User Name and
    Password can be deleted and a new one assigned to me.

6.  My User Name and Password will be deleted from the IAR system when I
    no longer require access for work-related activities.

7.  I will log off terminals when I am not actively using them to
    protect them against unauthorized access.

If I knowingly fail to comply with any of the above requirements, I may
be subject to disciplinary action. Reissue of a User Name and Password
to me after violation of any of the above statements will be dependent
on review by appropriate IAR officials.