# ![](media/image1.jpeg){width="0.4166666666666667in" height="0.2972222222222222in"} Internet Assisted Review Focus Group

**Date:** April 15, 2002

**Time:** 1:30--4:00 p.m.

**Location:** NGIT, 4^th^-Floor Conference Room

**Chair:** Eileen Bradley

**Chair:** Tracy Soto

**Next Meeting:** **April 25, 2002, 1:30--4:00 p.m., NGIT, Fourth-Floor
Conference Room (note date and day change)**

## Action Items

-   (Tracy) Research the standard system conventions for incorporation
    into the IAR upon a three-time entry failure of passwords and
    identifiers.

The minutes refer to the Scope document containing requirements, which
is available on the eRA website:

-   Scope document, which lists the requirements:

    <http://era.nih.gov/Docs/IAR_Scope_Document_Draft_04-25-02.doc>

-   PowerPoint document with the proposed IAR screens

<http://era.nih.gov/Docs/IAR_Agenda_Screens_04-15-02.ppt>

The focus group, which was scheduled to end on April 22, will continue
until June 10. The April 22 meeting has been changed to Thurs., April
25, at NGIT in the 4^th^ floor conference room. Subsequent meetings will
take place on Mondays, 1:30--4:00 p.m. at a different location, yet to
be determined.

## Registration Email to Reviewer

The focus group agreed on the following content for the letter that will
be generated by the system and emailed to Reviewers regarding
instructions for logging on to the IAR system and providing them with
their special URL.

> Dear Reviewer:
>
> Thank you for agreeing to be a participant on a peer review panel. To
> submit your written reviews electronically, you will need to log on to
> the eRA Internet Assisted Review (IAR) web site with a user name and
> password. To establish that user name and password, we have set up a
> special URL (address on the Internet) that is unique to you. To ensure
> confidentiality and security, we have also sent you a Personal
> Identifier Number (PIN) by separate means. These two unique pieces of
> identification are required to establish an NIH eRA account for
> critique submission.
>
> To establish your account, you must:

1.  Wait for your PIN to arrive by separate means. If you do not receive
    a PIN within one week of this email, please contact
    <sra@institute.nih.gov>.

2.  Open your Web browser and go to the URL
    [https://xxx.xxx.xxx.xxx](https://xxx.xxx.xxx.xxx/). (You can copy
    and paste this address into the "Location" window of your browser,
    and press *Enter.*)

3.  Follow the instructions on the screen to enter your PIN and to
    select your user name, password, and password reminder question.

> If you have questions or problems in setting up your account, please
> call the NIH eRA Helpdesk at 1-888-xxx-xxx.
>
> Thank you for your time and effort.
>
> Sincerely,
>
> SRA, Ph.D./MD
>
> Scientific Review Administrator
>
> Somewhere in the Bowels of the Beast
>
> Bethesda, MD 2089x-xxx
>
> 301-xxx-xxxx

## Letter with PIN to Reviewer

The focus group agreed on the following content for the letter that will
be generated by the system and given (via USPS, FEDEX, or fax) to
Reviewers regarding instructions for logging on to the IAR system and
providing them with their unique PIN. This is the second piece of
information required for a secure registration on to the IAR system.

> Dear Reviewer:
>
> Thank you for agreeing to participate as a Reviewer for the (name of
> committee), which was the subject of an email you should have received
> this week, entitled, *NIH IAR Review Registration.* You will submit
> your critiques electronically through the Internet Assisted Review
> (IAR) module.
>
> To provide a secure system that ensures data integrity, we ask that
> you follow these steps the first time you log on to the IAR module:

1.  Go to your special URL, which was sent to you by your SRA.

2.  Type your unique Personal Identification Number (PIN) (see your
    PIN\* below).

3.  Create your user name, password, and a password reminder question
    and answer.

4.  Follow the instructions to proceed to the IAR module login screen.

> Keep your user name and password handy for logging on to the IAR
> system.
>
> To log on to the IAR system again, follow these steps:

1.  Type ([www.URL](http://www.URL/) address).

2.  Bookmark this page for future use.

3.  Type your user name and password at the prompt.

> If you have questions or problems in setting up your account, please
> call the NIH eRA Helpdesk at 301-402-7469 or 866-504-9552.
>
> Thank you for your time and effort.
>
> Sincerely,
>
> SRA, Ph.D./MD
>
> Scientific Review Administrator
>
> Somewhere in the Bowels of the Beast
>
> Bethesda, MD 2089x-xxx
>
> 301-xxx-xxxx
>
> \*Your Personal Identification Number (PIN): xxxxxxx

## IAR Screens

+-----+-----------------+---------------------------------------------+
| **  | **Screen Name** | **Changes/Comments**                        |
| Pag |                 |                                             |
| e** |                 |                                             |
+=====+=================+=============================================+
| 6   | Registration:\  | Add, at the bottom, "\* required field."    |
|     | Enter PIN       |                                             |
+-----+-----------------+---------------------------------------------+
| 7   | Registration:\  | Change the text of this sentence, "Please   |
|     | Account         | enter a User Name...," so that it reads:    |
|     | Creation        |                                             |
|     |                 | > You will use these when you log on to the |
|     |                 | > IAR.                                      |
|     |                 |                                             |
|     |                 | Change the text next to "\*Answer to read:" |
|     |                 |                                             |
|     |                 | > (case sensitive)                          |
|     |                 |                                             |
|     |                 | Add an asterisk (\*) next to the "I accept  |
|     |                 | this User Agreement" checkbox.              |
+-----+-----------------+---------------------------------------------+
| 12  | ~~Resetting~~   | Change "Resetting" to "Replacing" in the    |
|     | Replacing a     | title.                                      |
|     | Forgotten       |                                             |
|     | Password        | Delete *(one word)* and add *(case          |
|     |                 | sensitive).*                                |
+-----+-----------------+---------------------------------------------+
| 15  | Reviewer's List | Add above the table, "All times are         |
|     | of Meetings     | EST/EDT.                                    |
+-----+-----------------+---------------------------------------------+
| 16  | Reviewer's      | Change the word "Post" to "Submit."         |
|     | Application     |                                             |
|     | List---~~POST~~ | Move the lines for Meeting Dates and        |
|     | Submit Phase    | Critique Due to the right of the *Meeting   |
|     |                 | Title* and *Meeting Identifier* lines.      |
|     |                 |                                             |
|     |                 | All underlined Column titles are sortable.  |
+-----+-----------------+---------------------------------------------+
| 18  | Reviewer's      | Agreed to the placement of the Reminder for |
|     | Upload          | Human Subjects... as shown on p.18 (not the |
|     | Critique/Score  | placement shown on pp. 19 or 20).           |
|     |                 |                                             |
|     |                 | Add instructions for the Browse             |
|     |                 | button/field. This is a new business rule   |
|     |                 | for attaching files and must be explained   |
|     |                 | to users.                                   |
+-----+-----------------+---------------------------------------------+
| 22  | Reviewer View   | Change the word "Post" to "Submit."         |
|     | All             |                                             |
|     | Criti           | Add a page break between Critiques.         |
|     | ques---~~POST~~ |                                             |
|     | Submit Phase    | Do not include the Reviewer's name.         |
+-----+-----------------+---------------------------------------------+
| 23  | SRA             | Q: Should Reviewer of Application be        |
|     | View---~~POST~~ | default?                                    |
|     | Submit Phase    |                                             |
|     | (by             | > A: *Reviewer* is default in *Submit*      |
|     | application)    | > phase. *Application* is default in Read   |
|     |                 | > and *Edit* phases.                        |
|     |                 |                                             |
|     |                 | Q: Does screen need to display Title?       |
|     |                 |                                             |
|     |                 | > A. No.                                    |
|     |                 |                                             |
|     |                 | Q: Can SRA delete?                          |
|     |                 |                                             |
|     |                 | A.  Yes.                                    |
|     |                 |                                             |
|     |                 | Q: Can we combine PI/Appl# into one field?  |
|     |                 |                                             |
|     |                 | B.  No.                                     |
|     |                 |                                             |
|     |                 | Q: Is average score relevant in Submit      |
|     |                 | Phase?                                      |
|     |                 |                                             |
|     |                 | A.  Yes.                                    |
|     |                 |                                             |
|     |                 | Change "POST" to "Submit" on *Meeting       |
|     |                 | Phase* line.                                |
|     |                 |                                             |
|     |                 | Enable sorting for Application, PI Name and |
|     |                 | Score columns.                              |
|     |                 |                                             |
|     |                 | Move (view all) to Application column,      |
|     |                 | under number.                               |
|     |                 |                                             |
|     |                 | For View, will assemble into a common       |
|     |                 | format, even though original files may be   |
|     |                 | uploaded in different applications (Word,   |
|     |                 | WordPerfect, etc.)                          |
+-----+-----------------+---------------------------------------------+
| 24  | SRA             | Classify as Version 2, Should.              |
|     | View---~~POST~~ |                                             |
|     | Submit Phase    | Change "POST" to "Submit."                  |
|     | (by Reviewer)   |                                             |
|     |                 | Delete Title column.                        |
|     |                 |                                             |
|     |                 | Add ability to toggle between "Show" and    |
|     |                 | "Not Show" Discussants, Mail Reviewers and  |
|     |                 | Readers                                     |
+-----+-----------------+---------------------------------------------+

### Requirement Changes

+-------+--------------------------+-----------------------------------+
| **Num | **Requirement**          | **Comments/Changes**              |
| ber** |                          |                                   |
+=======+==========================+===================================+
| **5.  | If the Reviewer fails to | Delete.                           |
| 4.4** | enter a valid identifier |                                   |
|       | three times, the system  |                                   |
|       | alerts the SRA.          |                                   |
+-------+--------------------------+-----------------------------------+
| **5.  | If the Reviewer fails to | Change "valid identifier" to      |
| 4.5** | enter a *valid           | "valid PIN."                      |
|       | identifier* three times, |                                   |
|       | the system will expire   |                                   |
|       | the original             |                                   |
|       | registration invitation  |                                   |
|       | URL. The Reviewer will   |                                   |
|       | be instructed to contact |                                   |
|       | their SRA or the eRA     |                                   |
|       | Helpdesk for the PIN and |                                   |
|       | a new registration       |                                   |
|       | invitation URL.          |                                   |
+-------+--------------------------+-----------------------------------+
| **5.  | The IAR Registration     | Classify this requirement as a    |
| 4.7** | module should allow      | Version 1, Must.                  |
|       | Reviewer to indicate     |                                   |
|       | that the name displayed  |                                   |
|       | is not their name and    |                                   |
|       | should alert the         |                                   |
|       | Reviewer to contact      |                                   |
|       | their SRA/GTA or eRA     |                                   |
|       | Helpdesk.                |                                   |
+-------+--------------------------+-----------------------------------+
| **5.  | If Reviewer indicates    | Classify this requirement as a    |
| 4.8** | displayed name is not    | Version 1, Must.                  |
|       | their name, system       |                                   |
|       | should inactivate        |                                   |
|       | Reviewer's account.      |                                   |
+-------+--------------------------+-----------------------------------+
| **5.4 | All system-generated     | New requirement.                  |
| .15** | email communication with |                                   |
|       | the Reviewer should      | Classify this requirement as a    |
|       | include the IAR banner   | Version 1, Should.                |
|       | screen URL.              |                                   |
+-------+--------------------------+-----------------------------------+
| **5.4 | After the Reviewer       | New requirement.                  |
| .16** | enters a correct PIN and |                                   |
|       | successfully registers   | Classify this requirement as      |
|       | (creates user name,      | Version 1, Must.                  |
|       | password and answers     |                                   |
|       | question), the original  |                                   |
|       | registration invitation  |                                   |
|       | URL will expire.         |                                   |
+-------+--------------------------+-----------------------------------+
| **5.4 | After the Reviewer       | Delete. This is clarified in      |
| .17** | enters an identifier,    | 5.4..5 and 5.4.16.                |
|       | the URL from the         |                                   |
|       | original registration    |                                   |
|       | email must become        |                                   |
|       | inactive.                |                                   |
+-------+--------------------------+-----------------------------------+
| **5.  | If answer is not valid,  | Classify this requirement as      |
| 6.8** | the IAR Password module  | Version 1, Must.                  |
|       | must display a message   |                                   |
|       | indicating invalid       | Answer is displayed in IAR        |
|       | answer and prompt        | Control Center and SRA/GTA can    |
|       | Reviewer to try again or | provide this to Reviewer when     |
|       | contact SRA/GTA.         | they call.                        |
+-------+--------------------------+-----------------------------------+
| **5.6 | If all three attempts to | Classify this requirement as a    |
| .10** | provide valid username   | Version 1, Must.                  |
|       | fail, Reviewer will be   |                                   |
|       | alerted to contact their | User name is displayed in IAR     |
|       | SRA/GTA or eRA Helpdesk. | Control Center and SRA/GTA or eRA |
|       |                          | Helpdesk can provide this to      |
|       |                          | Reviewer when they call.          |
+-------+--------------------------+-----------------------------------+
| **5.6 | If all three attempts to | Classify this requirement as a    |
| .11** | provide valid answer     | Version 1, Must.                  |
|       | fail, Reviewer will be   |                                   |
|       | alerted to contact their | Answer is displayed in IAR        |
|       | SRA/GTA or eRA Helpdesk. | Control Center and SRA/GTA can    |
|       |                          | provide this to Reviewer when     |
|       |                          | they call.                        |
+-------+--------------------------+-----------------------------------+

For all requirements regarding failure if passwords or answers to user
name questions are incorrectly entered three times, what happens on the
screen when this failure occurs? What screen or message appears and
where does it take the user? This answer should be based on standard
system conventions.

Action: (Tracy) Research the standard system conventions for
incorporation into the IAR upon a three-time entry failure of passwords
and identifiers.

### Reviewer's View

+-------+--------------------------+-----------------------------------+
| **Num | **Requirement**          | **Comments/Changes**              |
| ber** |                          |                                   |
+=======+==========================+===================================+
| *     | If a critique has been   | Classify as Version 1, Must.      |
| *12** | submitted for an         |                                   |
|       | application, the List of | Change "Post" to "Submit."        |
|       | Applications should      |                                   |
|       | display these action     |                                   |
|       | items: View, Post,       |                                   |
|       | Delete.                  |                                   |
+-------+--------------------------+-----------------------------------+
| *     | If critique has not been | Classify as Version 1, Must.      |
| *13** | submitted for an         |                                   |
|       | application, the List of | Change "Post" to "Submit."        |
|       | Applications should      |                                   |
|       | display the Post action  |                                   |
|       | item.                    |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The All Applications     | Classify as Version 1, Must.      |
| *17** | list should show all     |                                   |
|       | applications for the     |                                   |
|       | meeting, including those |                                   |
|       | with conflicts.          |                                   |
+-------+--------------------------+-----------------------------------+
| *     | On the All Applications  | Classify as Version 1, Must.      |
| *18** | list, if a Reviewer is   |                                   |
|       | in conflict with an      |                                   |
|       | application, the         |                                   |
|       | assignment role should   |                                   |
|       | show "COI" and the       |                                   |
|       | Reviewer should not be   |                                   |
|       | able to perform any      |                                   |
|       | action (Submit, View,    |                                   |
|       | Delete).                 |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The system must allow    | Classify as Version 1, Should.    |
| *35** | the Reviewer to generate |                                   |
|       | a printable report of    | Make "Critiques will be assembled |
|       | critiques and            | by PI name" a separate            |
|       | preliminary scores they  | requirement.                      |
|       | have posted for all      |                                   |
|       | applications.            | Add a new requirement to separate |
|       |                          | critiques with a page break.      |
|       | Critiques will be        |                                   |
|       | assembled by PI name.    |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The printable report of  | Classify as Version 1, Should.    |
| *36** | critiques and            |                                   |
|       | preliminary scores must  |                                   |
|       | show the following: PI   |                                   |
|       | Name, Grant Number,      |                                   |
|       | Critique Submitted Date, |                                   |
|       | Score (if entered),      |                                   |
|       | Assignment Role,         |                                   |
|       | Critique Text.           |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The score entry for the  | Classify as Version 1, Must.      |
| *40** | unassigned application   |                                   |
|       | is NOT allowed.          |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The system must allow    | Classify as Version 1, Must.      |
| *61** | the SRA/GTA to generate  |                                   |
|       | a printable report of    | Put a page break between          |
|       | critiques and            | critiques.                        |
|       | preliminary scores       |                                   |
|       | batched by application.  |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The system must allow    | Classify as Version 2, Should.    |
| *62** | the SRA/GTA to generate  |                                   |
|       | a printable report of    |                                   |
|       | critiques and            |                                   |
|       | preliminary scores       |                                   |
|       | batched by Reviewer.     |                                   |
+-------+--------------------------+-----------------------------------+
| *     | The system must allow    | Classify as Version 1, Must.      |
| *63** | the SRA/GTA to generate  |                                   |
|       | a printable batch report | View All: reports available in    |
|       | of critiques and         | Word so can be customized or in   |
|       | preliminary scores for   | PDF.                              |
|       | the entire meeting.      |                                   |
|       |                          | Add icon for user to choose       |
|       |                          | either the PDF or Word format.    |
+-------+--------------------------+-----------------------------------+

# Attendees

Binder, Roberta, NIAID

Bradley, Eileen, CSR

David, Tracey, CSR

Fox, Daniel, NGIT

Gibb, Scarlett, OD

Lassnoff, Cynthia, NIAID

Levy, Adam, NGIT

Seppala, Sandy, OCO

Sinnett, Ev, CSR

Soto, Tracy, OD

White, Roy, NHLBI

Wojcik, Brian, NCI