ALMA proposal review process
EA community meeting

Proposal Handling Team
April 17, 2023

Goal of presentation
ALMA distributed peer review

Large Programs review

Basics of distributed peer review
One member of each proposer team* commits to participate in the review process

Each reviewer reviews 10 proposals (Proposal Set) for each submitted proposal

The process• Stage 1• Reviewers identify con icts of interest• Reviewers rank the proposals from 1 to 10 (best to weakest) and provide acomment• Stage 2• Access to anonymized reviews• Ranks and comments can be modi edfifl* Excluding Large Programs

Reviewer timeline for Cycle 10May 101) Proposal PI designates the reviewer in Observing Tool (OT)

Proposal deadline
1) Reviewer specify scienti c expertise in User Pro le2) Reviewer provide list of con icts of interest in User Pro le3) Deadline to provide alternative reviewer, if necessary

May 15Expertise & con icts

flfi

fl

fl

Stage 2fi

June 29 - July 131) Read reviews from other reviewers (optional)
2) Modify your ranks and comments as needed (optional)

May 24 - June 28fiStage 11) Plenary session (optional, but highly recommended)
2) Declare any con icts of interest in assigned proposals by June 13) Complete reviews by June 28 @ 15 UT (MANDATORY!)PI designates the reviewer
May 101) Proposal PI designates the reviewer in Observing Tool (OT)

Proposal deadline

Student PIs can be reviewers, but need to specify a mentor who will assist in the review.

PI designates the reviewer
May 101) Proposal PI designates the reviewer in Observing Tool (OT)

Proposal deadline

A single reviewer can be assigned a maximum of FIVE Proposal Sets

Reviewer expertiseMay 151) Reviewer specify scienti c expertise in User Pro le2) Reviewer provide list of con icts of interest in User Pro le3) Deadline to provide alternative reviewer, if necessary

Expertise & con icts

1) Log in to the ALMA Science Portal
2) Edit your User Pro le3) Go to the Expertise tab4) Select keywords that match your scienti c expertise

fifi

fi

flfi

flfifi5) Go to the Con rm tab to saveHow the PHT uses keywords to assign proposals
Priority #1

Assign proposals with the same keyword as the reviewer’s selected keywords.

Priority #2

Assign proposals in the same scienti c category as the reviewer’s expertise.

Priority #3

Assign proposals in other scienti c categories.

fifi

If a reviewer does not specify their expertise, the keywords of their proposal will be used.

Reviewers can specify their conflicts of interest
May 151) Reviewer specify scienti c expertise in User Pro le2) Reviewer provide list of con icts of interest in User Pro le3) Deadline to provide alternative reviewer, if necessaryExpertise & con icts

1) Log in to the ALMA Science Portal
2) Edit your User Pro le3) Go to the Con icts of Interest tab4) Identify ALMA users for which you have a con ictfi

fi

flflfiflfiflfi5) Go to the Con rm tab to saveTime to select new reviewer, if necessary
1) Reviewer specify scienti c expertise in User Pro le2) Reviewer provide list of con icts of interest in User Pro le3) Deadline to provide alternative reviewer, if necessaryMay 15Expertise & con icts

A single reviewer can be assigned a maximum of FIVE Proposal Sets

• After the proposal deadline, the PHT will contact those reviewers who werefifififififlfi

flselected to receive more than ve Proposal Sets
• It is expected that for these cases, the reviewer identi es a new reviewer among
the proposal co-Is• If no alternative has been identi ed by the deadline, the PHT will cancel thereviewer's extra proposals

DPR plenary session
May 24 - June 28
Stage 1

1) Plenary session (optional, but highly recommended)
2) Declare any con icts of interest in assigned proposals by June 13) Complete reviews by June 28 @ 15 UT (MANDATORY!)• The PHT will host three Webinars:• Thursday May 25, 17:00 UTC• Friday May 26, 13:00 UTC• Wednesday May 31, 02:00 UTC• During this sessions, the PHT will explain the di erent aspects offf

fldistributed peer review, and will be available to answer questions• The presentation and slides will be posted in the ALMA Science
Portal before the Webinars• Attending to one of the sessions is not mandatory, but it is highly
recommended

Stage 1: Review assigned proposals
May 24 - June 28
Stage 1

1) Plenary session (optional, but highly recommended)
2) Declare any con icts of interest in assigned proposals by June 1
3) Complete reviews by June 28 @ 15 UT (MANDATORY!)

Declare any additional con icts in your assigned proposals• for example: observing the same object(s) with the same goals

fl

fl

fl

fl

If you identify a con ict after you submitted your con icts, contact the PHT to be assigned
another proposal.

Stage 1: Review assigned proposals
May 24 - June 28Stage 11) Plenary session (optional, but highly recommended)
2) Declare any con icts of interest in assigned proposals by June 1
3) Complete reviews by June 28 @ 15 UT (MANDATORY!)• Rank the proposals from 1 (strongest) to 10 (weakest) based on scienti c merit.
• Write comments that summarize the strengths and weaknesses of the proposal• Comments will be sent to the PI verbatim.
• Reviewer’s proposal will be canceled if the reviews are not submitted on time!
• Extensions will not be granted since Stage 2 starts on June 29.

fiflThe reviewer can be changed after the proposal deadline in exceptional circumstances by having
the proposal PI contact the PHT. The Stage 1 deadline though will remain the same.

Review criteria
Overall scienti c merit• Does the proposal clearly indicate which important, outstanding questions will beaddressed?
• Will the proposed observations have a high scienti c impact on this particular eldand address the speci c science goals of the proposal?
• Does the proposal clearly describe how the data will be analyzed in order to achieve
the science goals?

Suitability of the observations to achieve the scienti c goals

fifi

fi

fififfifi• Is the choice of target (or targets) clearly described and well justi ed?
• Are the requested signal-to-noise ratio, angular resolution, largest angular scale,and spectral setup su cient to achieve the science goals?
• Does the proposal justify why new observations are needed to achieve the goals?
•• For Joint
Joint Proposals,
Proposals, does
does thethe proposal
proposal clearlyclearlydescribe
describewhy
whyobservations
observationsfrom
frommultiple
multiple observatories
observatories areare required
required toto achieve
achievethe
thesciencesciencegoals?
goals?

Stage 2: Finalize the ranks and reviews
June 29 - July 13
Stage 2

1) Read reviews from other reviewers (optional)
2) Modify your ranks and comments as needed (optional)

Read comments from the other reviewers to see if you overlooked any critical strengths orweaknesses.

Update your ranks and comments as needed.

fiStage 2 is optional. If a reviewer does not complete Stage 2, the Stage 1 ranks/comments are
considered nal.

Cycle 10 Large Programs review

ALMA Proposal Review
Committee (APRC)

External Science Assessors

Review criteria Large Programs
Overall scienti c merit
Suitability of the observations to achieve the scienti c goals

Speci c for Large Programs (LP)

fifi

fi

fi

fi

• Does the LP address a strategic scienti c issue and have the potential to lead to amajor advancement or breakthrough in the eld that cannot be achieved by
combining regular proposals?
• Are the data products that will be delivered by the proposal team given the scopeof the proposal and will the products be of value to the community?
• Is the publication plan appropriate for the scope of the proposal?

More information
almascience.org/proposing/alma-proposal-review• Dual-anonymous guidelines• Description of the distributed peer review• Detailed guidelines for the reviewers• FAQ

Contact the PHT
ALMA Helpdesk (help.almascience.org)

Department: "Proposal Review Support"

Thank you!
Questions?