ALMA Cycle 9 Proposal Statistics
Daisuke Iono (NAOJ)
Based on slides prepared byJohn Carpenter, Andrea Corvillón, Pei-Ying Hsieh, and Evgenia Koumpia
December 20-21, 2022
ALMA/45m/ASTE Users MeetingNumber of submitted proposals by Cycle
Slight increase in the number of submitted proposals in Cycle 9.

Number of proposals

2000

1836

1500

1578

1571

341773

1735

1769

789

1661

1381
1131

1000

919

500

00

1

2

5Cycle6Time requested by Cycle
A record amount of time requested on all three arrays, although the rate of increase is smaller than in Cycle 8.

Time requested (hours)

30,00012-m Array7-m Array
Total Power Array

25,00020,00015,00010,0005,000
0012345Cycle

6789Oversubscription rate on 12-m array by Cycle and region
Record oversubscription rate in East Asia, North American, and Chile. Europe still has the highest oversubscription

Oversubscription rate

9Europe
East Asia
North AmericaChile

753

10

12345Cycle

6789

Success rate (Grade A+B) vs. requested 12-m array time
Success rate was highest in the 30-40 hour range, but proposals with >40h continue to be highly competitive

Distribution within EA
Early career (i.e. student, PD) occupies 50% of ALMA time in Japan.

Career stage within Japan

Cycle 9 proposal review process overview
Number of PIs who have participated in the Distributed Peer Review (DPR):• Total number of proposals evaluated in DPR: 1729 proposals• 1064 individual PIs, and 1087 individual reviewers (due to delegation)
DPR is a two-stage process:• Stage 1: Scientific review. Submission of 1 proposal requires a review of 10 proposals.
Ranked/commented based on science.
• Stage 2: Review comments by other reviewers. Modify ranks/comments if needed.
• 49% of all reviewers completed Stage 2 (58% in Cycle 8)• 20% of all reviewers modified a rank (25% in Cycle 8)• 25% of all reviewers modified a comment (30% in Cycle 8)Some proposals in DPR were cancelled:• Did not complete their reviews in good faith (e.g. cut-and-pasted the same review for multiple
proposals)

Feedback from reviewers and PIs
Reviewer survey• Positive comments on reviewer tool• Top concern is the proposal assignments (i.e. proposals assignment inconsistent with reviewer’sexpertise)• Reviewers ratings of their expertise on assignments• 90% with “my field” or “some knowledge”• 10% with “little/no knowledge”• JAO is investigating combining machine learning with keywords to better identify reviewer
expertise: PeerReview4All (Stelmakh et al. 2018)• Also in communication with the ESO proposal handling team
PI survey• Top concern was the quality of the comments and expertise of some reviewers• 7 out of 10 reviews rated the comments as “very” or “somewhat” helpful

SystematicsRegion• The EA proposal ranking remains lower comparedto NA and EU• However, EA had the best (median) rank so far.

higherExperience• First-time PIs have the lowest ranks, but no strong
correlations thereafter (same trends as in previouscycles)

lowerSystematics
Observing Modes• VLBI, ToO, and full polarization modes have above-average ranks compared to other modes
Other Issues investigated but no significant trends•••••Science category
Popularity of a proposed topic
Number of non-expert reviewers
Amount of requested time
Receiver bandGender• Next slides

No significant trend was found.

Difference between male/female ranks in Cycle 9: all regions• Female PIs had poorer proposal
overall ranks than male PIs inCycle 9• p=0.06 (i.e., about 2sigma
confidence that the result could
happen by chance)

higherlower

11Male/female ranks in East Asia and all other regions
Largely attributed to East Asia (~ 3.4 sigma confidence); trend in EU and NA at 1 sigma

EA

CL/EU/NA+Other12East Asian ranks by Cycle• There were some marginally significant
differences in Cycle 5 and 6 (p=0.03-0.10)• Cycle 9 is the most extreme difference
seen for East Asia to date

Investigated many demographics for EA (reviewer expertise, reviewer
region, PI experience, proposal category) and found no obvious cause.
Believe that it is mainly random noise, but we will monitor it closely infuture cycles

Summary
Number of Proposals, Oversubscription Rate and Distribution within EA••••

Record amount of time requested on all three arrays.
Record oversubscription rate in EA, NA, and CL. EU has the highest oversubscription rate.
Japan maintains ~75% of successful proposals in EA
In Japan, students, PD and Assistant Profs occupy 80% of all successful proposals.

Cycle 9 proposal process and feedback• 1087 individual reviewers for 1729 proposals• Top concerns DPR are (1) the proposal assignment, (2) quality of the comments (and expertise of some reviewers)

Systematics••••

EA ranking remains lower compared to NA and EU, but Cycle 9 was the best so far
First-time PIs have the lowest ranks, but no significant correlations thereafter
Ranks were different between female/male, particularly in EA.
No obvious cause for the lower ranking in one gender over the other. Will monitor this closely in the future cycles.