## 2 Materials and Methods
### AMS Score Generation with the Paired Comparison Method  
The easiest way to generate a single AMS score would be a simple average across all items. However, this approach does not reflect that some items may be more relevant for astronaut behavioral health and mission success than others. We therefore recruited 19 SMEs (13 male, mean \(\pm\) SD age 39.3 \(\pm\) 14.9 years, range 21-74 years) to assess the relative spaceflight importance of each of the AMS items with the goal to derive the AMS score as a weighted average of items using SME-determined item relevance as the weight. SMEs were chosen based on their experience and involvement in astronaut behavioral health research.  
As it is hard to rank items when the number of items is large, we used the PC method [15] where each item is compared to each other item (i.e., a full PC). For the 14 investigated AMS items, this translates to \(13+12+11+\ldots+1=91\) pairwise comparisons. The relevance of each item is then determined by the proportion of comparisons in which an item was scored as more relevant than its comparator. A proportion of 0 would mean the item was never considered more relevant while a proportion of 0.5 would mean the item was considered more relevant in 50% of the pairwise comparisons. This proportion was then used as the weight for generating the AMS score. As the importance of the individual AMS items may differ depending on mission duration, we asked SMEs to perform the PC twice: once for short duration missions (defined as 6 months or shorter) and once for long-duration missions (defined as > 6 months).  
The PC was performed with a software developed in Visual Basics for Applications (Microsoft Excel). SMEs were first provided with background information and instructions on how to operate the software (see Appendix A). For each of the 91 comparisons they were asked the following: \"Out of the two items below, please choose the one that you believe is more relevant for astronaut behavioral health during LONG duration spaceflight (>6 months)\" or \"Out of the two items below, please choose the one that you believe is more relevant for astronaut behavioral health during SHORT duration spaceflight (<6 months)\". SMEs were provided with a brief description and the anchors of each of the two items to be compared (see Table 1). They were asked to press the number key 1 if they believed the left item was more relevant and the number key 3 if they believed the left item was more relevant. The date and time of the decision were recorded with 1 s resolution. The comparisons were randomized so that each SME received a unique order of paired comparisons. Additionally, the randomization process assured that each item was presented as the left item in 40-60% of comparisons. Eight SMEs performed the PC for short-duration missions first, the other eleven subjects performed it second.  
Statistical Analysis of Paired Comparison Data  
A coefficient of consistency (CoC) was calculated to measure the consistency of each SME's ratings [15]. This measure is based on the number of circular mistakes (e.g., A > B, B > C, and C > A). The CoC value ranges from 0 to 1, with 1 indicating the complete absence of circular mistakes. The inclusion of individual SME's was based on an a priori-defined cut-off of 0.35. Exact 95% confidence intervals were calculated for the PC proportions. We also investigated whether PC proportions and decision times differed between short- and long-duration PCs with a paired t-test.