Three open questions in polygenic score portability

Domain: neuroscience
Published: 2026-01-26
Editorial outcome: minor_revision

Abstract
The broad adoption of polygenic scores (PGS) is hindered by their limited portability to people that differ—in genetic ancestry or other characteristics—from the GWAS samples used to construct them. Here, we measure PGS prediction accuracy as a continuous function of individuals’ genome-wide genetic dissimilarity to the GWAS sample (genetic distance). Our results highlight three gaps in our understanding of PGS portability. First, variation in individual-level prediction accuracy is only weakly predicted by genetic distance. In fact, it is explained comparably well by socioeconomic measures. Second, trends of portability vary across traits. For several immunity-related traits, prediction accuracy drops near zero even at intermediate genetic distances—potentially reflecting fast evolutionary turnover of genetic variants associated with immunity. Third, even qualitative trends of portability can depend on how we measure predictive performance. For instance, for type 2 diabetes, precision remains roughly constant, while recall surprisingly increases with genetic distance. Together, our results show that portability cannot be understood through global ancestry groupings alone. Other, understudied factors influence portability, including the specifics of trait evolution, genetic architecture, social context, and the construction of the PGS. Addressing these gaps can aid in the development of PGS and inform more equitable applications.

Introduction
Polygenic scores (PGS), genetic predictors of complex traits based on genome-wide association studies (GWAS), are gaining traction among researchers and practitioners1–3. Yet a major problem hindering their broad application is their highly variable performance across prediction samples4–7. Often, prediction accuracy appears to decline in groups unlike the GWAS sample—in genetic ancestry, social setting, environmental exposures, or other sample characteristics4,6,8–11, restricting the contexts in which PGS can be used reliably.

This so-called portability problem is a subject of intense study. Typically, portability is evaluated through variation in the within-group phenotypic variance explained by a PGS (i.e., the coefficient of determination, R2) among genetic ancestry groups. Indeed, population genetics theory gives clear predictions for the relationship between genetic dissimilarity to the GWAS sample and PGS prediction accuracy under some models (neutral evolution12–14, directional15, or stabilizing selection13,15), all else being equal (including, e.g., assumptions about environmental effects).

However, inference based on empirical variation in R2 can be misleading for various reasons. For one, R2 can be arbitrarily low even when the model fitted to the data is correct. It also cannot be compared across transformations of the data. R2 is not comparable across datasets, because, for instance, it depends on the extent of variation in the independent variable16–18. In the context of inference about the causes of PGS portability, these issues can manifest in different ways19. For example, heterogeneity in within-group genetic variance and environmental variance can each greatly affect group differences in R2.

A related issue is that the impacts of environmental and social factors on portability are not well-understood, despite evidence illustrating these impacts can be substantial4,6,19–21. To complicate matters, such factors may be confounded with genetic ancestry, limiting our ability to make inferences based on the typical decay of R2 between PGS and trait value in ancestries less represented in GWAS samples3,4,6,20,22.

With these limitations of R2, and the possible confounding with environmental and social factors, it remains unclear how well genetic ancestry would predict the applicability of PGS for individuals. Recent work implied that individual-level prediction accuracy should be largely explained by genome-wide genetic dissimilarity to the GWAS sample (see Fig. 3 in Ding et al. 23 and Fig. 5 in Tsuo et al. 24). However, we note that this work focused on the relationship between genetic distance and the length of the prediction interval, i.e., expected uncertainty in prediction under a relationship with the prediction accuracy of the realized trait value. Understanding the drivers of variation in prediction accuracy is especially pertinent for personalized clinical risk predictions and decisions regarding their reporting to patients2,25.

This motivated us to empirically study PGS prediction accuracy at the individual level. In what follows, we highlight three puzzling observations that also point to three gaps in our understanding of the portability problem: (1) Genetic dissimilarity to the GWAS sample poorly predicts portability at the individual level, (2) portability trends (with respect to genetic distance) can be trait-specific; and (3) portability trends depend on the measure of prediction accuracy. Informed by our results, we suggest avenues of future research that can help bridge these gaps.

Results
We examined PGS portability as a function of genetic distance from the GWAS sample in the UK Biobank (UKB). For each of the 15 continuous physiological traits, we performed a GWAS in a sample of 336,923 individuals. For 69,500 individuals not included in the GWAS sample (henceforth referred to as the prediction sample), we predicted the trait value using the PGS and covariates. Using a principal component analysis (PCA) of the genotype matrix of the entire sample, we quantify each individual’s genetic distance from the GWAS sample as distance from the centroid of GWAS individuals’ coordinates in PCA space (Fig. 1A). This measure is quicker to compute, yet highly correlated with Fst between the GWAS sample and single individuals in the prediction sample (Pearson’s r > 0.9835, p-value < 2.2 × 10−16), albeit noticeably less reflective of Fst at intermediate genetic distances (Fig. 1B). The imperfect correlation may be a result of our use of only the top 40 PCs12,26. Under some theoretical conditions (such as neutral evolution, additive contribution of genotype and environment, fixed environmental variance)—Fst should perfectly predict variation in prediction accuracy due to genetic ancestry12,14,27. We standardized genetic distance such that its mean is 1 across GWAS sample individuals.

In the prediction sample, we observed a continuum of genetic distance from the GWAS sample with several clear modes, the main one at short distances: 38,992 individuals have a genetic distance of up to 10, and the remaining 30,508 individuals at distances between 10 and 199.6 (Fig. 1B, C). To ground our expectations, we estimated the mean genetic distance for three 1000 Genomes28 subsamples: Utah residents of primarily Northern and Western European descent (CEU) average at 0.5, Han Chinese in Beijing, China (CHB) average at 101.6, and Yoruba in Ibadan, Nigeria (YRI) average at 193.0 (Fig. 1C).

For each of the 15 continuous physiological traits, we measure the prediction accuracy at the group and individual level with slightly different prediction models (“Methods”). In both cases, we fitted a prediction model regressing the trait to the polygenic score and other covariates. To evaluate group-level accuracy, we split individuals into 250 bins of genetic distance, comprising of 278 individuals each. Within each bin we measure the partial R2 of the polygenic score and the trait value. To evaluate individual-level accuracy, we measured the squared difference between the PGS-predicted value and the trait, after residualizing the trait for covariates.

For some traits, such as height, group-level prediction accuracy decayed monotonically with genetic distance from the GWAS sample, as expected and reported previously (Fig. 2A)12,23,29. A major factor driving this decay appears to be an associated decay in heterozygosity in the PGS marker SNPs (single nucleotide polymorphisms) (Fig. 4B, Supplementary Fig. 22; see Patel et al. 15 and Wang et al. 29). Lower heterozygosity in PGS markers impacts the genetic variance a polygenic score can capture because it makes for a less variable predictor. The impact of genetic distance on LD with causal variation is less straightforward13,29,30.

Previous work implied that variation in individual-level prediction accuracy should be largely explained by genetic distance23,24. However, that was not the case in our analysis. While individual-level accuracy generally decayed with distance for most traits, this correlation was weak (Fig. 2B, and Supplementary Fig. 2). Even a flexible cubic spline fit of genetic distance explained little of the variance in prediction accuracy (e.g., R2 = 0.51% for height).

In fact, individual-level prediction accuracy is explained comparably well by socioeconomic measures (Fig. 3B, and Supplementary Figs. 18–21). For example, we observed a steady mean increase in squared prediction error across quantiles of Townsend Deprivation Index31 for 11 of the 15 traits examined, suggesting poorer prediction in individuals of lower socioeconomic status (Fig. 3A, C, and Supplementary Figs. 14, 16, 17; the four exceptions being white blood cell-related traits, Supplementary Fig. 15; see also similar reports in Mostafavi et al. 4, Hou et al. 6, and Nagpal and Gibson 21). Like genetic distance, the Townsend Deprivation Index only explains between 0.02% and 0.53% of the variance in squared prediction error across traits with a cubic spline. Notably, however, for the majority of traits, more variance is explained by this measure of socioeconomic status than by genetic distance (Fig. 3B).

One might expect a qualitatively similar, monotonic relationship between genetic distance and prediction accuracy across traits. Previous analyses (that have not examined individual-level prediction accuracy) observed similarly monotonic8, and even linear12,23, relationship regardless of the trait examined. However, we observed variation in this relationship among traits. Unlike the case of height, the prediction accuracy for many other traits did not decay monotonically with genetic distance. Weight and body fat percentage peaked in accuracy at intermediate genetic distances (Fig. 2D, and Supplementary Fig. 2).

In other traits we examined, in particular white blood cell-related traits, group-level prediction accuracy dropped near zero even at a short genetic distance (Fig. 2E, and Supplementary Fig. 3). There are multiple possible drivers of trait-specific portability trends. We considered, in particular, variable selective pressures on the immune system across time and geography. We hypothesized that these would lead to less portable genetic associations (across ancestry) compared to other traits. To test this prediction, we re-estimated the effects of index SNPs (SNPs included in the PGS, as ascertained in the original GWAS sample) in two subsets of the prediction sample, one closer and another farther (in terms of genetic distance) from the GWAS sample. The prediction sample-based allelic effect estimates were least consistent with the original GWAS for lymphocyte count, compared, e.g., to triglyceride levels, a trait of similar SNP heritability (Fig. 4A). To further illustrate this point, 31.7% of the index SNPs for lymphocyte count had a different sign when estimated in the original GWAS and in the closer GWAS, compared to 9.6% for triglyceride levels.

The rapid turnover of allelic effects may also interact with statistical biases. Consider, for example, the winner’s curse, whereby effect estimates are inflated due to the ascertainment of index SNPs and the estimation of their effects in the same sample32. Winner’s curse would be most severe in large effect PGS index SNPs: These SNPs are typically at lower frequencies in the GWAS sample than small effect index SNPs, because GWAS power scales with the product of squared allelic effect and heterozygosity15,33,34. If causal effects on lymphocyte count change rapidly, then large effect index SNPs may be under weaker selective constraint in the prediction sample than in the GWAS sample, and segregate at high allele frequencies. Indeed, for lymphocyte count, the heterozygosity of large-effect variants increases with genetic distance from the GWAS sample (Fig. 4B; see Supplementary Fig. 22 for other traits). As a result of the trends of heterozygosity, the variance in the polygenic score (a sum over index SNP heterozygosity multiplied by their squared effect estimates) quickly increases with genetic distance for white blood cell count, lymphocyte count, and monocyte count, despite decreasing for all other traits we have examined (Fig. 4C, and Supplementary Fig. 66). And so, taken together, the PGS variance increases quickly and allelic effect estimates become non-predictive even close to the GWAS sample (Supplementary Fig. 24). Together, this may drive the immediate drop in prediction accuracy of white blood cell-related traits.

Finally, the trends of portability, even qualitative trends, can depend on the measure of prediction accuracy. For triglyceride levels, lymphocyte count, and white blood cell count, group-level prediction accuracy is near zero far from the GWAS sample (Fig. 2, and Supplementary Figs. 3C and 5E) whereas at the individual level, prediction accuracy increases (Fig. 2F, and Supplementary Figs. 3D and 5E).

Trait-specific, measure-specific portability trends are of specific importance for diseases, since the choice of performance metrics often ties in with an intervention or policy that is being considered. For example, when a broadly accessible, safe intervention exists, we might prioritize the identification of all cases over all else and therefore focus on recall (number of true positives divided by number of true cases). In other cases, for example when the intervention includes a risky treatment, we might prioritize avoiding false positives and hence focus on precision (true positives over all positive classifications). Therefore, we sought to know whether trends of portability depend on the measures of predictive performance in disease.

To address this question, we analyzed precision and recall in two diseases with over 5000 cases in the prediction sample. For asthma, precision and recall depended on genetic distance in a qualitatively similar way (Fig. 5A). In contrast, for type 2 diabetes, precision appeared roughly constant for medium and large genetic distances, while recall generally increased with distance far from the GWAS sample (Fig. 5B). In conclusion, predictive performance showed trait-specific, measure-specific trends in disease risk prediction as well.

Methods
Data

Data overview. All analyses were conducted with data from the UKB, a large-scale biomedical database with a sample size of 502,490 individuals43. In this study, we considered 406,423 individuals who passed quality control (QC) checks, which included the removal of 359 individuals who withdrew from the study as of December 17, 2024, 651 samples identified by the UKB as having sex chromosome aneuploidy (data field 22019), and an additional 14,418 individuals whose self-reported biological sex (data field 31) differed from sex determined from that implied by their sex chromosome karyotype (data field 22001). We removed 963 individuals who are outliers in heterozygosity or genotype missingness (data field 22027) and 6851 individuals with genotype missingness greater than 2% (data field 22005). We then removed 72,825 individuals with 3rd-degree relatives or closer (data field 22020). In total, we removed 96,067 individuals. In the selection of the GWAS sample, individuals who passed all filtering steps and were labeled by the UKB as White British (WB)—those who self-identified as White and British, and closely clustered together in the PC space (336,923 individuals, data field 22006)—were included in the GWAS sample. The remaining 69,500 individuals who passed filtering (Non-White British, NWB) were used as the prediction set. In the Supplementary Materials, we investigate the sensitivity of our analysis to a different partitioning between the GWAS sample and prediction set.

Genotype data. We started with 765,067 biallelic variants out of a total of 784,256 genotyped variants on the autosomes. We first removed 10,543 SNPs within the major histocompatibility complex and extended region in strong LD with it (chromosome 6, positions 26,477,797–35,448,354 in the GRCh37 genome build). We excluded variants with a Hardy–Weinberg equilibrium p-value lower than 1×10−10 among White British (WB) individuals (PLINK 2.044,45 flag –hwe 1e-10), removing another 45,930 variants. We also removed an additional 39,996 variants by setting the minor allele frequency threshold among WB to > 0.01% (PLINK 2.044,45 flag –maf 0.0001). After filtering, we had 668,598 variants which we analyzed going forward.

Phenotype data. We analyzed 15 highly heritable continuous traits, as determined based on Neale Lab SNP heritability estimates46 (Supplementary Table 1). These included both physiological measurements and biomarkers: standing height (data field 50), cystatin C level (data field 30720), platelet count (data field 30080), mean corpuscular volume (MCV, data field 30040), weight (data field 21002), mean corpuscular hemoglobin (MCH, data field 30050), body mass index (BMI, data field 21001), red blood cell count (RBC, data field 30010), body fat percentage (data field 23099), monocyte count (data field 30130), triglyceride level (data field 30870), lymphocyte count (data field 30120), white blood cell count (WBC, data field 30000), eosinophil count (data field 30150), and LDL cholesterol level (data field 30780).

In addition, we analyzed 2 binary traits, type 2 diabetes (ICD-10 code E11) and asthma (ICD-10 code J45). We marked an individual as positive for a disease if they are either positive for main ICD-10 (UKB field 41202) or secondary ICD-10 (UKB field 41204), and included all subtypes for both diseases. For all analyses, we only used the measurement from the first visit for each individual, and removed individuals with missing trait data.

Genetic distance calculations

The fixation index (Fst) is a natural metric, a single number, to measure the divergence between two sets of chromosomes. We considered using it to measure the distance between the pair of chromosomes of an individual and chromosomes in the GWAS sample; however, calculating Fst was computationally costly. Since previous work12 showed it is tightly correlated with Euclidean distance in the PC space in the UKB, we used Euclidean distance as a single number proxying genetic distance from the GWAS sample. We used the pre-computed PCs provided by the UKB (data field 22009).

The genetic distance is the weighted PC distance between an individual coordinate vector x in the subspace spanned by the first K PCs and the centroid of the M individuals {xm}M m=1 in the GWAS sample, with C = 1/M sum M m=1 xm, and is the

sqrt(sum K k=1 (lambda_k (x_k - C_k)^2) / sum K k=1 lambda_k),

where lambda_k is the kth eigenvalue.

To identify K, the number of PCs we used, and to confirm the approximation is reasonable for our data, we examined the correlation of genetic distance with Fst as a function of K on a small subset of the prediction sample.

We randomly selected 10,000 prediction sample individuals with a weighted PC distance greater than the weighted PC distance of 95% of the GWAS set (based on weighted PC distance calculated using K=10). For those individuals, we estimated their Fst and weighted PC distance to the GWAS centroid for K in 1,..., 40 (UKB provides pre-computed individual-level coordinates for each of 40 PCs). We estimated Fst in this subsample with the Weir and Cockerham method47 using the –fst flag in PLINK 1.945,48.

Since the PC distance calculated from using K=40 correlated most strongly with Fst (Pearson’s r > 0.9835, p-value < 2.2 × 10−16) (Supplementary Fig. 1), we used this number of PC to estimate the genetic distance for all test individuals (Fig. 1B, C). We note that genetic distance is less reflective of Fst for intermediate genetic distances (Fig. 1B).

We divided the raw genetic distances by the (raw) mean genetic distance among GWAS sample individuals. To gain intuition about these standardized units of genetic distance, we wished to estimate where on this scale we would find individuals from three subsamples from the 1000 Genomes Project Phase 3 dataset28: CEU, Utah residents (CEPH) from primarily Northern and Western European descent; CHB, Han Chinese in Beijing, China; and YRI, Yoruba in Ibadan, Nigeria. To this end, we ran a PCA with a dataset that includes both the UKB individuals and the CEU, CHB, and YRI individuals. We identified the UKB individuals with the shortest weighted Euclidean distance to the centroid of each of the three 1000 Genomes populations, and used the genetic distance of those three UKB individuals in our PCA of only UKB individuals as a proxy of where the three 1000 Genomes subsamples fall on the scale of our genetic distance measurement (Fig. 1C).

The distribution of genetic distance is heavily right-skewed, with most individuals falling close to the GWAS centroid. Since we wanted to focus on the individuals far away from the GWAS set, we only analyzed data for individuals with a genetic distance greater than the 97.5th percentile of genetic distance from among GWAS sample individuals (Fig. 1C), with the exception of the analyses behind Fig. 3 and Supplementary Figs. 14–21.

For group level analyses, we binned the prediction samples by genetic distance using 250 equally-sized bins, with 278 individuals per bin.

PGS and evaluating PGS prediction accuracy

GWAS. For each trait, we used the –glm flag from PLINK 2.044,45 to run GWAS on the GWAS set. We used the following covariates: the first 20 PCs from UKB (data field 22009), age (data field 21022), age2, sex (data field 31), age × sex, and age2 × sex (× denotes the product of two variables, conferring to an interaction term).

Using PLINK 2.044,45 with the flag –glm along with –1, we also ran a GWAS for 2 binary disease traits: asthma (ICD-10 code J45) and type 2 diabetes (ICD-10 code E11), using the same covariates as the ones used for continuous traits.

For both continuous and disease traits, we used the –clump flag in PLINK 1.945,48, setting the threshold for clumping to 0.01 (–clump-p 1 0.01), LD r2 threshold to 0.2 (–clump-r2 0.2), and window size to 250 kb (–clump-kb 250).

PGS construction. After clumping and thresholding the SNPs with marginal association p < 1×10−5, we calculated PGS for each individual with weights for every phenotype. The calculations were carried out with the –score flag in PLINK 2.044,45.

PGS prediction accuracy at the group level. To evaluate prediction accuracy for continuous trait PGS at the group level, we linearly regressed the phenotype on the covariates (array type (data field 22000), age, age2, sex, age × sex, and age2 × sex) and PGS within each genetic distance bin (phenotype ~ covariates + PGS), which is the full model. These analyses rely on standard linear-model assumptions, including linearity of effects, independence of observations, and approximately normally distributed residuals. We then performed another linear regression of the phenotype on the covariates, excluding the PGS, within each bin (phenotype ~ covariates), which is the reduced model. Using these two squared correlations, we calculated partial R2 for the PGS with the sum of squared errors of these two models as

R2 = (SSE(reduced model) - SSE(full model)) / SSE(reduced model),

which represents the prediction accuracy of PGS for each bin.

As a baseline prediction accuracy, we identified the 25 bins (of 278 individuals each; 10% of the bins) with the median genetic distance most similar to the mean genetic distance for GWAS individuals; this reference group represents individuals from the prediction set that are closest to the typical GWAS individuals in terms of genetic distance. The mean PGS prediction accuracy across these 25 bins served as the baseline value. Throughout the paper, we report the prediction accuracy at the group level as a bin’s squared partial correlation between the PGS and the trait divided by this baseline value.

For binary traits, we calculated the precision (fraction of true cases out of predicted cases) and the recall (fraction of predicted cases out of all cases) in each genetic distance bin across 250 bins of 278 individuals each. To get the predicted phenotype, we used the PGS and predicted the phenotype to be positive if their PGS is in the 75th percentile of the PGS of the GWAS set for type 2 diabetes disease and 70th percentile for asthma. These thresholds were determined based on the percentile (considering 5% increments) cutoff which maximized F1 score in the GWAS set for each polygenic score.

F1 score is defined as

F1 = (2 × precision × recall) / (precision + recall),

where precision is defined as

precision = TP / (TP + FP),

and recall is defined as

recall = TP / (TP + FN),

where TP is true positives, FP is false positives, and FN is false negatives. Using this procedure, the optimal thresholds for asthma and type 2 diabetes were the 70th and 75th percentiles, respectively, as noted above.

Prediction error at the individual level. For the individual-level prediction error, we first derived phenotypic values adjusted for covariates Z in two steps, involving residualizing some covariates in each genetic ancestry bin independently and some covariates globally. First, we regress raw phenotype values Y independently in each bin on covariates, Y ~ array type + age2 + sex + age × sex + age2 × sex. In bins in which only a single individual was genotyped with a particular array type, we did not include array type as a covariate. We then regress the residual X (where X = Y - Yb and Yb is the fitted value from the first step) globally on covariates, X ~ genetic distance polynomial + sex + sex × genetic distance, where genetic distance polynomial is a 20-degree polynomial in genetic distance. Finally, we regress the residual of this second regression, Z = X - Xb onto the PGS in a simple (univariate) linear regression. We refer to the squared residual of this regression, (Z - Zb)^2, as the unstandardized squared prediction error. Similar to the group-level analysis, we computed the mean unstandardized squared prediction error in the 25 reference bins as a baseline value (Supplementary Table 1 detail the baseline values across traits). The squared prediction error, the measure of individual-level prediction accuracy we refer to throughout, is the unstandardized squared prediction error divided by the baseline value.

Spline fits. For both the individual-level and group-level analysis, Fig. 2 and Supplementary Figs. 2–9 show cubic spline fits. We fitted these splines using 8 knots with bs() from the R library splines49. The knot positions were chosen based on the density of the individual genetic distances, such that there is an equal number of samples between any two knots. This resulted in knots at genetic distances of 2.39, 3.54, 5.99, 11.20, 23.75, 45.40, 65.49, and 167.53.

Mean trends in individual-level prediction accuracy. In the Results section of the main text, we discuss various individual predictors of squared prediction error. In addition to genetic distance, we considered 2 measures of individual-level prediction error: The Townsend Deprivation Index31 (data field 189) and average yearly total household income before tax (data field 738). For genetic distance and the Townsend Deprivation Index, we considered five uniformly-spaced bins, and computed the mean squared prediction error and the standard error of this mean (Fig. 3A, and Supplementary Figs. 14–17). For household income, which the UKB provides as categorical data conferring to ranges in British Pounds (£), we converted the categories into an ordinal variable coded as 1, 2, 3, 4, and 5, and computed the mean squared prediction error, and standard error of the mean in each. This also allowed us to use the income categories directly as measures in the regression models used for comparison of variance in prediction error explained that we discuss below.

Comparison of variance in prediction accuracy explained across measures. For this analysis, we used all the individuals in the prediction set and did not filter for the individuals with a genetic distance greater than 97.5th percentile of genetic distance from among GWAS sample individuals. We compared the variance in squared prediction error explained for 8 raw measures: genetic distance, Townsend Deprivation Index31 (data field 189), average yearly total household income before tax (data field 738), educational attainment (data field 6138; which we converted into years of education according to Table S1 in Carter et al. 50), minor allele counts for SNPs with different magnitudes of effects (three equally-sized bins of small, medium, and large squared effect sizes, see Supplementary Fig. 23), and minor allele counts of all SNPs. Minor here is with respect to the GWAS sample, and the count is the total sum of minor alleles across index SNPs of the magnitude category. Namely, for each measure, we independently fit three different models: (1) a linear predictor, fitted using ordinary least squares; (2) a discretized predictor, using one predicted value for each of the 5 bins where all five bins had identical widths; and (3) a cubic spline, with 16 knots placed based on the density of data points at the individual level, such that there was an equal number of data points between each pair of consecutive knots. After fitting the models, we calculated the R2 values to determine the variance explained by each measure-method combination. We then computed 95% central confidence intervals for these R2 values to assess the reliability of the estimates (Fig. 3B, and Supplementary Figs. 18–21).

Additional analyses on lymphocyte count

Comparing allelic estimates across three GWASs. To test whether allelic effect estimates are similar across genetic ancestry, we performed two additional GWASs for each trait in two subsets of the prediction sample: close group (genetic distance ≤10, with 96,457 individuals) and far (genetic distance >10, with 32,822 individuals). For both groups, we adjusted for 20 PCs of the genotype matrix of the respective set of individuals, using the –pca approx 20 flag in PLINK 2.044,45. After running GWAS independently in the two groups, for each index SNP of the original PGS, we divided allelic effect estimates in the original GWAS, close, and far set by the allelic effect estimate in the original GWAS. Figure 4A shows the mean ± standard deviation across PGS index SNPs for each of three traits.

Heterozygosity at index SNPs as a function of genetic distance. For each PGS, we calculated the heterozygosity of each index SNP in each bin from allele counts using the –freq flag from PLINK 1.945,48. We stratified index SNPs into three equally-sized bins based on their squared effect sizes (Supplementary Fig. 23). Figure 4B and Supplementary Fig. 22 show the mean heterozygosity (across stratum SNPs) for each stratum in each genetic distance bin.

Variance of PGS as a function of genetic distance. For each phenotype, we calculated the variance of PGS in each bin relative to the mean of the variance of PGS in the 25 bins close to the GWAS set. In Fig. 4C, we plotted the values in each bin as well as a linear fit for lymphocyte count. For other traits, we only plotted the linear fit.

Heritability associated with each index SNP. We estimated the heritability explained by each index SNP as

2p(1-p)beta^2,

where beta is the estimated allelic effect and p is the allele frequency. In Supplementary Fig. 24, we compared the distribution of index SNP heritability across traits and with allelic effect estimates and heterozygosities calculated both in the original GWAS sample, the close prediction sample and the far prediction sample. For each trait, the SNPs used are also stratified into three equal-sized strata (small, medium, and large) based on their squared effect sizes, as discussed above.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Wang and colleagues use the UK Biobank to highlight what they call three open questions that remain to be understood about the (poor) portability of polygenic scores (PGS). They do not provide answers to any of these three remaining questions and leave this for future work. I am not sure what to get from this paper. I also have concerns about the validity of some of the analyses.\nMain comments:\n- “Previous reports suggested that the relationship between genetic distance and prediction accuracy is similar across traits [17,27,7]”. I don’t think this is true; these papers present the trend averaged across traits, but also acknowledge differences across traits (e.g. Fig 4 of Ref 7 and Fig 2 of Ref 27).\n- At least one of the open questions that the authors mention is why is there some PGS portability issue and why is it different across traits ? The consensus on the main reason behind the poor portability of PGS is that we use tagging variants in the PGS (instead of causal variants). These tagging variants then can have different allele frequencies, and especially different LD with the causal variants across ancestries, which can make them bad proxies of causal variants in other ancestries, resulting in a decrease in PGS performance. I don’t think this is mentioned in this paper; instead rather vague possible explanations are mentioned, which I think don’t help move the field forward.\n- I have many concerns on how PGS and genetic distances were derived in this paper.\n1/ PGS are derived using the C+T method with p<1e-5. This will likely use a very restricted number of variants, which could result in non-normal PGS distribution, which could cause R2 to be biased or at least highly variable. Also, I don’t understand how the authors can derive individual PGS intervals or accuracies when not using Bayesian sampling of causal effects (as done in Refs 6 & 7).\n2/ In Ref 27, genetic distance is directly computed from the (squared) Euclidean distance of PCs (which are already weighted by eigenvalues, so I do not think these should be weighted more).\n3/ Partial-R2 values between 0 and 4 are odd. Use the true definition of a partial correlation: the correlation between the PGS and the outcome, after they have been both residualized by covariates. You can use e.g. bigstatsr::pcor() in R. Why not use the deprivation index in the adjusted covariates? Also, I don’t think it is valid to compute an R2 in possibly very heterogeneous groups (the same bin on genetic distance can include very different genetic ancestries). If you want to use more groups than in Ref 27, have a look at the many homogeneous non-UK groups defined in https://doi.org/10.1093/bioinformatics/btac348.\n4/ Use only 16 PCs from the UK Biobank; PC19-40 capture only LD and should never be used (cf. https://doi.org/10.1093/bioinformatics/btaa520).\n5/ I don’t think there has been any filtering on relatedness in these analyses, which can bias predictions for related individuals.\nAdditional comments:\n- Not sure it is valid to compute Fst between one individual and a group.\n- Discussion materials should be more separated from the Results.\n- “We standardized genetic distance such that its mean is 1 across GWAS sample individuals.” -> what is the rationale behind this? Instead, you could scale using the max distance and max Fst, as done in Ref 27.", "reviewer_2": "Review for “Three Open Questions in Polygenic Score Portability”\nJoyce Y. Wang et al described three open questions for PGS analysis, including high variability, trait-specific portability and measurement impact. Overall, the study highlights that understanding PGS portability requires considering more than just global ancestry, including trait evolution, genetic architecture, social factors, and PGS construction. This approach could lead to more effective and equitable genomic research.\nThe study topics are very interesting. Manuscripts are well-written with clear illustration. Detailed major comments can be found in the following section.\nMajor:\n1. A “table 1” should be included to describing the distribution of data set. Since authors examined variables like Townsend Deprivation Index, Household Income etc, it would be necessary to compare between GWAS sample and PGS sample to make sure there is no significant differences between the group. Besides variables used in the analysis, some other variables should be included in table 1 include age, sex, location (England, Scotland or Wales).\n2. Line 99-100 & Section Prediction accuracy is weakly predicted by genetic distance & Figure 2: Authors split samples into a bin of approximately 260 individuals and compared the prediction accuracy against the genetic distance from the GWAS sample. (1) In Figure 2B regarding weight, the smoothed curve around 100 on x-axis seems to be influenced by a few points on the top. Some further analysis could be done to rule out the randomness of the sample split. For example, would that be possible to vary the number of bins (currently 500) to see if the current trend would maintain. (2) Additionally, currently analysis uses cubic spline fits with 8 knots to capture the trajectory of the trend. It would be helpful to fit some models, like GAM (generalized additive model) to show if the trend is significant or not. And adding the CI (confidence interval) around the smoothed line. (3) Figure 2 500 bins plot with individual ones are not quite corresponding to each other. The overall trend between the 2 are not consistent. For example, E and F, the whole trajectory in E is below 1. However, in F the line is above 1.\n3. Line 183-189 in Discussions: Authors observed some trait-specific trends in portability. It would be great to have some additional analysis to examine what groups of traits might need additional inspections. Expanding on the potential implications of these findings for equitable genomic research could strengthen the conclusion.\n4. Line 287-297: Authors detailed described how GWAS are constructed within UKB. (1) Did authors also take sample relatedness into consideration? Either model the sample relatedness or excluded those individuals that are closely related. These individuals might influence GWAS performance. Some more recent software, like REGENIE would be a better approach for modeling GWAS. (2) UKB data is collected at 3 regions, ie England, Scotland and Wales, in multiple centers in different batch. It would be great to also control assessment center (field id 54) and genotype array (field 22000) in the analysis.\n5. Line 298-300: Authors use clumping and thresholding method to construct PGS model. Would that be possible to utilize other methods, like LDpred2 and PRSCS to examine the performance of PGS? Literatures have showed that these methods show a better performance comparing to clumping and thresholding strategy.\nMinor:\n1. Figure 1C: would that be possible to have a similar histogram for UKB samples?\n2. It would be interesting to examine relationship between genetic PCs and Fst estimated in the study.\n3. Line 171-176 & Discussions: It would be great to examine some admixed ancestry population performance considering the mosaic genetic structure. Fst is still a measure of genetic distance in a global level.\n4. Line 340: field should be 22189 instead of 189.\nIn all, the manuscript described important questions in PRS analysis that GWAS summary statistics have different characteristics, especially ancestry with individual level data. While the manuscript is well written and ideas are new, some detailed information need to be refined.", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_4": "Wang et al. highlight three gaps for PGS portability with comprehensive analysis in the UKB. But I still have some concerns.\n1. You only used 15 highly heritable continuous traits to study PGS's portability. To my knowledge, the selected traits are polygenic. I assume that the genetic architecture may be a contributing factor to PGS portability. I would like you to add the binary traits, such as type II diabetes (T2D) and Alzheimer’s disease (AD). For example, the genetic architecture of AD might be a mixture of polygenic and sparse. Both the difference between continuous trait and binary trait and the difference between genetic architecture might result in different trends between PRS and genetic distance.\n2. You only used a simple method to construct PGS, which might result in a low prediction performance. I recommend incorporating a PGS method into the sensitivity analysis. Based on your pipeline, you can use polygenic methods without a validation set, such as SBLUP and DSBLMM. It's intriguing to see if a better PGS yields a different trend for the same trait.\n3. Based on the partial R2 equation, I assume that the value should be lower than 1. I'm confused as to why some groups' partial R2 of weight, BMI, and body fat percentage is greater than 1. I also suggest that you provide the raw R2 to demonstrate PGS's performance.\n4. The variance of PGS for WBC, lymphocyte count, and monocyte count increases with genetic distance. Please conduct further analysis to delve deeper into the trend, particularly examining the specific genetic architecture of immunity-related traits. If you add asthma, an immunity-related trait, the trend should be similar.\n5. You investigate the heterozygosity and SNP effect size in different traits. I assume that the LD structure is also a factor in PGS portability. Please perform additional analysis of LD structure, such as LD score."}, "author_response": "Wang et al. - Response to Reviews\nWe thank the four reviewers for their thoughtful assessment and comments. We appreciate the opportunity to improve our work through revision and resubmission.\nThe reviews clarified two main goalposts for our revision. First, the reviewers had various methodological concerns and questions about the sensitivity of our conclusions to analyses choices. Through a wide range of new analyses, we believe we have addressed these concerns thoroughly. For example, we now show that our conclusions about trait-specificity and measure-specificity extend to the portability of PGS for disease risk prediction. We also show that our main conclusions are robust and insensitive to choices of GWAS covariates and methodology, adjustments for confounding, polygenic scoring methods, measure of genetic distance, and fitting and visualization of continuous functions to describe portability trends.\nIn addition, one of the reviews conveyed doubts about the significance of our findings. We would like to make the case for why we see our results as substantially advancing the literature. We highlight empirical examples of the drivers and nature of the polygenic score portability problem. This includes crucial conclusions that may be intuitive, but are obscured or severely underappreciated in the current literature.\nBelow, we discuss how our three main conclusions innovate with respect to the current literature:\n(1) Genetic ancestry only explains a negligible proportion of the variation in predictive performance among individuals. Work such as Ding et al., Nature 2023 also refers to a “prediction accuracy” at the individual level, and gives the strong impression (e.g., Figures 1d,2c from Ding et al., copied below) that genetic ancestry nearly fully accounts for the variable performance of polygenic scores. (We discuss in our response and in the manuscript (lines 58-66) why we do not consider their statistic as in fact measuring prediction accuracy.)\nTaken from figure 1 in Ding et al., Nature 2023\nOur paper (lines 16-18, 104-124) shows that for a measure of realized prediction accuracy, the opposite is true: Genetic ancestry explains a small fraction at the individual level (e.g., Fig. 2B,D,F); and that SES measures explain a comparable (across traits, slightly larger) fraction (Fig. 3B).\nFig. 2: Righthand panels show that genetic distance explains little of the variation in Individual-level prediction accuracy\n(2) Trends of portability vary across traits. While some previous work contain figures where trait-specificity can be observed, we discuss it explicitly for body fat-related polygenic scores and for immune-related polygenic scores. We dove deeper into the latter as a case study, showing how natural selection and ascertainment biases can combine to drive trait-specific trends (Figs. 4, S22-24; lines 133-162; more discussion of implications in lines 199-208).\n(3) Even qualitative trends of portability can depend on the measure of prediction accuracy used.\nTo our knowledge, ours is the first work to demonstrate that the choice of measure can entirely change the conclusion about PGS portability and implications for their application. To our previous main text case study of white blood cell count, we have now extended this conclusion to risk prediction, showing that trends look qualitatively different for different measures of type 2 diabetes risk predictive performance—recall and precision (lines 163-182; more discussion of implications in lines 209-219).\nPoint-by-Point Response\nReviewers' comments:\nReviewer #1 (Remarks to the Author):\nWang and colleagues use the UK Biobank to highlight what they call three open questions that remain to be understood about the (poor) portability of polygenic scores (PGS). They do not provide answers to any of these three remaining questions and leave this for future work. I am not sure what to get from this paper. I also have concerns about the validity of some of the analyses.\nBased on our read of the literature, the three problems are largely missed in the current literature. The impact of factors other than global ancestry on portability, the sensitivity to the measure of portability, and trait specificity are not understood and underappreciated, despite results from Privé et al., AJHG 2022, Mostafavi*, Harpak*, eLife et al. 2020 and Hou et al., Nature Genetics 2024 all being highly cited.\nThis is in contrast to the general impacts of genetic ancestry on portability (LD differences, e.g. Privé et al., Wang et al., Nature Communications 2020, Yair and Coop, Royal Society B 2023; and allelic turnover, e.g. Carlson et. al., PLoS Genetics 2022) which are widely-appreciated and heavily studied. We discuss this concern further in the general introduction to this Response to Reviews, pages 1-4\nAs for concerns about the validity of analyses, we believe our response below and extensive additional analyses address all outstanding reviewer comments.\nMain comments:\n- “Previous reports suggested that the relationship between genetic distance and prediction accuracy is similar across traits [17,27,7]”. I don’t think this is true; these papers present the trend averaged across traits, but also acknowledge differences across traits (e.g. Fig 4 of Ref 7 and Fig 2 of Ref 27).\nOne might expect a qualitatively similar, monotonic relationship between genetic distance and prediction accuracy across traits. Previous analyses (that have not examined individual level prediction accuracy) observed similarly monotonic (Martin et al., 2019), and even linear (Privé et al., 2022, Ding et al., 2023), relationship regardless of the trait examined. However, we observed variation in this relationship among traits. Unlike the case of height, the prediction accuracy for many other traits did not decay monotonically with genetic distance. Weight and body fat percentage peaked in accuracy at intermediate genetic distances.\n- At least one of the open questions that the authors mention is why is there some PGS portability issue and why is it different across traits ? The consensus on the main reason behind the poor portability of PGS is that we use tagging variants in the PGS (instead of causal variants). These tagging variants then can have different allele frequencies, and especially different LD with the causal variants across ancestries, which can make them bad proxies of causal variants in other ancestries, resulting in a decrease in PGS performance. I don’t think this is mentioned in this paper; instead rather vague possible explanations are mentioned, which I think don’t help move the field forward.\nThe previous results on (population genetic) theory-based expectations for portability are discussed explicitly and referenced thoroughly. For example, lines 37-43 states:\nThis so-called “portability'” problem is a subject of intense study. Typically, portability is evaluated through variation in the within-group phenotypic variance explained by a PGS (i.e., the coefficient of determination, R2) among genetic ancestry groups. Indeed, population genetics theory gives clear predictions for the relationship between genetic dissimilarity to the GWAS sample and PGS prediction accuracy under some models (neutral evolution (Privé et al. 2022, Yair et al. 2022, Carlson et al. 2022), directional (Patel et al. 2024), or stabilizing selection (Yair et al. 2022, Patel et al. 2024), all else being equal (including, e.g., assumptions about environmental effects).\nBy and large, existing models all predict a monotonic decrease of prediction accuracy with genetic distance. They predict or assume the majority of variation in prediction accuracy will be explained by genetic distance.\nWe are indeed challenging this so-called “consensus” in this manuscript. While we do not doubt linkage disequilibrium differences between the GWAS sample and the prediction sample are a major factor underlying our and previous observations, we highlight underappreciated factors that add to and interact with ancestry differences. For example: the statistical problems with focusing on R2 measures, the fact that with some measures prediction accuracy do not even decay with genetic distance, trait-specificity relating to genetic architecture and portability of causal allelic effects.\n5\n- I have many concerns on how PGS and genetic distances were derived in this paper.\n1/ PGS are derived using the C+T method with p<1e-5. This will likely use a very restricted number of variants, which could result in non-normal PGS distribution, which could cause R2 to be biased or at least highly variable.\nThe majority of analyses are about individual-level portability, as we think the view through R2 can conflate various confounded explanations (lines 44-65).\nThat said, even in the analysis of R2 trend there is no assumption that the dependent variable is Normally distributed. We are therefore unclear about the claim above about bias. We are similarly unsure why a large variance of R2 should take away from our conclusions.\nWe also note the manuscript is concerned with realized prediction accuracy of PGS as used in practice. It is not concerned with constructing optimal PGS, measuring or improving common practices in polygenic scoring.\nNevertheless, we hope our extensive sensitivity analyses in response to reviewers’ comments show that our conclusions are insensitive to many choices in the GWAS (lines 734-752; Figs. S50-S57) and construction of PGS (lines 753-765; Figs. S58-S61) or analysis (lines 761-783; Figs. S38-S49; S62-S65).\nAlso, I don’t understand how the authors can derive individual PGS intervals or accuracies when not using Bayesian sampling of causal effects (as done in Refs 6 & 7).\nWe measure prediction accuracy with respect to the prediction of trait value. For continuous traits and the individual level, this is the squared difference between the predicted trait value and the true trait value. (We note that in order to not have the analysis affected by how well covariates other than the PGS predict the phenotype, we first regress out covariates; lines 96-103; 349-363).\nWhile Ding et al. also refer to “prediction accuracy,” their statistic in fact does not examine the true phenotype value at all, and therefore should not be thought of as the accuracy in prediction of the realized phenotype. Instead, Ding et al.’s statistic is a prediction interval, or its length, i.e. expected uncertainty in prediction under an assumed model. Importantly, it is calculated based on the independent variable (genotype) alone and does not take the dependent variable (phenotype) as input (lines 56-65).\n2/ In Ref 27, genetic distance is directly computed from the (squared) Euclidean distance of PCs (which are already weighted by eigenvalues, so I do not think these should be weighted more).\nWe do not weigh it twice.\n3/ Partial-R2 values between 0 and 4 are odd. Use the true definition of a partial correlation: the correlation between the PGS and the outcome, after they have been both residualized by covariates. You can use e.g. bigstatsr::pcor() in R. Why not use the deprivation index in the adjusted covariates?\nRegretfully, we previously only explained this visualization choice in the Methods section and understand this caused confusion. We now expand on it in the caption of Figure 2 and have edited the figure to call out the fact that relative prediction accuracies are shown.\nOur manuscript focuses on individual level prediction. We only use R2 to discuss the correspondence, and sometimes inconsistency, with assumptions in previous literature. However, computing R2 is valid regardless of how heterogeneous the group is. We agree that the inference based on the comparison of R2 between groups can be misleading—with one of the reasons being heterogeneity in genetic variance among groups. This reason is discussed in the introduction, in lines 108-110 of the Results section and supported by Fig. S22.\n8\n4/ Use only 16 PCs from the UK Biobank; PC19-40 capture only LD and should never be used (cf. https://doi.org/10.1093/bioinformatics/btaa520).\nTo address this concern, we recalculated of genetic distance, now defining it using the first 16 rather than 40 PCs. None of the portability trends and our conclusions based on them changed (lines 712-717; lines 761-773; Figs. S62-S65). As an example, we show below a comparison of both individual- and group-level results for red blood cell-related traits with the two measures of genetic distance.\n9\n5/ I don’t think there has been any filtering on relatedness in these analyses, which can bias predictions for related individuals.\nThanks to reviewers’ attention on this topic, we discovered a mistake in our code resulting in relatives not been being properly excluded from analysis. We fixed the mistake and close relatives are removed throughout now. In particular, we filtered by conditioning on individuals that were used in the PCA calculation of the UKB (i.e., relying on the original filtering of individuals with <=3rd degree relatedness, performed by the UKB; lines 241-242). As a result, the composition of the GWAS sample and prediction sample and downstream analyses (effectively all analyses) have been updated. Importantly, our conclusions remain the same.\nAdditional comments:\n- Not sure it is valid to compute Fst between one individual and a group.\nIt is valid. Fst is defined with respect to two subsets of chromosomes (from a larger set). Here, one of the subsets is composed of only two chromosomes.\n- Discussion materials should be more separated from the Results.\nIn a paper such as this one, aiming to highlight underappreciated facets of a problem, we view signposting, including minimal interpretation of results that comes alongside their reporting, as helpful to understanding and evaluating our findings.\n- “We standardized genetic distance such that its mean is 1 across GWAS sample individuals.” -> what is the rationale behind this? Instead, you could scale using the max distance and max Fst, as done in Ref 27.\nSigned Florian Privé\nReviewer #2 (Remarks to the Author):\nReview for “Three Open Questions in Polygenic Score Portability”\nJoyce Y. Wang et al described three open questions for PGS analysis, including high variability, trait-specific portability and measurement impact. Overall, the study highlights that understanding PGS portability requires considering more than just global ancestry, including trait evolution, genetic architecture, social factors, and PGS construction. This approach could lead to more effective and equitable genomic research.\nThe study topics are very interesting. Manuscripts are well-written with clear illustration. Detailed major comments can be found in the following section.\nWe thank the reviewer for their thoughtful evaluation and comments.\nMajor:\n1. A “table 1” should be included to describing the distribution of data set. Since authors examined variables like Townsend Deprivation Index, Household Income etc., it would be necessary to compare between GWAS sample and PGS sample to make sure there is no significant differences between the group. Besides variables used in the analysis, some other variables should be included in table 1 include age, sex, location (England, Scotland or Wales).\nWe have added detailed analyses of the distributions of age, sex and location in the GWAS and prediction samples (lines 695-711; Figs. S25-S29).\nThe fact that the distribution of age, sex, location and other environmental and social factors may differ between the GWAS sample and the prediction sample is indeed what we want to highlight here. As shown in e.g. Mostafavi et al., eLife 2020 and Hou et al., Nature Genetics 2024, age, sex and SES differences can greatly impact prediction accuracy even in the absence of major genetic ancestry differences. Here, we wish to highlight that the two groups of factors are also often confounded.\n12\n2. Line 99-100 & Section Prediction accuracy is weakly predicted by genetic distance & Figure 2: Authors split samples into a bin of approximately 260 individuals and compared the prediction accuracy against the genetic distance from the GWAS sample. (1) In Figure 2B regarding weight, the smoothed curve around 100 on x-axis seems to be influenced by a few points on the top. Some further analysis could be done to rule out the randomness of the sample split. For example, would that be possible to vary the number of bins (currently 500) to see if the current trend would maintain.\nTo examine the sensitivity of our analyses to binning parameter choices, we re-performed the analysis of portability trends with a different partitioning: we doubled the number of (half-size) bins. Namely, we binned the individuals in the prediction set into 500 bins of 139 individuals each, instead of 250 bins of 278 individuals each (the partitioning which is now used in the main text analyses). In Figs. S38-S41, we show relative prediction accuracy trends using the mean prediction accuracy in 50 bins with median genetic distances that are closest to the mean genetic distance for GWAS individuals as the baseline value.\nComparing the two binning choices, we did not notice any substantial change in the portability trends for any of the quantitative traits analyzed (including weight—see panels C,D in Fig. S38 and attached below; lines 721-728).\n13\n(2) Additionally, currently analysis uses cubic spline fits with 8 knots to capture the trajectory of the trend. It would be helpful to fit some models, like GAM (generalized additive model) to show if the trend is significant or not. And adding the CI (confidence interval) around the smoothed line.\nTo examine whether our conclusions are sensitive to continuous function fits we applied for visualizations, we fitted a generalized additive model (GAM) instead of a cubic spline The GAM fits appear to be smoother at the group level for most traits, but have the same general trends as the cubic spline fit (Figs. S42-S45; lines 779-783).\nTo address the concern about the number of knots used in the spline, we have also reperformed the analyses using cubic splines with 12 instead of 8 knots. Here too, we found no qualitative changed to the trends we observed previously and discuss in the text (Figs. S38-S41; lines 774-778).\n14\n(3) Figure 2 500 bins plot with individual ones are not quite corresponding to each other. The overall trend between the 2 are not consistent. For example, E and F, the whole trajectory in E is below 1. However, in F the line is above 1.\nIndeed, one of our main findings is that portability trends depend on the measure of prediction accuracy. In Fig. 2E,F, this is exemplified with a measure of group-level prediction accuracy compared with a measure of individual-level prediction accuracy. See specifically the section titled “The measure of predictive performance can alter our view of portability” in lines 163-167. In the new Figure 5b, we can see the same can happen for different measures of type 2 diabetes risk prediction accuracy, recall and precision.\n15\n3. Line 183-189 in Discussions: Authors observed some trait-specific trends in portability. It would be great to have some additional analysis to examine what groups of traits might need additional inspections. Expanding on the potential implications of these findings for equitable genomic research could strengthen the conclusion.\nWe have exemplified trait-specificity for various traits, including body fat related traits (lines 129-132), immune related traits (lines 133-162) and type 2 diabetes (168-182).\nThe unusual patterns observed for immune-related traits include the rapid decrease in prediction accuracy even at short genetic distances. We propose a testable hypothesis:\n1. When selection pressures vary across time/geography, such as would be expected for the immune system, there will be a more rapid turnover of the loci contributing to trait variation. Genetic effects themselves become less portable. Indeed, when comparing to other traits (including a trait with similar SNP heritability), we find evidence that genetic effects on lymphocyte count are less portable.\n2. The turnover in genetic basis across populations can interact with statistical biases, resulting in another unusual pattern: the frequencies of the largest effect index SNPs are increasing with genetic distance.\n3. Since these alleles make for a predictor that is highly variable away from the GWAS sample (due to (2)) but quickly becomes less predictive (due to (1)), prediction accuracy decays quickly.\nFollowing the reviewer’s suggestion, we expanded the discussion of potential implications in the Discussion section (lines 199-208). We do not want to frame our conclusions as prescriptive in terms of increasing equity in genomic research. However, we strongly agree that understanding the importance of social, environmental and evolutionary context for PGS portability is crucial for equitable genomic research. This is a central motivation for this work (lines 11-13; 28-29; 34-36; 218-219).\n4. Line 287-297: Authors detailed described how GWAS are constructed within UKB. (1) Did authors also take sample relatedness into consideration? Either model the sample relatedness or excluded those individuals that are closely related. These individuals might influence GWAS performance. Some more recent software, like REGENIE would be a better approach for modeling GWAS.\nThanks to reviewers 1 & 2 comment and attention here, we discovered that, due to an error in our code, relatives were not properly removed. We fixed the bug and relatives are now removed. In particular, we kept just one of each set of individuals with <=3rd degree relatedness (lines 241-242). As a result, the composition of the GWAS sample and prediction sample and downstream analyses (effectively all analyses) have been updated. Nevertheless, in the vast majority of analyses, our conclusions remain the same.\nIn addition, to test the sensitivity of our conclusions to the GWAS method we used, in particular the adjustment for population structure using PCs, we reanalyzed portability trends for PGS constructed\n16\nbased on GWAS performed with REGENIE. REGENIE adjusts for population structure using a linear mixed model approach. Other research from us (Smith et al. 2025, “A litmus test for confounding in polygenic scores”) and others suggests that this type of adjustment better mitigates some types of confounding, such as confounding due to finer-scale population structure.\nPrediction accuracies were generally higher throughout in REGENIE GWAS-based polygenic scores. In a couple of the lower heritability traits (lymphocyte count and monocyte count), portability trends became noisier. Beyond these differences, trends remained the same (lines 740-747; Figs. S54-S57).\n(2) UKB data is collected at 3 regions, ie England, Scotland and Wales, in multiple centers in different batch. It would be great to also control assessment center (field id 54) and genotype array (field 22000) in the analysis.\nPer the reviewer’s concern, we re-performed the analysis of portability trends across the 15 quantitative traits using genotype array and participant country of residence as additional covariates in the GWAS. We find that this results in lower PGS prediction accuracy for lymphocyte and monocyte counts, but makes no difference to the qualitative portability trends reported throughout (lines 729-739; Figs. S50-S53).\n5. Line 298-300: Authors use clumping and thresholding method to construct PGS model. Would that be possible to utilize other methods, like LDpred2 and PRSCS to examine the performance of PGS? Literatures have showed that these methods show a better performance comparing to clumping and thresholding strategy.\nWe performed an analysis to test sensitivity to the choice to analyze PGS constructed using clumping and thresholding by replacing it with PRS-CS (Ge et al., Nature Communications 2019). PRS-CS applies Bayesian shrinkage to GWAS-based allelic effect estimates. It also aims to consider LD patterns more rigorously rather than via the clumping and thresholding heuristic.\nWe compared our results for the 15 clumping and thresholding based polygenic scores for quantitative traits analyzed in the main text. For weight and BMI at the group level, the peak at intermediate genetic distances became less prominent for polygenic scores constructed with PRS-CS. For lymphocyte count and white blood cell count, the decrease in prediction at the group level became more gradual. However, the general trends of portability of all other traits did not change (lines 748-760; Figs. S58-S61).\n1. Figure 1C: would that be possible to have a similar histogram for UKB samples?\nThe samples used to produce the histograms in Fig. 1C are UKB samples.\n19\n2. It would be interesting to examine relationship between genetic PCs and Fst estimated in the study.\nThe relationship between Fst and genetic distance is shown in Fig. 1B. We added a supplementary analysis of the relationship between Fst and individual genetic PCs (lines 712-717; Figs. S30-S33)\n3. Line 171-176 & Discussions: It would be great to examine some admixed ancestry population performance considering the mosaic genetic structure. Fst is still a measure of genetic distance in a global level.\nWe agree and are similarly interested in examining whether measures of local genetic ancestry can help explain portability trends. However, we view the hefty analysis required to ask this question as beyond the already substantial scope of this manuscript. We discuss this as a fruitful area for future research in lines 187-195.\n20\n4. Line 340: field should be 22189 instead of 189.\nFields 22189 and 189 are both Townsend index. 189 is what we used it. Examining the UKB showcase, both data fields have the same distribution. The only difference appears to be that 189 has more decimal places.\nIn all, the manuscript described important questions in PRS analysis that GWAS summary statistics have different characteristics, especially ancestry with individual level data. While the manuscript is well written and ideas are new, some detailed information need to be refined.\nWe thank the reviewer for their helpful comments and kind words about the manuscript.\nReviewer #3 (Remarks to the Author):\nI co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.\nReviewer #3 (Remarks on code availability):\nScripts are well-written and easy to replicate.\nWe thank the reviewer for their kind note regarding replicability.\nReviewer #4 (Remarks to the Author):\nWang et al. highlight three gaps for PGS portability with comprehensive analysis in the UKB. But I still have some concerns.\n1. You only used 15 highly heritable continuous traits to study PGS's portability. To my knowledge, the selected traits are polygenic. I assume that the genetic architecture may be a contributing factor to PGS portability. I would like you to add the binary traits, such as type II diabetes (T2D) and Alzheimer’s disease (AD). For example, the genetic architecture of AD might be a mixture of polygenic and sparse. Both the difference between continuous trait and binary trait and the difference between genetic architecture might result in different trends between PRS and genetic distance.\nThere are some clear theoretical expectations for continuous traits under simplified models. However, to our knowledge, no clear predictions exist for binary disease traits. At the same time, we agree it is of interest to examine whether our conclusions hold for disease traits.\nFor asthma, we saw that precision and accuracy depended on genetic distance in a qualitatively similar way. For type 2 diabetes, precision appeared roughly constant for medium and large genetic distances, while recall generally increased with distance far from the GWAS sample (lines 168-182; Fig. 5). Therefore, predictive performance shows trait-specific, measure-specific trends in disease risk prediction as well.\nWe note that we chose these diseases since they are among the very few that had a sufficient number of cases for the analysis to be interpretable. For example, for Alzheimer Disease, which was requested by the reviewer, we had to increase the size of each genetic ancestry bins substantially such that each bin would contain positive cases; yet that led to a small number of bins overall—each capturing a wide range of genetic distances—making the trends across bins uninterpretable.\n2. You only used a simple method to construct PGS, which might result in a low prediction performance. I recommend incorporating a PGS method into the sensitivity analysis. Based on your pipeline, you can use polygenic methods without a validation set, such as SBLUP and DSBLMM. It's intriguing to see if a better PGS yields a different trend for the same trait.\nOverall, our manuscript is not concerned with an optimal construction of polygenic scores. It discusses general trends of portability of (imperfect) polygenic scores. They are built in a manner largely reflective of common practice today, albeit not necessarily some gold standard. The explanation for the variable portability trends may very well partly relate to their suboptimal construction.\nThat said, it is important to show, as the reviewer alludes to here and elsewhere, that our qualitative conclusions are insensitive to specific choices we have made during the analysis.\nPer the reviewer’s suggestion, we re-analyzed portability trends after constructing PGS with another method that is in broad use in human genetics, PRS-CS (Ge et al., Nature Communications 2019; lines 748-760). PRS-CS applies Bayesian shrinkage to GWAS-based allelic effect estimates. It also aims to consider LD patterns more rigorously and holistically, rather than via the clumping and thresholding heuristic.\nWe first focused on a comparison with the results for the 15 clumping and thresholding based polygenic scores for quantitative traits analyzed in the main text. For weight and BMI at the group level, the peak at intermediate genetic distances became less prominent for polygenic scores constructed with PRS-CS. For lymphocyte count and white blood cell count, the decrease in prediction at the group level became more gradual. However, the general trends of portability of all other traits did not change (Figs. S58-S61).\n23\n3. Based on the partial R2 equation, I assume that the value should be lower than 1. I'm confused as to why some groups' partial R2 of weight, BMI, and body fat percentage is greater than 1. I also suggest that you provide the raw R2 to demonstrate PGS's performance.\nTo make the visualizations of portability trends comparable across traits, we do not show the raw partial R2 (at the group level) or the raw squared error (at the individual level) on the y-axis. Instead, we show these values divided by “baseline” values for each trait and measure. These baseline values (which are shown in Tables 1-2) correspond to the average prediction accuracy in prediction sample individuals that are most similar to the GWAS sample individuals, in terms of genetic distance from the GWAS sample centroid.\nWe apologize that this visualization choice was previously only explained in the Methods section. We understand this caused confusion. We now spell this out in the caption of Figure 2 and have edited the figure to call out the fact that relative prediction accuracies are shown.\n24\n25\n4. The variance of PGS for WBC, lymphocyte count, and monocyte count increases with genetic distance. Please conduct further analysis to delve deeper into the trend, particularly examining the specific genetic architecture of immunity-related traits. If you add asthma, an immunity-related trait, the trend should be similar.\nWe discuss the genetic architecture (joint distribution of allelic effects and allele frequencies), the differences in heterozygosity trends across different effect sizes and hypothesize how the increase with PGS variance may be driven by variable selection pressures on the immune system (lines 133-162; 199-208; Figs. 4, S22-S24,S66).\nPer the reviewer’s suggestion, we have examined the case for asthma. We found arguments in the literature supporting the hypothesis of substantial ancestry specificity of genetic associations with asthma (e.g., Washington III et al., Genome Medicine 2022). However, for asthma, this does not appear to be a strong enough effect to reverse the relationship between the variance of asthma PGS and genetic distance. Namely, it does not follow suit with that observed for white blood cell count or lymphocyte\n26\ncount. Instead, like the vast majority of quantitative traits and like the other disease we analyzed, type 2 diabetes, we see a monotonic decrease in genetic variance with genetic distance (Fig. S66).\n5. You investigate the heterozygosity and SNP effect size in different traits. I assume that the LD structure is also a factor in PGS portability. Please perform additional analysis of LD structure, such as LD score.\nWe apologize, but we were unsure what analysis the reviewer is asking to see and what question it would address.\nTo address the more general question, yes, absolutely: LD differences between the GWAS sample and prediction sample are plausibly among the most important factors leading to the decay of prediction accuracy with genetic distance. We discuss this fact throughout, including the previous theoretical and empirical support and their caveates (lines 37-43; 51-55) and consistency of this prediction with our observation (lines 104-115; 515-159; 187-195). In this manuscript, however, we show that summaries of genetic ancestry alone (including allele frequency turnover and LD differences) explains little of the variation in predictive performance at the individual level (Fig. 2B,D,F; Fig. S2-S9; Fig. 3B).\n27"}, {"round": 2, "reviewer_comments": {"reviewer_1": "I am still unconvinced by this paper and have several comments.\nMain comments\n- Gap 1 & 3: Results (trends of predictive performance vs genetic distance to training) using the prediction error at the individual level do not agree with the ones at the group level. This is very confusing (I guess this is the authors' point). The problem I see is that the squared difference in predicted vs true value depends heavily on things like phenotypic variance and also random shifts in PGS distribution (which are corrected for when using some proper partial correlation at the group level).\n- To get better quality results, the authors should present results\n+ with PRS-CS instead of C+T everywhere given the higher number of variants used with make the results less noisy (C+T could simply pick a few highly-transferable genetic variants by chance)\n+ with GAM instead of the cubic splines, which is much more robust and informative; you can also consider log-distances to have more evenly-spaced bins\n+ the authors should really consider log-transforming all the phenotypes they used (before residualizing) as they are probably all log-normally distributed (even height to a lesser extent -> just have a look at the distributions of males and females); this would make their result more robust to the long right tails of the phenotypic distributions.\n+ consider whether the individual-level predictions are just simply shifted due to differences in allele frequencies, which is corrected when adjusting PGS for PCs (cf. first main comment).\n- Gap 2: “portability trends (with respect to genetic distance) can be trait-specific” -> again, this has been shown in previous papers and I don't consider this as a knowledge gap. Moreover, I think the use of C+T with just a few variants in the PGS make them even more trait-specific simply by chance of picking +/- transferable genetic variants, which I consider misleading (therefore the recommendation of using a more genome-wide PGS method like PRS-CS).\nAdditional comments:\n- The figures in the response to reviewers were not of particularly good quality.\n- Ding et al. have looked at individual-level prediction accuracy; that was the whole point of the paper. They used some estimates, but still.\n- “The previous results on (population genetic) theory-based expectations for portability are discussed explicitly and referenced thoroughly.” -> which still says nothing on LD and AF differences..", "reviewer_2": "Joyce Y. Wang et al described three open questions for PGS analysis, including high variability, trait-specific portability and measurement impact. The manuscript had a great improvement since the last version. Majority of previous issues that were raised last time are properly addressed. Various sensitivity analyses validated the robustness of analysis conclusions. Here are some additional major and minor suggestions for authors to consider:\nSome major changes could be:\n1. Line 76, fit of GWAS and PRS analysis: authors subsetted 336923 individuals for GWAS analysis and applied to prediction sample for PRS. Notably, the GWAS was conducted exclusively in White British individuals, while the PRS was evaluated in a non-White British population. Given the known impact of population structure and genetic ancestry on PRS transferability, this discrepancy raises concerns about the validity of cross-ancestry comparisons. It would strengthen the analysis and interpretation if the authors could also include a subset of White British individuals in the PRS prediction phase. Comparing PRS performance between White British and non-White British individuals—using the same GWAS-derived weights—would provide a more rigorous assessment of ancestry-related biases and enhance the robustness of the conclusions. In that case, including the White-to-White group as a baseline would serve as a fundamental point of comparison.\n2. Currently, all analyses—including both GWAS and PRS—are conducted within the UK Biobank. To strengthen the generalizability of the findings, it would be valuable to assess external validity using independent cohorts. For example, applying GWAS results from an external dataset to train PRS and testing them in the UK Biobank, or vice versa, could help rule out potential UK Biobank-specific cohort effects and enhance the robustness of the conclusions. Some potential large biobank can be used include: All of Us etc.\nMinor issues:\n1. Line 76-78: please explain how the GWAS sample and prediction sample divided.\n2. Please keep tense consistency through the manuscript. Some places present tense (e.g. line 114) while some locations use past tense (e.g. line 113).\nIn summary, the manuscript shows significant improvement and effectively addresses most prior concerns. The analysis is robust, supported by thorough sensitivity checks. However, by addressing the major suggestions mentioned above, including a White British group in PRS evaluation, accounting for ancestry-socioeconomic confounding, and validating findings in external cohorts would further strengthen the study. Minor edits on sample description and tense consistency are also recommended.", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_4": "All of my concerns have been well addressed."}, "author_response": "Wang et al. - Response to 2nd Round Reviews\nWe thank all reviewers for their valuable comments and assessment of our work. We have revised our work further to address outstanding comments. Please find below a point-by-point response which also details our additional revisions.\nREVIEWER COMMENTS\nReviewer #1 (Remarks to the Author):\nI am still unconvinced by this paper and have several comments.\nWe regret not being able to meet the expectations of the reviewer. At the same time, we note that, as detailed in the previous Response to Reviews–and not contested by any of the reviewers–we have addressed each and every one of the concerns and requests for additional analysis raised in the previous round of review. Numerous sensitivity analyses performed in the previous round of revision, and additional ones carried out in this round, support the robustness of our observations and conclusions.\nMain comments\n- Gap 1 & 3: Results (trends of predictive performance vs genetic distance to training) using the prediction error at the individual level do not agree with the ones at the group level. This is very confusing (I guess this is the authors' point). The problem I see is that the squared difference in predicted vs true value depends heavily on things like phenotypic variance and also random shifts in PGS distribution (which are corrected for when using some proper partial correlation at the group level).\nThis are indeed central conclusions of our work, including both the improperness of R2 for most applications and its limited interpretability (lines 44-55; 164-168; 210-211); and the dependence on the measure of predictive performance, discussed in the Results subsection entitled “The measure of predictive performance can alter our view of portability.”, lines 164-183, and in the Discussion, lines 210-220. To our knowledge, ours is the first work to demonstrate that the choice of measure can entirely change the conclusion about PGS portability and implications for their application. In our previous revision, we extended this conclusion to risk prediction, showing that trends look qualitatively different for different measures of type 2 diabetes risk predictive performance—recall and precision (lines 164-183; Fig. 5).\n- To get better quality results, the authors should present results\n+ with PRS-CS instead of C+T everywhere given the higher number of variants used with make the results less noisy (C+T could simply pick a few highly-transferable genetic variants by chance)\nWe performed an analysis to test the sensitivity of our conclusions to the choice to analyze PGS constructed using clumping and thresholding and replacing it with PRS-CS (Ge et al., Nature Communications 2019). PRS-CS applies Bayesian shrinkage to GWAS-based allelic effect estimates. It also aims to consider LD patterns more rigorously rather than via the clumping and thresholding heuristic.\nWe find that, for weight and BMI, the peak observed in the group level analysis at intermediate genetic distances became less prominent for polygenic scores constructed with PRS-CS. For lymphocyte count and white blood cell count, the decrease in prediction at the group level became more gradual. However, the trends of portability of all other traits, and our qualitative conclusions reported in the main text, did not change (Figs. S58-S61).\nIn the examples (from Fig. S58) shown below, odd columns show the new sensitivity analysis with PRS-CS polygenic scoring; even column show main text choice of the GWAS and prediction samples.\n+ with GAM instead of the cubic splines, which is much more robust and informative; you can also consider log-distances to have more evenly spaced bins\nTo examine whether our conclusions are sensitive to continuous function fits we applied for visualizations, we fitted a generalized additive model (GAM) instead of a cubic spline fits. Trends remained qualitatively the same as with the cubic spline fit (Figs. S42-S45).\n+ the authors should really consider log-transforming all the phenotypes they used (before residualizing) as they are probably all log-normally distributed (even height to a lesser extent -> just have a look at the distributions of males and females); this would make their result more robust to the long right tails of the phenotypic distributions.\nWe appreciate the suggestion and view the question as an interesting theoretical investigation. However, it almost opposite to our empirical, applicative focus in this work. The sensitivity of group level performance metrics to the population and prediction sample distributions is the point, not an artifact. More broadly, our work is motivated and concerned with applicative performance, as opposed to optimizing the statistical capturing of variance in a theoretical construct that is more mathematically convenient. In central applications of polygenic scores, one typically aims to predict either the quantitative phenotype value in its given, defined units; or a categorical disease state. Prediction of some transformed phenotype, where the transformation depends on some reference sample, means missing this aim.\n+ consider whether the individual-level predictions are just simply shifted due to differences in allele frequencies, which is corrected when adjusting PGS for PCs (cf. first main comment).\nWe agree with this intuition, and this is indeed part of the rationale why we are indeed adjusting for genetic distance in our individual level. Genetic distance correlates strongly with Fst (Fig. S1) and with top PCs, predominantly with PC1 (Fig. S30-S33).\n- Gap 2: “portability trends (with respect to genetic distance) can be trait-specific” -> again, this has been shown in previous papers and I don't consider this as a knowledge gap. Moreover, I think the use of C+T with just a few variants in the PGS make them even more trait-specific simply by chance of picking +/- transferable genetic variants, which I consider misleading (therefore the recommendation of using a more genome-wide PGS method like PRS-CS).\nThis comment is addressed extensively in the previous Response to Reviews, revision and in the comment above.\nAdditional comments:\n- The figures in the response to reviewers were not of particularly good quality.\n- Ding et al. have looked at individual-level prediction accuracy; that was the whole point of the paper. They used some estimates, but still.\nWe are unsure what is meant by “they used some estimates, but still,” but we would like to clarify this is not a matter of more or less precise estimation of an estimand.\nIn this work, we measure prediction accuracy with respect to the prediction of trait value. For continuous traits and the individual level, this is the squared difference between the predicted trait value and the true trait value. While Ding et al. also use the same terminology of “prediction accuracy,” their “prediction accuracy” statistic in fact does not examine the true phenotype value at all. Indeed, it is not the accuracy in prediction of the realized phenotype. Instead, Ding et al.’s statistic is a prediction interval, or its length, i.e. expected uncertainty in prediction under an assumed model. Importantly, it is calculated based on the independent variable (genotype) alone and does not take the dependent variable (phenotype) as input (lines 56-65).\nWe note that our understanding of this different use of has been confirmed with the first and last (corresponding) authors Ding et al. through personal correspondence.\n- “The previous results on (population genetic) theory-based expectations for portability are discussed explicitly and referenced thoroughly.” -> which still says nothing on LD and AF differences..\nWith apologies, we do not understand this comment/question/concern. We would need for it to be clarified further in order to address it.\nReviewer #2 (Remarks to the Author):\nJoyce Y. Wang et al described three open questions for PGS analysis, including high variability, trait-specific portability and measurement impact. The manuscript had a great improvement since the last version. Majority of previous issues that were raised last time are properly addressed. Various sensitivity analyses validated the robustness of analysis conclusions. Here are some additional major and minor suggestions for authors to consider:\nSome major changes could be:\n1. Line 76, fit of GWAS and PRS analysis: authors subsetted 336923 individuals for GWAS analysis and applied to prediction sample for PRS. Notably, the GWAS was conducted exclusively in White British individuals, while the PRS was evaluated in a non-White British population. Given the known impact of population structure and genetic ancestry on PRS transferability, this discrepancy raises concerns about the validity of cross-ancestry comparisons. It would strengthen the analysis and interpretation if the authors could also include a subset of White British individuals in the PRS prediction phase. Comparing PRS performance between White British and non-White British individuals—using the same GWAS-derived weights—would provide a more rigorous assessment of ancestry-related biases and enhance the robustness of the conclusions. In that case, including the White-to-White group as a baseline would serve as a fundamental point of comparison.\nTo address the reviewer’s concern, we performed an additional sensitivity analysis, in which we considered a different choice of a GWAS sample and prediction sample, moving a subset of the set labeled by the UKB as White British into the prediction sample. In particular, the GWAS sample in this analysis included 300,000 randomly selected White British individuals and the remaining 36,923 White British individuals were included in the prediction sample (Text S3, lines 253-269). We find that among these individuals, whose genetic distance from the GWAS centroid is typically lower than 3 (Fig. S75), we find that group-level prediction accuracy remains approximately constant, as we have seen in the main text analyses (Figs. S76-S79). We note that the White British subset indeed spans a very narrow range of genetic distance (Fig. S71 and below).\nAs a result, we also find that qualitative trends of portability as a function of (the full range of) genetic distance remain qualitatively unchanged for all traits: See Figs S72-S75 and examples pasted below (odd columns show the new sensitivity analysis with a prediction sample that includes White British individuals; even column show main text choice of the GWAS and prediction samples).\n2. Currently, all analyses—including both GWAS and PRS—are conducted within the UK Biobank. To strengthen the generalizability of the findings, it would be valuable to assess external validity using independent cohorts. For example, applying GWAS results from an external dataset to train PRS and testing them in the UK Biobank, or vice versa, could help rule out potential UK Biobank-specific cohort effects and enhance the robustness of the conclusions. Some potential large biobank can be used include: All of Us etc.\nWe do not expect our results generalize in the sense that we will see the same trends of portability with ancestry across biobanks. This is precisely because of the sensitivities the reviewer alludes to. Namely, our main conclusions are about the low fraction of variation in realized prediction accuracy explained by ancestry (which will lead to weak replication of trends across studies in itself), sensitivity to social and environmental factors (which will differ greatly across countries and studies with different recruitment strategies), trait-specificity and measure-specificity (both varying across studies as well).\nWe sought to illustrate these sensitivities further and address the question raised by the reviewer–albeit through a different analysis than the one suggested by them. In particular, we expanded our analysis comparing the explanatory power of genetic distance to that of measures of socio-economic status. In the new analysis, we stratified our prediction sample by Townsend Deprivation Index, and investigated portability as a function of genetic distance within each stratum independently (Text S3, lines 241-252). Our results further illustrate that, across traits, the explanatory power of each of the factors (SES vs. genetic ancestry) is comparable (Fig. 3B) and that PGS prediction accuracy tends to be higher for more deprived individuals (Fig. 3A,C,S67-70).\nWe enthusiastically share the interest in learning about variation in portability through empirical, including cross-biobank comparisons of portability. Our lab is therefore investing considerable effort and funds in gaining access to multiple biobanks, and carrying out these analyses in datasets such as All of Us. However, primarily for the reason discussed above, we view these analyses as beyond the scope of this work–which already addresses many novel questions and supported by a diverse array of sensitivity analyses.\nMinor issues:\n1. Line 76-78: please explain how the GWAS sample and prediction sample divided.\nThis is discussed in further depth in the Data Overview subsection of the Methods, lines 233-250. We include the most relevant part below.\n2. Please keep tense consistency through the manuscript. Some places present tense (e.g. line 114) while some locations use past tense (e.g. line 113).\nThank you for pointing out this oversight. We have reviewed the manuscript and supplement and believe we have caught all previous misses (in particular, in the main text, this includes lines 75,98,114). Overall, we used past tense when describing our work and observations. We only use present tense when describing general truths. This follows from several style guides, including Council of Science Editors (“Describe your methods and results in the past tense, since they occurred. Use the present tense for conclusions and generally accepted facts”) and Nature Portfolio Author Guide “Use past tense for your experiments and results…, and present tense for general statements and interpretations.”\nIn summary, the manuscript shows significant improvement and effectively addresses most prior concerns. The analysis is robust, supported by thorough sensitivity checks. However, by addressing the major suggestions mentioned above, including a White British group in PRS evaluation, accounting for ancestry-socioeconomic confounding, and validating findings in external cohorts would further strengthen the study. Minor edits on sample description and tense consistency are also recommended.\nWe thank the reviewer for their thoughtful assessment of our work and for their helpful comments throughout."}]

Decision letter