Title: Multi-task learning with summary statistics
Venue: NeurIPS 2023
Accepted: yes
Keywords: multi-task learning

genetic risk prediction

summary statistics
Abstract: Multi-task learning has emerged as a powerful machine learning paradigm for integrating data from multiple sources, leveraging similarities between tasks to improve overall model performance. However, the application of multi-task learning to real-world settings is hindered by data-sharing constraints, especially in healthcare settings. To address this challenge, we propose a flexible multi-task learning framework utilizing summary statistics from various sources. Additionally, we present an adaptive parameter selection approach based on a variant of Lepski's method, allowing for data-driven tuning parameter selection when only summary statistics are accessible. Our systematic non-asymptotic analysis characterizes the performance of the proposed methods under various regimes of the source datasets' sample complexity and overlap.  We demonstrate our theoretical findings and the performance of the method  through extensive simulations. This work offers a more flexible tool for training related models across various domains, with  practical implications in genetic risk prediction and many other fields.

Multi-Task Learning with Summary Statistics

Abstract

Multi-task learning has emerged as a powerful machine learning paradigm for integrating data from multiple sources, leveraging similarities between tasks to improve overall model performance. However, the application of multi-task learning to real-world settings is hindered by data-sharing constraints, especially in healthcare settings. To address this challenge, we propose a flexible multi-task learning framework utilizing summary statistics from various sources. Additionally, we present an adaptive parameter selection approach based on a variant of Lepski's method, allowing for data-driven tuning parameter selection when only summary statistics are available. Our systematic non-asymptotic analysis characterizes the performance of the proposed methods under various regimes of the sample complexity and overlap. We demonstrate our theoretical findings and the performance of the method through extensive simulations. This work offers a more flexible tool for training related models across various domains, with practical implications in genetic risk prediction and many other fields.

1 Introduction

The growing availability of extensive and intricate datasets presents an opportunity to integrate data from multiple sources. Multi-task learning has emerged as a promising machine learning approach that enables the simultaneous learning of multiple related models, leveraging shared structure between tasks to enhance the performance on each task individually [36, 35]. In healthcare and biomedical research, the practical application of multi-task learning is often hindered by data-sharing constraints, which stem from concerns about the ownership and privacy of individual-level data [32, 8]. Patient data in these domains is typically sensitive and less likely to be publicly available or shared across study sites, limiting researchers' access to individual-level data from different domains.

To overcome this limitation, researchers have increasingly integrated summary statistics into analysis pipelines as a substitute for individual-level data [22, 13, 12]. Summary statistics are straightforward, interpretable measures derived from raw data that can offer insights into data distribution, variability, and relationships among variables. Furthermore, they can be aggregated across studies to facilitate data integration and reused in various research projects. Recently, the use of summary statistics has garnered interest in healthcare and biomedical research. For example, many genetic risk prediction methods rely on summary-level statistics such as associations from Genome-wide Association Studies (GWAS), Linkage Disequilibrium estimations (LD), and minor allele frequencies (MAFs) [6]. These summary statistics can help predict an individual's likelihood of developing specific diseases based on their genetic profile.

Inspired by a potential use case in genetic risk prediction, we propose a multi-task learning framework that enables simultaneous learning of multiple genetic risk prediction models using only publicly available summary statistics. Our proposed framework can be used in the context of predicting genetic risks for multiple traits leveraging potentially shared genetic pathways, and can also be used

to develop trans-ethnic genetic risk prediction models that account for potential heterogeneity across populations, improving generalizability and real-world applicability. Beyond genetic risk prediction, the ability to learn from summary statistics offers a versatile tool for developing models across a wide range of domains, including healthcare, finance, and marketing.

To summarize, the contributions of this work are threefold: First, we propose a flexible multi-task learning framework which allows training multiple models simultaneously using basic summary statistics characterizing marginal relationship between outcomes and features, which are often publicly available. We allow summary statistics corresponding to each task to be generated from distinct or potentially overlapping samples. Secondly, we conducted a systematic non-asymptotic analysis which characterizes how the performance of the proposed methods are influenced by the characteristics of summary statistics. In particular, we show that there are multiple regimes of performance depending on the sample complexity of the source datasets and their overlap. The theoretical results are supported with extensive simulations. Lastly, We propose an adaptive scheme for tuning parameter selection based on the variant of Lepski's method [14] given in [5]. This allows us to select a data-driven tuning parameter when only summary statistics are available and cross-validation is not feasible. We prove that tuning parameters chosen by this method satisfy an oracle inequality with high probability, and demonstrate the effectiveness of the method via simulations.

1.1 Related work

The use of summary statistics for regression modeling has been considered in the statistical genetics literature [6]. The lassosum method for polygenic risk prediction was introduced by [22], which considered fitting a L 1 penalized linear regression with summary statistics, and its theoretical properties were studied in depth by [15]. In [4], the authors extend these ideas to polygenic risk prediction with binary traits. The summary statistics used in these methods include the marginal associations between genetic variants and phenotypes, and statistics summarizing the covariance structures among all genetic variants oftentimes derived from a reference genotype dataset. Empirical studies have demonstrated that the efficacy of such models is significantly influenced by the choices of the GWAS summary statistics and the reference dataset [30]. However, there's still limited theoretical understanding regarding how the overlap of samples and the inherent heterogeneity between datasets impact the model performance. Moreover, most current approaches devise models for a single trait within a single ancestral population. Considering shared genetic architectures could potentially enhance performance by employing a multi-task learning strategy [23].The authors of [24] take this approach, and describe a multi-task estimator for multi-ancestry pQTL analysis. However, they do not consider the setting when only summary statistics are available for each task.

Our methods build upon classical multi-task learning techniques, and enable fitting models only using basic summary statistics which are often made publicly available. The sparse regularized estimator extends the group-sparse estimators studied in [19, 18], while the nuclear norm estimator expands on the low-rank regression model described in [25]. The nuclear norm approach is closely related to the linear representation learning problem [7, 31], which constrains the regression coefficients to a shared low-dimensional subspace.

Another recent line of work studies the multi-task learning problem under data-sharing constraints. In [17], the authors describe a federated multi-task learning linear regression model for privacypreserving data analysis. Similarly, [2] presents a computational framework for multi-task learning under DataSHIELD [11] constraints. The formulation of these methods is conceptually similar to ours, but they do not provide theoretical guarantees for their estimators, and we consider a more flexible setting where the summary statistics can be derived from different sources.

Finally, our methods are closely related to the one-shot federated learning paradigm, in which only one round of communication is permitted between the primary local research site and additional sites. [9] presents an algorithm for fitting logistic regression models using summary statistics from different research sites. The works of [20, 34] extend these ideas to linear mixed effects models and generalized mixed effects models, respectively. [21] presents a federated algorithm for fitting the Cox proportional hazards model, and [16] studies federated transfer learning methods for fitting generalized linear models. Nevertheless, the summary statistics addressed in our research are frequently reported in existing studies and can be employed across various models. This is in contrast to the one-shot federated algorithm, where the summary statistics are model specific and the implementation relies on the infrastructure of a collaborative environment.

2 Problem setup and methods

Consider the setting where we are interested in learning a total of Q tasks simultaneously. For each q ∈ [ Q ] , we posit the linear model

where Y ( q ) ∈ R n q , X ( q ) ∈ R n q × p , and ε ( q ) is mean-zero random noise. Each index q corresponds to the q th task. The dataset D ( q ) = ( Y ( q ) , X ( q ) ) contains the individual-level observations of the outcome and features respectively for the q th task. We consider the generic setting where the features D ( q ) might be collected from either overlapping or non-overlapping samples across tasks. Our estimand of interest is the matrix B ∗ = [ β (1) , ..., β ( Q ) ] ∈ R p × Q , where the q th column of B ∗ is β ( q ) . Furthermore, let e i denote the i th standard basis vector, so that β ( q ) = B ∗ e q .

If all the individual-level observations D ( q ) are available, a natural estimator of B ∗ is the regularized multi-task least-squares estimator

where P is a suitable penalty, chosen to enforce similarity structure between tasks, with tuning parameter λ > 0 . However, in many applications, we are less likely to observe D ( q ) . Rather, summary statistics which contains information of the feature-outcome and feature-feature relationships may be more likely to be made publicly available. Motivated by the use case in genetic risk prediction, we assume that only summary statistics ̂ S ( q ) and ˜ Σ ( q ) are observable, where ̂ S ( q ) = 1 n q X ( q ) T Y ( q ) are derived from { X ( q ) , Y ( q ) } , which we termed as the discovery data, and ˜ Σ ( q ) = 1 ˜ n q ˜ X ( q ) T ˜ X ( q ) is a sample covariance matrix computed from the proxy data ˜ X ( q ) ∈ R ˜ n q × p , which may or may not have overlap with X ( q ) .

Our goal is to estimate B ∗ using two sets of summary statistics ̂ S ( q ) and ˜ Σ ( q ) . We note that ˜ X ( q ) is not necessarily equal to X ( q ) . In practice, the studies which report ̂ S ( q ) may not be the same as the ones reporting ˜ Σ ( q ) . Intuitively, we hope that ˜ X ( q ) is generated from a similar population as X ( q ) , but this may not hold in general. In Section 3, our theoretical analysis reveals how the overlap between ˜ X ( q ) and X ( q ) and their distributional shift can influence the accuracy of multi-task learning.

To construct an estimator that uses only the information provided by ˜ D ( q ) , we notice that the leastsquares loss can be written as

By dropping the constant term, we arrive at a loss function that can be computed using only summarylevel information, namely the matrices X T Y and X T X . This motivates our general strategy for constructing an estimator only using summary statistics: we substitute ̂ S ( q ) and ˜ Σ ( q ) where appropriate in each least-square loss function in Equation 1 and arrive at the following optimization problem.

There are many possible choices of P for enforcing structure similarities across tasks. For instance, the recent works of [29] and [12] study low-rank and angle-based penalties for enforcing a shared orientation among the task-specific parameters. In this work, we study two estimators obtained under the ℓ 2 , 1 norm penalty, denoted ∥ . ∥ 2 , 1 , and the nuclear norm penalty, denoted ∥ . ∥ ∗ . These penalties are chosen for their intuitive interpretation: the ℓ 2 , 1 penalty is more likely to be effective if a common set of variables are active across the tasks. If the task-specific parameters tend to be 'correlated',

in the sense that they lie in a low-dimensional subspace, the nuclear norm penalty is preferred. In practice, certain domain knowledge can be incorporated to determine the penalty structure, or it can be chosen in a data-driven way in the existence of a validation dataset.

The corresponding estimators are expressed as follows:

The superscripts ( sp ) and ( lr ) stand for 'sparse' and 'low-rank' respectively.

3 Theoretical guarantees

Before presenting our theoretical results, we first introduce the relevant notation. Let N denote the total size of discovery observations and proxy observation across all Q tasks. Formally,

We note that N may double-count individuals who are part of both the proxy data and the discovery data. Define the subset I q ⊂ [ N ] as the index set for the discovery data points in the q th task; in other words i ∈ I q implies X i ∈ R p is a row of X ( q ) . We define ˜ I q analogously for the proxy data; i ∈ ˜ I q implies X i is a row of ˜ X ( q ) . Let ˜ ρ q = |I q ∩ ˜ I q | / ˜ n q denote the proportion of proxy samples which are also in the discovery dataset for the q th task. In the results that follow, let

and take γ = max q γ q . Additionally, let Ξ ∈ R p × Q be the matrix with its q th column equal to ( Σ ( q ) 1 -Σ ( q ) 2 ) β ( q ) , where Σ ( q ) 1 and Σ ( q ) 2 are the population-level covariance matrices of X ( q ) and ˜ X ( q ) respectively. The quantities γ and Ξ play important roles in our results that follow. In particular, γ is a multiplicative factor in our bounds that represents the cost of using proxy data rather than individual-level data. Similarly, Ξ will represent the cost of using a proxy dataset with a distributional shift from the discovery data. Finally, we will let n min and ˜ n min denote the smallest sample size of discovery and proxy data, respectively. All proofs are given in the supplement.

3.1 Guarantees for ℓ 2 , 1 -norm estimator

In this section, we formally state our assumptions and results for the ̂ B ( sp ) estimator. The assumptions are standard for high-dimensional regularized estimators, see [26] for a deeper discussion of these conditions.

Assumption 3.1 (Sub-gaussian design and noise) . The following holds for each q ∈ [ Q ] : The rows of X ( q ) are independent and identically distributed according to a sub-Gaussian distribution with covariance matrix Σ ( q ) 1 ∈ R p × p . Similarly, the rows of ˜ X ( q ) are independent and identically distributed according to a sub-Gaussian distribution with covariance Σ ( q ) 2 ∈ R p × p . The matrices Σ ( q ) 1 and Σ ( q ) 2 have bounded eigenvalues. The entries of ε ( q ) are independent and identically distributed according to a sub-Gaussian distribution with parameter σ 2 . The X ( q ) and ε ( q ) are independent of one another.

Assumption 3.2 (Shared support) . There exists a subset S ∗ ⊂ [ p ] such that supp ( β ( q ) ) = S ∗ for each q .

Definition 3.1 (Sparse cone) . For any S ⊂ [ p ] , let

Assumption 3.3 (Restricted strong convexity) . There exists a constant κ > 0 and a sequence a N → 0 as N →∞ such that the following inequality holds for each ∆ ∈ C 3 ( S ∗ ) with probability at least 1 -a N :

Theorem 3.1. Under assumptions 3.1, 3.2, and 3.3, there exist constants c 1 and c 2 depending only on the σ 2 and the eigenvalues of Σ ( q ) 1 and Σ ( q ) 2 such that if n min ∧ ˜ n min ≥ c 1 ∥ B ∗ ∥ ∞ , ∞ ( Q +log p ) and λ = O ( √ γ ( Q +log p ) /n min + ∥ Ξ ∥ 2 , ∞ ) , the following inequality holds with probability at least 1 -e -log p -a N :

In the subsequent discussion, we take q ∗ = arg max q ∈ [ Q ] γ q and ( n, n, ˜ ρ ) = ( n q ∗ , ˜ n q ∗ , ˜ ρ q ∗ ) so that the triplet ( n, ˜ n, ˜ ρ ) corresponds to the same sizes and overlap factor used to compute γ .

There are three main quantities in this upper bound that are of novel interest: the ratio of discovery data size to proxy data size n/ ˜ n , the proportion of overlap between the discovery and proxy data ˜ ρ , and the error in specifying the proxy data distribution Ξ = ( Σ (1) -Σ (2) ) β . The first two of these are captured by the factor γ . Our results show that with fixed n and ˜ n , the larger proportion of overlap leads to better estimation accuracy. When the proxy data and discovery data are precisely the same, meaning that I q = ˜ I q for all q , we recover the minimax rate of estimation for the ℓ 2 , 1 penalized multi-task learning problem established by Theorem 6.1 of [18]. If the proxy data and the discovery data are disjoint, meaning that I q ∩ ˜ I q = ∅ for all q , the error is increased relative to the minimax rate by a factor of (1 + n/ ˜ n ) ∥ β ∥ 2 2 . This recovers the result of Theorem 2.1 in [15] up to a constant factor, assuming that Ξ = 0 . The novelty of Theorem 3.1 is that we are able to characterize the convergence rate of ̂ B ( sp ) for any values of n/ ˜ n, ˜ ρ , and Ξ . Additionally, we emphasize that the form of the γ term implies that a price is paid anytime when X ( q ) is not fully contained in ˜ X ( q ) . Indeed, if ˜ ρ < 1 / 2 and we take ˜ n → ∞ we still have that γ > 1 as long as the signal is nonzero. Counter-intuitively, this indicates that an oracle model which has full access to the population-level covariance matrix of the covariates will perform worse in terms of estimation error than an estimator which has access to individual-level data. Furthermore, if ˜ ρ > 1 / 2 , our theorem predicts that the estimator will out-perform the oracle estimator that uses the population covariance matrix. These phenomena are validated in our simulation studies in Section 5.

3.2 Guarantees for the nuclear norm estimator

Now we state our results for the low-rank proxy data estimator, when the penalty is taken to be the nuclear norm. Once again, these assumptions are standard for high-dimensional regression problems with the nuclear norm [26, 33].

Assumption 3.4 (Low rank) . The matrix B ∗ has rank r << p ∧ Q . Let U ∗ and V ∗ denote the column space and row space of B ∗ respectively. Note that U ∗ and V ∗ each have dimension r .

Definition 3.2 (Subspaces) . Let U denote a dimension k ≤ p ∧ Q subspace of R p , and let V denote a dimension k ≤ p ∧ Q subspace of R Q . Define

Furthermore, for any subspace Ω of R p × Q , let ∆ Ω denote the projection of ∆ onto Ω .

We will denote M ∗ = M ( U ∗ , V ∗ ) .

Definition 3.3 (Low rank cone) . For any set M as defined above, let

Assumption 3.5 (Restricted strong convexity) . There exists a constant κ > 0 and a sequence b N → 0 as N →∞ such that the following inequality holds for each ∆ ∈ C 3 ( M ∗ ) with probability at least 1 -b N :

Theorem 3.2. Under assumptions 3.1, 3.4, and 3.5, there exist constants c 1 and c 2 depending only on σ 2 and the eigenvalues of Σ ( q ) 1 and Σ ( q ) 2 such that if n min ∧ ˜ n min ≥ c 1 ∥ B ∗ ∥ ∞ , ∞ ( Q + p ) and λ = O ( √ γ ( Q + p ) /n min + ∥ Ξ ∥ op ) , the following inequality holds with probability at least 1 -e -p -b N :

This theorem recovers precisely the same behavior with respect to γ and Ξ as Theorem 3.1. As γ → 1 , we achieve the minimax rate of estimation for low-rank regression as derived in [27] as long as Ξ = 0 .

4 Tuning parameter selection with Lepski's method

A key challenge of applying penalized regression models to summary statistics is that model tuning based on data splitting (e.g., training and validation) is no longer an option. Model selection methods based on information criteria require knowing the log squared loss log ∥ Y ( q ) -X ( q ) β ∥ 2 2 , which cannot be recovered from ̂ S ( q ) and ˜ Σ ( q ) [1]. To address this, we propose to use a tuning scheme based on Lepski's method [14], a classical tool of nonparametric statistics for adaptive estimation with unknown tuning parameters. The authors of [5] apply the ideas of Lepski to the LASSO, providing a fast algorithm for model tuning with non-asymptotic guarantees. In this section, we extend the methods in [5] to tune the multi-task estimators described in the present work.

The results and ideas in this section apply to both ̂ B ( sp ) and ̂ B ( lr ) , so without loss of generality, let ( ̂ B λ , P ) denote a generic estimator-regularizer pair with tuning paramter λ , which may refer to either ( ̂ B ( sp ) λ , ∥ . ∥ 2 , 1 ) or ( ̂ B ( lr ) λ , ∥ . ∥ ∗ ) . Additionally, let P ∗ denote the dual of P , meaning that

Finally, we let L denote the loss function for both estimators and let ∇L denote its gradient.

The intuition behind the adaptive tuning procedure is that the tuning parameter should be chosen large enough to control fluctuations in the gradient of the loss function, but not too large such that too much bias is incurred. [26] articulates that the performance of regression estimator with a convex penalty is contingent on the following event occurring with high probability:

where we use our problem's notation for continuity. The proofs of Theorem 3.1 and 3.2 involve showing that A ( λ ) holds with high probability under our stated conditions. It is straightforward to prove the following proposition, which states that conditional on A , the gradient of the loss function at our generic estimator ̂ B is close to the gradient at the true parameter B ∗ .

Proposition 4.1. Let ( ̂ B λ , P ) denote a generic estimator-regularizer pair. Conditional on the event A ( λ ) , there exists a constant C > 0 such that the following inequality is satisfied almost surely:

This proposition motivates the following definition, which we adopt from [5].

Definition 4.1. Let Λ = { λ 1 , λ 2 , ..., λ M } denote a grid of potential tuning parameters ordered such that 0 < λ 1 < λ 2 < ... < λ M < ∞ . Fix δ ∈ (0 , 1) . The oracle tuning parameter λ ∗ δ is defined as

The oracle tuning parameter provides the tightest upper bound in Proposition 4.1, but is unknowable in practice, since we do not observe B ∗ and hence cannot verify A . The aim of our Lepski-type method is to mimic the performance of λ ∗ δ in an entirely data-driven fashion. Letting Λ denote our ordered grid of potential tuning parameters, we follow [5] and choose the tuning parameter ˆ λ ∈ Λ that satisfies

where ¯ C is a constant chosen by the statistician. The following theorem states that ˆ λ recovers the behavior of λ ∗ δ with high probability, as long as ¯ C is sufficiently large.

Theorem 4.1. Let C denote the constant in Proposition 4.1. If ˆ λ is chosen as in Equation 5 with ¯ C ≥ C , then the following inequalities hold simultaneously with probability at least 1 -δ :

where C ∗ ≥ ¯ C .

This theorem is a generalization of Theorem 3 in [5], adapted to our setting. The primary advantage of this Lepski-style tuning scheme is that it can be performed using only the gradient of the loss function, which in our setting consists only of summary-level statistics. This is a marked improvement over other summary statistic-based estimators, which typically require an additional set of individual-level data for tuning.

The adaptive tuning scheme does hold some disadvantages. First of all, it requires a choice of constant ¯ C , which should be taken to be as close to the constant in Proposition 4.1 as possible. Remark 10 in [5] offers some guidance as to how to choose ¯ C but unfortunately their analysis corresponds only to the LASSO. Deriving the exact constant in Proposition 4.1 may be possible under stronger assumptions on the data-generating process (i.e. Gaussianity), and we view this as an area of future work. Furthermore, Theorem 4.1 offers only a bound on ∇L ( ̂ B ) -∇L ( B ∗ ) . Translating this to a bound on ̂ B -B ∗ will require strong element-wise conditions on each of the matrices ˜ Σ ( q ) , which we do not explore in the present work. Nevertheless, our simulations in Section 5 indicate that the adaptive tuning method performs well in terms of the MSE of ̂ B , suggesting that adaptive tuning is a good option for model selection when only summary statistics are available.

5 Numerical experiments and real data application

We validate our theory and demonstrate the effectiveness of multi-task learning in proxy data settings via extensive experiments. In each experiment, we take the proxy dataset to be well-specified;

in other words, we assume that Ξ = 0 . When Ξ is nonzero, this predictably leads to worse performance, which we demonstrate in the supplement. The code, further implementation details, and additional simulations which explore the use of our adaptive tuning procedure are also available in the supplement.

First, we consider the effect of varying proxy data size on empirical MSE per task. We generate synthetic Gaussian data with n min = 100 , p = 100 , ˜ n min = τn min for τ ∈ { 0 . 5 , 1 , 2 , 5 , 10 } , and ˜ ρ q = 0 for each q . The number of tasks was fixed at 8. Furthermore, we generate a row-sparse B ∗ matrix with 10 nonzero rows and a B ∗ with rank 2 for the sparse and low-rank multi-task estimators, respectively. We then fit the proxy data multi-task learning estimator and compare the prediction MSE per task to the estimator that has access to all of the individual level data and to the estimator that uses the true covariance matrix Σ . The results of this simulation are given in Figure 1.

Prediction MSE per task

20

Prediction MSE per task

6

15

Estimator

Estimator

5

True_Cov

True_Cov

10

5

0.5

2

5

10

0.5

1

2

5

10

n_tilde/n

n_tilde/n

We observe a performance gap between the estimators that use the true covariance matrix and the individual level estimators, as predicted by our theory in Section 3. The performance of the proxy data estimators increase with increasing proxy sample size, but are unable to match the performance of the individual level estimator, as expected.

Next we study the effect of varying the proportion of overlapping samples between the discovery and proxy datasets. Simiarly, we generate synthetic data with n = ˜ n = 100 , and vary ˜ ρ , which indicates the proportion of proxy data points that are also in the discovery dataset. With Q = 8 , we generate B ∗ in the same way as in the previous simulation. These results are given in Figure 2.

Once again, we observe the expected performance gap between the estimators with the true covariance and the individual-level estimators. As the proportion of overlap between the proxy dataset and the discovery dataset grows, we see that the performance of the proxy data estimator converges to that of the individual level estimator. This is anticipated by Theorems 3.1 and 3.2.

0.35

0.07

per task

0.30

pertask

0.06

Estimator

Estimator

白

IL_Cov

白

IL_Cov

W0.25

白

白

True_Cov

MSEp

0.05

空

白

True_Cov

0.20

0.04

日

00.250.50.751

00.250.50.751

Proportion of overlap

Proportion of overlap

Finally, we have applied our method to analyze real genetic data to demonstrate the real-world applicability of our method. We use a multi-site data obtained from the electronic Medical Records and Genomics (eMERGE) network [28], which includes individual-level genotype data from multiple research sites in the United States. Our goal is to predict levels of low-density lipoprotein (LDL) across five adult sites, treating the data from each site as a separate task. We split the data (with sample sizes n 1 = 3813 , n 2 = 546 , n 3 = 2666 , n 4 = 1435 , n 5 = 525 ) at each task into a training and test set (with a test set data size of 100 for each task) and evaluate the performance of our method using the prediction MSE on the test set. The training data from each site is used to construct the discovery summary statistics ̂ S ( q ) for each task. For approximating Σ ( q ) , we choose two different approaches: one is to use the half of the genotype data from each site (this approach is labeled as Proxy_MTL1); the other approach is to use X 1 (genotype data from site 1) to approximate Σ ( q ) for all the sites. This approach is labeled as Proxy_MTL2. We use these two approaches to demonstrate a potential trade-off in the construction of the reference panel: Proxy_MTL1 uses a well-specified reference dataset with a smaller sample size; Proxy_MTL2 uses a larger reference dataset that may suffer from a distribution shift. For comparison, we also fit a multi-task learning estimator that uses all of the individual level training data for each task, which is labeled 'Individual_MTL', and we fit a ridge regression estimator that models each task separately, for comparison to our multi-task learning approach. The ridge estimator uses the proxy sample covariance instead of the individual level covariance matrix for a fair comparison with our method. We repeat the train-test split process 10 times, which admits the distribution of prediction MSE values that we report in the figure. We use the nuclear norm penalized multi-task learning estimators in this application, because we believe that the genetic effects in this dataset are dense.

Task

0.70

per

Estimator

Prediction MSE p

白

Individual_MTL

Proxy_MTL1

Proxy_MTL2

Proxy_Ridge

0.65

Our results in Figure demonstrate that our method is highly practical when only summary-level information is available, as the prediction MSE of our method is nearly the same as the estimator which uses the individual-level data, despite a slight cost in performance. Furthermore, all multi-task learning estimators outperform the ridge-estimator, confirming that multi-task learning is a strong approach when there is shared structure between tasks.

6 Discussion, Limitations, and Broader Impacts

We have described a flexible multi-task framework incorporating summary statistics from distinct sources with a general data-driven tuning scheme for selecting tuning parameters. Our theoretical analysis sheds light on the intrinsic price of using summary-level information from distinct sources for statistical analysis, and suggests that more overlap between the sources, less distributional shift, and larger proxy data sample sizes can alleviate this cost. Our data-driven tuning scheme allows

models to be trained without sample splitting, making it more applicable to real-world settings with only summary statistics available.

The limitations of our work are summarized as follows. First of all, our methods depend on a linear relationship between the covariates and the outcomes. This assumption is often satisfied in our target application of genetic risk prediction [3], but it does limit the applicability of our method to other domains. To extend our framework to non-linear models, we may use the second-order Taylor approximation of the loss function as in [16]. However, the summary statistics used by such an algorithm are not found in existing literature or publicly available databases. Additionally, our theoretical results provide only upper bounds on the estimation error of the two estimators that we consider in this work. To fully characterize the cost of using summary statistics for multi-task learning, lower bounds resembling Theorem 2.2 of [15] are needed. We conjecture that our estimators converge at a minimax optimal rate, and we view the proof of this conjecture as an important future direction. Finally, we may also extend the framework of [10] to our summary statistic based setting to adjust for potential differences between tasks.

Nevertheless, our results have important implications beyond high-dimensional statistical theory. The trade-off between proxy data sample size and discovery-proxy overlap may inform how polygenic risk models are built in real-world applications: Practitioners should prioritise alignment between the sources of summary statistics that they use to build these models, rather than optimizing for large sample sizes. This guidance may lead to more accurate polygenic scores, which have emerged as an important predictive tool in the field of precision medicine.

We recognize that the development of polygenic risk scores, if done without care, may worsen existing health disparities [23]. This is a potential negative societal impact of our work. We hope that our multi-task learning framework may be used to incorporate data from diverse populations to improve generalizability and transportability of genetic risk predictions to overcome these negative impacts.

References

Kenneth P. Burnham and David R. Anderson. 'Multimodel Inference: Understanding AIC and BIC in Model Selection'. In: Sociological Methods & Research 33.2 (Nov. 2004), pp. 261-304. ISSN: 0049-1241, 1552-8294. DOI: 10.1177/0049124104268644 .

Han Cao et al. 'dsMTL: a computational framework for privacy-preserving, distributed multitask machine learning'. In: Bioinformatics 38.21 (Oct. 31, 2022). Ed. by Jonathan Wren, pp. 4919-4926. ISSN: 1367-4803, 1367-4811. DOI: 10.1093/bioinformatics/btac616 .

Nilanjan Chatterjee, Jianxin Shi, and Montserrat García-Closas. 'Developing and evaluating polygenic risk prediction models for stratified disease prevention'. In: Nature Reviews Genetics 17.7 (July 2016), pp. 392-406. ISSN: 1471-0056, 1471-0064. DOI: 10.1038/nrg.2016.27 .

Ting-Huei Chen et al. 'A Penalized Regression Framework for Building Polygenic Risk Models Based on Summary Statistics From Genome-Wide Association Studies and Incorporating External Information'. In: Journal of the American Statistical Association 116.533 (Jan. 2, 2021), pp. 133-143. ISSN: 0162-1459, 1537-274X. DOI: 10.1080/01621459.2020.1764849 .

Michaël Chichignoud, Johannes Lederer, and Martin Wainwright. A Practical Scheme and Fast Algorithm to Tune the Lasso With Optimality Guarantees . arXiv:1410.0247. type: article. arXiv, Nov. 8, 2016. arXiv: 1410.0247[math,stat] .

Shing Wan Choi, Timothy Shin-Heng Mak, and Paul F. O'Reilly. 'Tutorial: a guide to performing polygenic risk score analyses'. In: Nature Protocols 15.9 (Sept. 1, 2020), pp. 2759-2772. ISSN: 1754-2189, 1750-2799. DOI: 10.1038/s41596-020-0353-1 .

Simon S. Du et al. Few-Shot Learning via Learning the Representation, Provably . arXiv:2002.09434. type: article. arXiv, Mar. 30, 2021. arXiv: 2002.09434[cs,math,stat] .

Rui Duan, Yang Ning, and Yong Chen. 'Heterogeneity-aware and communication-efficient distributed statistical inference'. In: Biometrika (Feb. 12, 2021), asab007. ISSN: 0006-3444, 1464-3510. DOI: 10.1093/biomet/asab007 .

Rui Duan et al. 'Learning from electronic health records across multiple sites: A communication-efficient and privacy-preserving distributed algorithm'. In: Journal of the American Medical Informatics Association 27.3 (Mar. 1, 2020), pp. 376-385. ISSN: 1527974X. DOI: 10.1093/jamia/ocz199 .

Yaqi Duan and Kaizheng Wang. Adaptive and Robust Multi-task Learning . arXiv:2202.05250. type: article. arXiv, Apr. 1, 2022. arXiv: 2202.05250[cs,math,stat] .

Amadou Gaye et al. 'DataSHIELD: taking the analysis to the data, not the data to the analysis'. In: International Journal of Epidemiology 43.6 (Dec. 2014), pp. 1929-1944. ISSN: 1464-3685, 0300-5771. DOI: 10.1093/ije/dyu188 .

Tian Gu, Yi Han, and Rui Duan. Robust angle-based transfer learning in high dimensions . Apr. 7, 2023. arXiv: 2210.12759[stat] .

Tian Gu, Phil H. Lee, and Rui Duan. 'COMMUTE: Communication-efficient transfer learning for multi-site risk prediction'. In: Journal of Biomedical Informatics 137 (Jan. 2023), p. 104243. ISSN: 15320464. DOI: 10.1016/j.jbi.2022.104243 .

O. V. Lepski and V. G. Spokoiny. 'Optimal pointwise adaptive methods in nonparametric estimation'. In: The Annals of Statistics 25.6 (Dec. 1, 1997). ISSN: 0090-5364. DOI: 10.1214/ aos/1030741083 .

Sai Li, T. Tony Cai, and Hongzhe Li. 'Estimation and Inference with Proxy Data and its Genetic Applications'. In: arXiv:2201.03727 [math, stat] (Jan. 10, 2022). arXiv: 2201.03727 .

Sai Li, Tianxi Cai, and Rui Duan. Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approach . Aug. 27, 2021. arXiv: 2108.12112[cs, stat] .

Kunpeng Liu et al. 'Privacy-Preserving Multi-task Learning'. In: 2018 IEEE International Conference on Data Mining (ICDM) . 2018 IEEE International Conference on Data Mining (ICDM). Singapore: IEEE, Nov. 2018, pp. 1128-1133. ISBN: 978-1-5386-9159-5. DOI: 10. 1109/ICDM.2018.00147 .

Karim Lounici et al. 'Oracle inequalities and optimal inference under group sparsity'. In: The Annals of Statistics 39.4 (Aug. 2011). Publisher: Institute of Mathematical Statistics, pp. 2164-2204. ISSN: 0090-5364, 2168-8966. DOI: 10.1214/11-AOS896 .

Karim Lounici et al. Taking Advantage of Sparsity in Multi-Task Learning . arXiv:0903.1468. type: article. arXiv, Mar. 8, 2009. arXiv: 0903.1468[math,stat] .

Chongliang Luo et al. 'DLMM as a lossless one-shot algorithm for collaborative multi-site distributed linear mixed models'. In: Nature Communications 13.1 (Mar. 30, 2022), p. 1678. ISSN: 2041-1723. DOI: 10.1038/s41467-022-29160-4 .

Chongliang Luo et al. 'ODACH: a one-shot distributed algorithm for Cox model with heterogeneous multi-center data'. In: Scientific Reports 12.1 (Apr. 22, 2022), p. 6627. ISSN: 2045-2322. DOI: 10.1038/s41598-022-09069-0 .

Timothy Shin Heng Mak et al. 'Polygenic scores via penalized regression on summary statistics: MAK et al.' In: Genetic Epidemiology 41.6 (Sept. 2017), pp. 469-480. ISSN: 07410395. DOI: 10.1002/gepi.22050 .

Alicia R. Martin et al. 'Clinical use of current polygenic risk scores may exacerbate health disparities'. In: Nature Genetics 51.4 (Apr. 2019), pp. 584-591. ISSN: 1061-4036, 1546-1718. DOI: 10.1038/s41588-019-0379-x .

Aaron J. Molstad et al. Heterogeneity-aware integrative analyses for ancestry-specific association studies . June 8, 2023. arXiv: 2306.05571[stat] .

Sahand Negahban and Martin J. Wainwright. 'Estimation of (near) low-rank matrices with noise and high-dimensional scaling'. In: The Annals of Statistics 39.2 (Apr. 1, 2011). ISSN: 0090-5364. DOI: 10.1214/10-AOS850 .

Sahand N. Negahban et al. 'A Unified Framework for High-Dimensional Analysis of $M$Estimators with Decomposable Regularizers'. In: Statistical Science 27.4 (Nov. 1, 2012). ISSN: 0883-4237. DOI: 10.1214/12-STS400 .

Angelika Rohde and Alexandre B. Tsybakov. 'Estimation of high-dimensional low-rank matrices'. In: The Annals of Statistics 39.2 (Apr. 1, 2011). ISSN: 0090-5364. DOI: 10.1214/ 10-AOS860 .

the eMERGE Team et al. 'The eMERGE Network: A consortium of biorepositories linked to electronic medical records data for conducting genomic studies'. In: BMC Medical Genomics 4.1 (Dec. 2011), p. 13. ISSN: 1755-8794. DOI: 10.1186/1755-8794-4-13 .

Ye Tian, Yuqi Gu, and Yang Feng. Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness . arXiv:2303.17765. type: article. arXiv, Mar. 30, 2023. arXiv: 2303.17765[cs,stat] .

Ali Torkamani, Nathan E. Wineinger, and Eric J. Topol. 'The personal and clinical utility of polygenic risk scores'. In: Nature Reviews Genetics 19.9 (Sept. 2018), pp. 581-590. ISSN: 1471-0056, 1471-0064. DOI: 10.1038/s41576-018-0018-x .

Nilesh Tripuraneni, Chi Jin, and Michael I. Jordan. Provable Meta-Learning of Linear Representations . arXiv:2002.11684. type: article. arXiv, Dec. 31, 2021. arXiv: 2002.11684[cs, stat] .

Joshua R Vest and Larry D Gamm. 'Health information exchange: persistent challenges and new strategies: Table 1'. In: Journal of the American Medical Informatics Association 17.3 (May 2010), pp. 288-294. ISSN: 1067-5027, 1527-974X. DOI: 10.1136/jamia.2010. 003673 .

Martin Wainwright. High-dimensional statistics: a non-asymptotic viewpoint . Cambridge series in statistical and probabilistic mathematics 48. Cambridge ; New York, NY: Cambridge University Press, 2019. ISBN: 978-1-108-49802-9.

Zhiyu Yan et al. 'A privacy-preserving and computation-efficient federated algorithm for generalized linear mixed models to analyze correlated electronic health records data'. In: PLOS ONE 18.1 (Jan. 17, 2023). Ed. by Sathishkumar V. E., e0280192. ISSN: 1932-6203. DOI: 10.1371/journal.pone.0280192 .

Yu Zhang and Qiang Yang. 'A Survey on Multi-Task Learning'. In: IEEE Transactions on Knowledge and Data Engineering 34.12 (Dec. 1, 2022), pp. 5586-5609. ISSN: 1041-4347, 1558-2191, 2326-3865. DOI: 10.1109/TKDE.2021.3070203 .

Yu Zhang and Qiang Yang. 'An overview of multi-task learning'. In: National Science Review 5.1 (Jan. 1, 2018), pp. 30-43. ISSN: 2095-5138, 2053-714X. DOI: 10.1093/nsr/nwx105 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work considers multi-task learning in settings where for each task only summary statistics X.T@Y and ẍ.T@ẍ are made available. The setting is motivated by healthcare and biomedical related research, where sharing individual level microdata is restricted by regulations due to privacy concerns. The authors assume a linear and sparse or low-rank underlying model. A regularized least-squares type of optimization framework is used, and it is highlighted that only summary statistics are needed to solve it. A main contribution of the paper is theoretical analysis bounding the the quality of the sparse or low-rank estimator taking into account quantities such as overlap and distributional shift between the so called discovery and proxy data used to calculate the summary statistics. Additionally, the problem of hyperparameter tuning is considered as basic (cross-)validation methods are not applicable without access to individual data, and Lepski’s method is proposed as a practical alternative that requires only summary level data as well as some prior knowledge in form of a constant C parameter. Experiments on simulated data show results that are consistent with the theoretical analysis.\n\nSTRENGTHS:\n- In general, approaches that allow learning from summary statistics are becoming ever more relevant to the community as regulation on sharing individual level data is tightening \n- The exact formulation of the multi-task summary statistic learning problem considered in the paper, where separate discovery and proxy data sets are assumed appears to be novel, as consequently the theoretical bounds that take this into account provide also novel insights about the effects this can have on learning. \n- Both L1,2 and nuclear norm regularization based variants of the learning problem are analyzed\n- The proposed approach for model selection provides a practical tool for hyperparameter tuning when only summary statistics are available, and might have implications also for other variations of the problem setting?\n- Fairly clear writing and technically rigorous\n\nWEAKNESSES:\n- Real-world relevance: while applications such as polygenic risk prediction are mentioned, there are no convincing examples, real applications of benchmark data sets provided where the assumptions of the learning setting (having related tasks, only X(q).T@Y(q) and ẍ(q).T@ẍ(q) available, perhaps X(q) != ẍ(q)) would hold. Thus the question about practical significance of the exact considered problem setting and proposed solution remain unclear.\n- Related to above point - no experiments on real-world data that would demonstrate the benefits of the approach\n- Somewhat restrictive assumptions such as assumptions about the linearity of the underlying model and relationships between the tasks\n\nQUESTIONS:\nCan you clarify on the assumption that the discovery and proxy data sets can be separate, when would this in practice be an important consideration?\nFigure 3: am I reading this correctly that the holdout method does not benefit at all from having larger validation sample size? Any explanation for this?\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

comments: SUMMARY:
The paper proposed a multi-task method to learn individual models without having access to the raw data but using summary statistics data for each task. The method is only applicable to linear models.

STRENGTHS:
1.  Theoretical guarantee for the optimal estimator 
2.  Good experiments on simulated data

WEAKNESSES:
1. Missing experiments on real data
2. Applicable only to linear models.

QUESTIONS:
1. Why is the discovery data estimated from the actual observed data, unlike the sample covariance matrix was estimated from a proxy data?
2. The proposed method works only for linear models. Can the same analysis be applied using non-linear (deep learning) models?
3. In Figure 1, can you explain the behavior that the proxy covariance gave lower MSE than the tru covariance?

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an approach to learning predictive models from summary statistics in the setting of multi-task learning.  Linear model is assumed and least square solution was derived with either the \\ell_{2,1}-norm or kernel norm of the parameter matrix to capture relativeness among tasks. Theoretical results for bounding the estimator when proxy data are used to obtain the covariance matrix and hyperparameter tunning are provided. Synthetic data was used in their empirical evaluation.\n\nSTRENGTHS:\n+ The studied problems, both training predictive models from summary statistics and the use of proxy data to estimate covariance are of great importance, given the constraints on data sharing in medical research and the lack of availability of covariance in current practice of GWAS. \n+ Theoretical results can be very useful in assessing the impact of the use of proxy data. \n+ The approach proposed to tune the hyperparameters is important given there is typically no validation set consisting of individual level data for such tunning.\n\nWEAKNESSES:\n- Even though the studied problems are significant, the proposed approach lacks technical innovations, seeming trivial extensions of existing works for both the bound derivation and hyperparameter tunning. The authors may point out the technique challenges in these extensions, highlighting their technical contribution. \n- Lack of evaluation on practical datasets. It is unclear how useful this method is in practical problems.\n\nQUESTIONS:\n- No test set is mentioned in the description of their experiment. Is there a separate test set? I do not think training MSE is a good measure as it is affected by the sparse driving term in the optimization.  \n-  I found the results shown in Figure 2 is a bit confusing. Given n = \\tilde{n}, \\tilde{p} = 1 means the proxy data is exactly the true data, meaning the proxy covariance is exactly the true covariance. Also, covariance is the only input from the proxy data to obtain the solution according to the estimator. These together mean the solution obtained with proxy_cov should be the same as that with true_cov at \\tilde{p}=1. How the two plots show the former is the same as that with IL_cov?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of multi-task learning in settings where only summary statistics (instead of individual-level data) is available, which is a common scenario e.g. in medicine. The paper presents a framework for linear relationships between the covariates and the outcomes which uses summary-level information from distinct sources with a general data-driven tuning scheme for selecting tuning parameters. The results from the theoretical analysis of this framework are confirmed with numerical experiments on synthetically generated data.\n\nSTRENGTHS:\nThe paper addresses an important problem for fields where individual level data is not publicly available, which is often the case in health care applications. The numerical experiments confirm the theoretical analysis.\n\nWEAKNESSES:\nWhile the paper addresses an important topic, the presented framework comes with strong limitations. Particularly, the framework can only be applied for linear models which drastically reduces the potential applications of this approach. Additionally, the experimnts are limited to rather simple, synthetic examples that indeed show of the expected behaviour based on the theoretical results, but do not provide any insights on the applicability of the framework in real-world scenarios. Finally, the presentation could be improved (see comments below). \n\nComments on presentation:   \n- on related work: first, related work should also be a numbered section (not an unnumbered subsection of introduction). second, there was also quite recently some work by Meija et al. (2022) on estimating causal effects only based on summary statistics reported in medical studies, that seems quite related to the presented problem setup. There the authors show that merging datasets with maximum entropy improves the predictive power compared to using the observed marginal distributions as predictors.   \n- Line 99: public available -> publicly available. \n- After equation (1) you already start using variables with and without tilde ($\\tilde{\\bf X}$ and ${\\bf X}$, $\\tilde{\\mathcal{D}}$ and $\\mathcal{D}$), but you have not really introduced what the difference between them is and only introduce proxy and discovery data (without explaining the terms) in section 3. You should introduce the proxy data (and what that means) much earlier.   \n- Line 124: in a data-driven in the ... -> in a data-driven way in the ...  \n- Line 135 and equation above: In the equation use \\left( and \\right) for the brackets to scale them appropriately. Further, you use $\\varepsilon$ as random noise before (above line 88), and now define $\\Varepsilon$ as cost of using proxy data that is different from the discovery data. That's confusing. Similarly confusing is that you have now $\\tilde{\\Sigma}$, $\\Sigma_1$ and $\\Sigma_2$. Isn't $\\Sigma_2$ the same as $\\tilde{\\Sigma}$? (if not, please clarify what's the difference).  \n- Line 175: what is $\\tilde{p}$? wasn't there only $p$ and $\\tilde{\\rho}$ before?\n- position parameters of figure 2 and 3 should be set to [t]\n- Overall structure could be improved. Suggest to have a section on notation and assumptions used throughout the paper before the current section 2. Then you can introduce the problem setup. In that section you right now already mention how summary statistics get included. One could consider doing that only in the next section that focusses on your approach. This way it should be (hopefully) clearer what you propose and what is new about it. Right now it's difficult to understand what are common assumption, what is common knowledge, and what was added on top by you.\n\nQUESTIONS:\n- The framework is designed for multi-task learning with linear relationships between the covariates and the outcomes. Can you provide real-world examples that satisfy this constraint? \n- You note that ${\\bf X}^{(q)}$ and $\\tilde{\\bf X}^{(q)}$ are not necessarily the same. Can you give a practical example where they are the same, and one where they are not?\n- To obtain equation (2), you replace ${\\bf X}^T{\\bf X}$ with $\\tilde{\\bf \\Sigma}$, although you before mention that $\\tilde{\\bf \\Sigma}\\propto \\tilde{\\bf X}^T \\tilde{\\bf X}$, and ${\\bf X}$ and $\\tilde{\\bf X}$ are not the same. Why can you do this replacement here then?\n\nLIMITATIONS:\nauthors adequately addressed the limitations and potential negative societal impact of their work", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMulti-task learning is a powerful machine learning paradigm for integrating data from multiple sources to improve overall model performance. However, data-sharing constraints in healthcare settings hinder its application. To address this challenge, a flexible multi-task learning framework utilizing summary statistics from various sources is proposed, along with an adaptive parameter selection approach based on a variant of Lepski's method. A systematic non-asymptotic analysis characterizes the performance of the proposed methods under various regimes of sample complexity and overlap. Extensive simulations demonstrate the theoretical findings and the performance of the method, offering a flexible tool for training related models across various domains with practical implications in genetic risk prediction and other fields.\n\nSTRENGTHS:\nMulti-task learning is a promising approach to integrating data from multiple sources and improving individual task performance.\nData-sharing constraints in healthcare and biomedical research limit access to individual-level data, making summary statistics a valuable substitute. The proposed multi-task learning framework allows for simultaneous learning of multiple models using only publicly available summary statistics. A systematic non-asymptotic analysis characterizes the performance of the proposed methods under various regimes of sample complexity and overlap. An adaptive scheme for tuning parameter selection based on a variant of Lepski's method is proposed, allowing for data-driven tuning when only summary statistics are available. The framework has practical applications in genetic risk prediction and can be used to develop trans-ethnic prediction models. The ability to learn from summary statistics offers a versatile tool for developing models across various domains.\n\n\nIndeed, a major advantage of this method is that it has theoretical guarantees. Through systematic non-asymptotic analysis, this method provides theoretical guarantees for the performance of the multi-task learning framework based on summary statistics, especially under different regimes of sample complexity and overlap. These theoretical guarantees help us better understand the performance of this method and provide guidance for practical applications. Additionally, this method proposes an adaptive parameter selection approach based on a variant of Lepski's method, which can perform data-driven tuning of parameters when only summary statistics are available. The effectiveness of this method is also theoretically guaranteed, which can help us better select parameters and improve the performance of the model in practical applications. Therefore, this method has great practical value and provides a feasible approach for multi-task learning under data sharing constraints.\n\nWEAKNESSES:\n1. One major drawback of this method is the lack of comparison with classical multi-task learning methods. While this method provides a novel and useful approach for dealing with data sharing constraints, comparing its performance with existing multi-task learning methods would help to further evaluate its effectiveness. By comparing their performance, we can gain a better understanding of the strengths and weaknesses of these methods and provide better guidance for practical applications. Therefore, future research could consider comparing this method with existing multi-task learning methods.\n2. Another potential drawback of this method is that it may not be able to integrate with existing deep learning models. This method is based on training multiple models using basic summary statistics, while deep learning models typically require large amounts of raw data and complex network structures for training. Therefore, integrating this method with existing deep learning models may face many challenges, such as how to convert summary statistics into input data and how to design network structures suitable for summary statistics. While this method is useful in utilizing summary statistics, it may not be applicable in tasks that require processing large amounts of raw data and performing complex computations. Therefore, in practical applications, appropriate methods and models need to be selected based on task requirements.\n\nQUESTIONS:\n1. How does the performance of this method compare to existing classical multi-task learning methods?\n2. Can this method be integrated with existing deep learning models, and if so, how?\n3. In what type of tasks is this method most suitable, and what type of tasks may require other methods and models?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}