Co-Regularization Enhances Knowledge Transfer in High Dimensions
Authors: Shuo Shuo Liu
Haotian Lin
Matthew Reimherr
Runze Li
Venue: NeurIPS 2025 poster ()
Keywords: High Dimensional Statistics
Transfer Learning
Generalized Linear Models
Statistical Learning
Primary area: theory

Abstract: Most existing transfer learning algorithms for high-dimensional models employ a two-step regularization framework, whose success heavily hinges on the assumption that the pre-trained model closely resembles the target. To relax this assumption, we propose a co-regularization process to directly exploit beneficial knowledge from the source domain for high-dimensional generalized linear models. The proposed method learns the target parameter by constraining the source parameters to be close to the target one, thereby preventing fine-tuning failures caused by significantly deviated pre-trained parameters. Our theoretical analysis demonstrates that the proposed method accommodates a broader range of sources than existing two-step frameworks, thus being more robust to less similar sources. Its effectiveness is validated through extensive empirical studies.

Co- Co-Regularization Enhances Knowledge Transfer in High Dimensions
Shuo Shuo Liu Haotian Li
Matthew Reimherr
Runze Li'
Thee Pennsylvania State University
2 AAmazon
Abstract
Most existing transfer learning algorithms for high-dimensional models employ a two-step regularization framework, whose success heavily hinges on the assumption that the pre-trained model closely resembles the target. To relax this assumption, we propose co-regularization process directly exploit beneficial knowledge from the source domain for high-dimes -ighesimensional generalized linear models. The proposed method learns the target parameter by constraining the source parameters to be close to the target one, thereby preventing fine-tuning failures caused by significantly deviated pre-trained parameters. Our theoretical analysis demonstrates that the proposed method accommodates broader range of sources than existing two-step frameworks, thus being more robust to less similar sources. Its effectiveness validated through extensive empirical studies.
1 Introduction
Transfer learning (TL) is a technique that leverages knowledge from source domains to improve learning performance in related but not necessarily identical target domain (Torrey and Shavlik, 2010). The success of TL usually relies on the philosophy that the pre-trained models over source domains carry informative knowledge and thus benefit the target domaín learning by fine-tuning the pre-trained models. This "model-based" two-step transfer framework is referred to as hypothesis transfer learning (HTL.) in the community.
Recently, HTL has also drawn great attention and been studied in various high-dimensional models (Bastani, 2021; Li et al., 2022; Tian and Feng, 2023). These algorithms are usually established in the well-recognized pre-training and fine-tuning paradigm. Specifically, these algorithms pool the source datasets and obtain the pre-trained parameters via classical sparse learning techniques, such as Lasso regression (Tibshirani, 1996). The target parameter is then learned by minimizing regularized empirical risk minimization over the target data, with the regularization the pre-trained parameter and the target parameter. The underlying assumption in these works to show provable learning gain is that the distance between pre-trained and target parameters is small. Later, Gu et al. (2024); Lin and Reimherr (2024b) revealed that the transfer learning gain dynamic of HTL further relies on the geometric angles and signal strength between source and target parameters, which thus raises the demand for obtaining a "good" pre-trained parameter as the lever.
the distance between
Most of the aforementioned multi-source transfer learning methods indiscriminately fuse all source and/or target domains to obtain pre-trained estimator for the target domain. However, this approach overlooks the fact that different source domains may vary in their relevance and contribution to the target task. Besides, since pre- pre-trained parameters are optimized to perform well over the source
"Co-first author.
This work does not relate to the authors' positions at Amazon. Correspondence to Shuo Shuo Liu (shuoliu.academic @ gmail.com).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

domain (Li et al., 2022), they are not guaranteed to be sufficiently similar to the target parameter. and thus, fine-tuning might fail to generalize well on the target domain. Therefore, the above observations raise the following question:
How to better exploit the knowledge from sources that can enhance the learning of the target?
In this work, we take a step towards addressing the aforementioned questions by proposing multi- source integration transfer framework. We summarize our contributions as follows.
For a given set of sources, we propose a co-learning process termed Co- Co-Regularization Transfer (CoRT) for high-dimensional generalized linear models (GLMs). This process enables the target parameter to be a representation of the source parameter while minimizing the source empirical risk, thereby enhancing the transfer of knowledge that is beneficial to the target and preventing fine-tuning failures due to significantly deviated pre- trainedi parameters over sources. We develop convergence rates for the oracle CoRT theoretically support its effectiveness and demonstrate that it accommodates a wider range of sources, achieving improved rates compared to existing two-step pre-training and fine-tuning frameworks.
In the presence of outlier source domains and without prior knowledge about the sources, we propose a data-driven adaptive algorithm to identify these outlier sources, which prevents negative transfer in CoRT. Unlike existing source detection algorithms,, whose performance heavily hinges on correctly selected hyperparameters. our approach eliminates such a need by leveraging majority vote mechanism. We also theoretically show that this adaptive algorithm converges to the oracle.
Beyond point estimation, show that the proposed CoRT framework can also facilitate high- dimensional statistical inference by improving the coordinate-wise confidence interval (CI) for the target parameter. The CI construction is based on a desparsifying process of the CoRT estimator, and we theoretically show that the constructed CIs are narrower in length compared to those in single-task settings under certain conditions.
Related Works. Existing two-step HTL algorithms are based on the biased regularization (Schölkopf et al., 2001), which has been extensively applied and studied under supervised learning settings, e.g., Kuzborskij and Orabona (2013, 2017); Wang and Schneider (2015); Du et al. (2017);
Lin and Reimherr (2024a,b).
Recently, this technique has been intensively adopted for various high-dimensional models in statistics with theoretical guarantees (Bastani, 2021; Li et al., 2022; Tian and Feng, 2023; Zhang and Zhu, 2022; Li et al., 2024; Liu, 2024; He et al., 2024). Specifically, Bastani (2021) studied the single-source TL in the high-dimensional linear regression and leveraged the biased regularization, which can be viewed as an extension from Kuzborskij and Orabona (2013) the high-dimensional linear model. However, since Bastani (2021) considered an end-to-end training setting, the statistical convergence rate depends on both the source and target sample sizes. thereby demonstrating how large sample size in the source domains can enhance transfer efffectiveness. Li et al. (2023) extended the algorithm in Bastani (2021) to multiple sources by fusing all source datasets into the target dataset build one pre-trained parameter. The convergence rates are shown to be faster than those using the target data under certain conditions. To exclude the impact from outlier sources, data-driven algorithm using the Q-aggregation method (Rigollet and Tsybakov, 2011) 1) is developed. Later, Tian and Feng (2023) extended the work of Li et al. (2022) to high- dine-siieenion GLMs. Instead of aggregation, they proposed a sample splitting-bant based algorithm for outlier source detection. However, the threshold hyperparameter selection in the detection algorithm can significantly affect its performance and needs to be manually tuned. Li et al. (2024) studied the estimation and inference of the high-dimensional GLMs by constraining the empirical score functions. Both Tian and Feng (2023) and Li et al. (2024) provided coordinate-
confidence intervals leveraged by desparsified Lasso and correction scores, respectively. It is worth mentioning that the aforementioned works on HTL in high-dimensional linear models primarily employ the two-step learning framework and the -norr for regularization.
Broadly speaking, our work is related to multi- source domain adaptation (Sun et al., 2015). It has achieved extensive success in various applications, but the theoretical analysis remains somewhat lacking. Most theories are based on the assumption that the target distribution can be approximated by a mixture of source distributions, making the error bounds depend on divergence measures between domains (Blitzer et al., 2007; Ben-David et al., 2010; Mansour et al., 2008; Hoffman et al., 201 (018). These bounds are overly general and unable to provide fine-grained analysis about how larger source

---

sample size and minor domain discrepancies contribute to faster learning rate. is worth noting that multi-task learning is closely related to transfer learning, though their objectives differ. Multi-task learning aims to solve multiple learning tasks simultaneously by leveraging shared structure across tasks, like the Data Shared Lasso framework (Gross and Tibshirani, 2016; Ollier and Viallon, 2017) in high dimensions. In contrast, transfer learning focuses on improving performance a specific target task by transferring knowledge from source data.
2 Transfer via Co-Regularization
Problem Setup. Consider + tasks, we observe the k-th task data Dik) == [(xx (M), (2)
Mi ") )m2
which are generated from the following canonical form of GLMs
IP
(M) U |x, (( p(y, (8) exp
(N)T ((x))
(xx; (T)) 3(A)]
for k = 0, 1, K, where k: = O denotes the target model and k + 0 denotes source models, 3
denotes the parameter, and & are some known functions. For a given pair (xx(6), y()) from the k-th
model, the GLMs model the relationship between input and output via B(y (x()))))))))))))()))))))))))([) = (x 3 3(8)).
where sl' is the first-order derivative of the cumulant function. The goal is to learn the target ((0) using both the target and source data.
Similar and Outlier Tasks. In order facilitate the algorithm design and develop corresponding statistical foundations, it necessitates some assumptions on the similarities between the source and target domains. We instantiate the statistical heterogeneity between the k-th source model and the target model as the l norm between (3k) and ( Specifically, we define the k-th parameter
contrast as ((k) = (b)) - (0 for 1 S S K, and the k-th source considered as a "similar" source if k E S = (1 k K: 10 S hi where is an unknown nonnegative parameter that controlling the similarity level. Intuitively, a larger h indicates larger statistical heterogeneity and vice versa. For the sources in set Sj = S 1: I K : l & S,), we can interpret them as "outlier" sources. Such notations have been widely adopted in the theoretical studies of transfer learning (Bastani, 2021; Li et al., 2022; Tian and Feng, 2023) or adversarial attacks/contamination (Qiao, 2018; Konstantinov and Lampert, 2019; Konstantinov et al., 2020). In practice, such oracle set S is typically unavailable, necessitating algorithms to identify those similar sources to better boost the learning of the parameters of interest. For simplicity, we denote Sn, as S and ngg ExES nk-
2.1 Co-Regularization Enhances Knowledge Transfer
To facilitate introducing the proposed Co-Regularization transfer scheme, we first focus on the case where there is no outlier source task, i.e., all sources are in S, and anchor on this to develop an algorithm that can adapt to an unknown similarity level h.
Many existing transfer learning algorithms for high- dimensional models rely on two-step paradigm achieve knowledge transfer. Specifically, they lever- age data fusion technique in the first step by fusing all data from source models in S to obtain a pre- trained model and then fine-tune the model via the target data in the second step; see Appendix B for de- tails. However, has been empirically observed and theoretically revealed (Tian and Feng, 2023; Lin and Reimherr, b:24a.b; Gu al., 2025) that if the fused source model and target model differ significantly in terms of signal strength and angle, fusing all source data will result a pre-trained model that suffers from significant bias, and thus it will be extremely difficult to have the subsequent fine-tuning steps to mitigate this bias, given the limited target data.
Source
D
Predrumna y
Targe
Trrgetnoce
Figure 1: Left: Two-step pre- pre-training and fine- tuning learning scheme (Li et al., 2022: Tian and Feng, 2023); Right: Co-Regularization transfer scheme (CoRT).
Consider the following scenario: while minimizing the empirical risk over the target domain, one simultancously train the source parameter (k)) on the source domain by constraining the (k() to
3

---

be close to (00), which ensures only the beneficial knowledge is acquired by ( Specifically, we can set 3(t) ((0) 8(h) during training instantiate such constraint. Based this idea, propose the Co- Ce-Regularization Transfer (CoRT), which is a co-learning process by minimizing the negative log- se-likelihood with Co-Regularization. Figure 1 provides illustration of co-regularization and its comparison with classical two-step pre-trate pre-training and fine-tume mne-tuning transfer learning schemes.
Definition 1 (Co-Regularization Transfer). Define the negative log- log-likelihodd over the D(k) as (
((k) n Ei (xx!+) 9)
yi (x: (K)T
(K)
(), then solve the following objective
B ( 310,(60%))
argmin
LD (3 (0 JAES)
(
where
(1)
LD (B (8 JRES) ) E
E(e( (((( + , (DW)))))))))))) + Pre (8thl)
J (3 D(O) ) + (B
RES
The regularization terms Pro and Pxt satisfy some general conditions that are widely used for high-ddiennsooall regression; see Appendix f a more detailed discussion.
Remark 1. In general, the regularizers Pho and Pxx can be customized and set many famous regularizers, including the convex Lasso ('lbsshirani 1996) and nonconvex SCAD (Fan and Li, 2001 ). The estimation procedure follows by iteratively updating (00) and ( JRES using the coordinate descent algorithm (Breheny and Huang, 2011).
Note that CoRT simultaneously estimates the target parameter (0) and the contrasts ((k) One specific advantage of CoRT compared to the well-known Trans-GLM (Tian and Feng. 2023) with oracle that CoRT allows one to provide insights into which covariates are transferable in the k-th source domain. Specifically, one obtain the set containing the transferable covariates in the k-th source model by {j : 8/4) = 0, 1 S j S p). contrast, Trans-GLM fuses all the sources in S
together to estimate the pre-trained parameter 3g and the fused level contrast g Bs, which is unable to identify source-level transferable covariates. Similar limitation also exists in other two-step algorithms for multiple sources (Bastani, 2021: Li et al., 2022).
This paper focuses on GLMs primarily because this allows us to make head-to-head comparisons with the two-step seminal framework both theoretically and empirically; Table 1 and Section 4.1. However, we would like to emphasize that the CoRT framework is model-agnostic, making it able to be generalized beyond GLMs naturally by simply replacing (a)) and (A))
Comparison to Multi-Task Learning. Although the objective of CoRT (1) is similar to those objectives of multi-task learning (Zhang and Yang, 2021), their internal logic is fundamentally different. We compare CoRT to the following multi-task learning with biased-regularizzaton (MTL- BR) framework, which, under the same problem setup, can be expressed as
MTL-BR: 300) (805) JKKS
argmin g g Jeeou keOUS
Σ {{( (3++00), D(R) + Ps. (0 (2)
The MTL-BR is the backbone of various modern multi-task learning algorithms (Denevi et al., 2019; Duan and Wang, 2023; Tian al., 2025). It decouples the learning of (x() as B + (()) where denotes the shared parameter and (A)) denotes the task-specific parameter. By comparing with (2), it is clear that the CoRT objective (1) is asymmetric w.r.t. the source and target models and is target-centric, i.e., putting (0)) centre. Consequently, the tasks in MTL equally affect the shared parameter, whereas the source tasks in CoRT affect the target only as their parameters are regularized towards (0)) Besides, the target-centric nature of CoRT allows (0)) to remain in the source loss function and exploit the benefit of using source samples; see Theorem 1 for justification.
2.2 Adaptive Knowledge Transfer
In real-world applications, it is often challenging to determine which source belongs to the oracle set for effective knowledge transfer. Therefore, the algorithm should exclude outlier sources during training. which is practically challenging task (Pan and Yang, 2009).

---

To address this practical challenge, we propose a data-driven Algorithm 1 using the majority vote to adaptively detect "similar" sources without accessing h. The algorithm is based on the intuition that if incorporating the data from a similar source enhances learning over the target domain, then the error of g(ok), trained on D(0) U D(6), should not exceed that of (3(), trained solely on D(O).
Algorithm 1: Adaptive Co-Regularization Transfer
1: Input: Datasets ((A) for 0 I k S K.
2: Data splitting: randomly split the target data D(0) into T (an odd number) parts of equal size and denote them as ((0) " [
3: for k to K do
4: for = 1 to T do
5:
Obtain 3 (
by running Lasso on D
((O))(()) (0)
[t]
6: Obtain 3 ((0k) by running Lasso on (D(0) D U(())
[4]
[1]
7: Calculate errors over D {( D( and (()(00))
(0))
D respectively,
8: end for
9: Define C. (3/O0) as ET { (()
(0k) ) min
BE (E 3 E(B, 1(0))
10: end for
11: Obtain the estimator of S as S = -(m h: C (3108) 2 - 1)/2
12: Obtain (0) by running CoRT with D(k) k E S u (0).
One notable algorithm for identifying "similar" and "outlier" sources is from Tian and Feng (2023), which divides each source into three folds for validation and relies on two hyperparameters to determine whether a source should be included in S. These hyperparameters play crucial role in the detection and can degrade performance if incorrectly selected, sometimes performing worse than single-task learning. We emphasize that while our adaptive algorithm presents similar spirit to that in Tian and Feng (2023), it essentially differs in two key aspects: (1) we utilize a T-fold vross-validation to assess the impact of incorporating a specific source task; (2) we determine $ through a majority vote approach, eliminating the need for hyperparameter tuning.
2.3 Inference for Target Tasks
In this section, we aim construct an asymptotic element-wise confidence interval (CI) for (()) Given the difficulty of building CI in high-dimensional setting, we adopt the desparsified technique from Van de Geer al. (2014) to our CoRT framework, which consists of two main steps. First, we construct the covariance matrix for target tasks and an asymptotic unbiased estimator through the desparsified technique. We then construct CI via the precision matrix. In the following, we briefly introduce the CI construction process and refer to Algorithm 4 in Appendix D for all details.
Step 1: Target Covariance Matrix. The population target covariate matrix is defined as >
(0) 1 x (0)T (300
E[x (0) ) ((x(0))) ((0)), and the empirical version is defined as
(
ng ( 8(0),
where X (0) (()) V diag (x(((x
(0)T ((0)). ((00))
(x
( ((0) (()
Step 2: Asymptotic Unbiased Estimator. We now obtain the explicit form of the desparsified estimator for (3(0 under the CoRT framework, which is asymptotically unbiased.
Proposition 1. Let (00 denote the estimator of (0) obtained by minimizing the objective (l) and let Ô be a relaxed inverse of ((0)) goop then the desparsified estimator of (300
(000 = =3(00 +
1
{y(0) (0) 3 (' 3(0)
(3)
no
5

---

We highlight that the desparsified estimator (3) is not simple extension of those in the literature (Van de Geer et al., 2014; Gueuning and Claeskens, 2016). Specifically, the desparsified estimator is constructed under the multi-source CoRT framework, while the classical desparsified estimators in the literature are constructed a single-source setting.
Remark 2.
With the asymptotic normality of (() (proved in Theorem 3), can construct the (1 - a--CI for each coordinate of z(o) The remaining task to construct an estimator for the precision matrix é, which we finish by incorporating data from similar source tasks and applying the node-wise regression technique introduced by Meinshausen and Bühlmann (2006). This process also provides an "improved"" estimator for the precision matrix compared to the single-task learning setting as indicated by Theorem 3, and thus can contribute to the precision matrix estimation problem.
3 Theoretical Analysis
In this section, present the main theoretical analyses for CoRT, focusing on estimation and inference, along with their interpretations. Complete details, including the assumptions and proofs, can be found in the Appendix E and F, respectively. First, state some standard assumptions under high-dimensional multi-source settings.
Assumption 1. There exists some positive constant aw such that ()(()) S for all t R.
Assumption restricts v² in a bounded region, and note that both linear and logistic regression satisfy it (Negahban et al., 2012; Loh and Wainwright, 2015). The purpose of this assumption is to ensure finite variance.
Assumption For the k-th source, let (A) impose the following restricted strong convexity (RSC) condition
(D (B+ A) VLD (B). A). We
1. ((k) (A) Z 0'kl A
log
I V|| A 2 S 1;
TIR
nuk
2. (kk) (A) 2' A 2
T2A
logg
A A 2 1.
where O'j > 0 and Tyk with 4arjk 2 for j = [1 and k E S U (0}.
Assumption 2 assumes the RSC condition for the loss function in each source. This is widely used to study the error bounds in high-dimensional statistics (Loh and Wainwright, 2015).
S SU {0}, are independent and identically distributed (i.id) sub- Gaussian random vector with mean zero and covariance matrix (8), The smallest and largest eigenvalues of (k) are bounded below and above by constant, respectively.
Assumption 3. x:
The sub-Gausee Sob-Gaussian assumption allows us to establish concentration inequalities for linear functions of those predictors. Particularly, it deals with sparse high- nigh-dimensional data where only a small number of predictors may be relevant due to the exponentially decaying tail bounds. The bounded eigenvalues of the covariance matrix ensure the well-posedness of the problem and the stability of the estimator.
3.1 Main Theoretical Results for Estimation.
For CoRT with S, we have the following & ///tiesttiaaiion and prediction error bounds. Theorem (Errors for CoRT), Denote N := no + ns. Suppose Assumptions 1- hold. For k E S. assume that (| llo S s. [|6(() llo S C's for constant C, and (s log b/)p + hå (log
o1). Taking d x v /l, then
log $
log p
l estimation error: 13(0))((0)) S
(sh)
1/ h2.
log p
log
l estimation error: [|) -000
+
$
log p N
log p) N
llog
Prediction error: (3 3(00 s
+
(sh)+ 7
+
N

---

hold with probability at least cpp for some positive c. Here D2 (3((),, ,3(0)) is the prediction error defined D2 (3) ((0) (V Ve((o), -300)
Table 1: Comparison of high-dimensional transfer learning models.
Allow both convex and nonconvex PA,?
Conditions for improvement over Lasso with no < Rgg
Model
E upper bound
naive-Lasso
zloge Bi
x
(Tibshirani, 1996)
Trans-LLasso
sloge N
1/2
A h
X
ht <
log
1/2
(Li al., 2022)
Trans-GI.n
1/2
loR.P Thu
h2
A h
X
h <
(Tian and Feng, 2023)
Plu
CoRT (ours)
h+
/ h <8 (tttting log
1/2
Discussion. IF S sVlogp/N, then the & estimation and prediction error bounds become /s logg,/, which is the convergence rate of fusing all source data and the target data. Besides, when h & W/Nlogp/ng and no < n.s, the upper bound is sharper than implementing Lasso the target data only (naive-Lasso), i.e., v /oogp//oo. We present comparison between CoRT and existing two-step algorithms for high- -ith-mimeniionll models, as reported in Table 1. Specifically, when presenting an improved error bound over the naive-Lasso, CoRT results show it allows for higher heterogeneous source tasks, i.c., a larger h, compared to the other approaches. This illustrates the capability of CoRT to more efficiently leverage the knowledge from less "similar" sources.
Furthermore, the two-step framework, like Trans-Lasso and Trans-GLM, typically requires the model/parameter shift assumption, i.e., only the parameters ((k) differ across domains. To accommo- date covariate shifts, they require additional similarity assumptions (Assumption 4 in Tian and Feng (2023)) between the covariance matrices. This amplifies the error bounds by factor Cy; due to an asymptotic bias, which can diverge when the covariance matrices (k) are dissimilar. In contrast, CoRT avoids similarity assumptions on (k) and accommodates both shifts simultancously, hence achieving robustness against covariate shift.
Remark 3. Proving Theorem presents new challenges. The existing two-step framework typically applies Lasso twice, making its error bounds decompose naturally by each step. In contrast, CoRT requires different technical tools since the objective (1) incorporates both target and source data, which we must express via composite design matrix, thus introducing non-i.i.d samples under co-regularization. Bounding the estimation error of GLMs typically needs an RSC condition. However. the existing RSC condition for high-dimennsonal GLMs fails in the non-i.i.d case. We extend RSC to a non-i.idd setting and derive all subsequent theoretical results (see Appendix E).
We next show that Algorithm I can adapt to unknown similarity, i.e., achieving the same performance as knowing& S with high probability.
Assumption (An informal version of Assumptions 7). The heterogeneity between "similar" ((k) (k E S) and "outlier" ((()) (k: E 8 is sufficiently "large" so that they can be "separable".
Theorem 2 (Consistency of S). Let S be the estimated set from Algorithm 1. Suppose Assumption 1-
4 hold, then, for any 8 > 0, there exists n²() such that when minkesu(0) 2 n(8), P(Š S) 2
1 holds.
3.2 Main Theoretical Results for Inference.
We present analyses for error bounds of the target precision matrix θ and CI via CORT. To streamline the presentation, we provide informal versions of the assumptions used to derive theoretical results in the inference process, deferring formal details to Appendix F.
Assumption (An informal version of Assumption 8). The difference in the precision matrices of the similar source and target domains can be well-controleed by certain quantities hy. hr, and has Assumption 6 (An informal version of Assumption 9). Under some standard assumptions for dinensinmalionall GLMs, the CoRT desparsified estimator from Proposition I is consistent.
7

---

Theorem 3. Suppose Assumption 5-6 hold, and let R1 = (s (slogp/N) + (logp/N)åhá, and
{= R1 + If (logp/N)i + hf nē + h2 + hi. Suppose R2 o(no= ), then j = 1, :. ,P, the
followings hold with probabilities at least 1 - |
| - 0yll2 E, 1( dis Moreover, assume 1 (2)
jjو l 2 h1 + s²C.
θ jzl = (1()), then
Σ $ 6 vnolb; 5 NNO, 1), with
D (k) N 3(o)
A.
(4)
(E(DS
Compared to the convergence rate of desparsified Lasso (Van de Geer al., 2014), our improves ) hi << (~ () [8 A (x )²), no < N, 11 hi < 8t (logp) ////3////////nn when h < s( N 2)
"o
hg < ( (#log2) and hg << ( 2 Therefore, the CI constructed by CoRT narrower than the no
no
target-only ones. Again, compared to the two-step CI construction in Tian and Feng (2023), approach accommodates "less similar" source tasks to achieve the same desired improved CI.
Empirical Results
In this section, conduct empirical studies on synthetic data and real COVID-19 data to demonstrate the effectiveness of CoRT.
4.1 Simulation
In this section, we compare CoRT to the existing methods, including (1) naive-Lasso: Lasso regression on the target data only, which serves the baseline of our study; (2) Trans-GLM (Tian and Feng, 2023): the two-step transfer model for Gs.s. CoRT (Lasso) and CoRT (SCAD) represent the training with Lasso and SCAD regularizers, respectively.
Simulation with S: In this part, our goal is to show the performance of CoRT against naive-Lasso and Trans-Gia GLM given known S. For the target model, set (3 (0.5, -0.4, 0.7, -0.3, 0.8, 0195), where 0995 means 495 repetitions O. The target covariates x; are generated from (N Σ) with element Σ 5 = 0.5li-J' for = 1, -,,...,0 and 1 S j.j' S p. For the k-th source model, we set + Rp, where R, is p-dimensional independent Rademacher variable. The source covarite is generated from (0p, E Ec. with (O,,,, 0.32I,) and the same Σ as used in the target data. We set h E (5, 10, 20, 40), no E {50, 75, 100). and ni = 200 for all 1: = 1.... [S].
g B
Estimation results of logistic regression are presented in Figure 2a (Prediction results in Appendix G). The & estimation errors for both transfer algorithms decrease as |S] increases, and both outperform the baseline naive-Lasso, In addition, while the simulation settings satisfy the condition < 8 & log poggp//n no in Theorem of Tian and Feng (2023) in which regime the Trans-GLMM is better than the naive-Lasso CoRT also outperforms Trans-GLM which supports our claim that CoRT has a wide positive transfer regime against Trans-GLM.
We also examine the performance of CI construction through the precision matrix estimation, the interval length, and the coverage probability, where these three aspects are evaluated by (1) the averaged absolute error 1/p 1/
(300 θ Ojyl: (2) the averaged CI length of g (3) the
averaged coverage probability of 3000 for j == 1.... p. respectively. Figure 2b shows the results with different [S]. For linear regression, CoRT has the lowest estimation error of the precision matrix and the best coverage probability, particularly when the number of transferable source data increases. However, the CI length tends to be wider, which also happens to Trans-GLM For logistic regression, CoRT shows superior performance in all three metrics while desparsified Lasso and Trans-GLM are unable to maintain the nominal level 95% and have large estimation errors of the precision matrix. We include more experiments to demonstrate the performance of logistic regression, including Fly 40, h 30,p p- 800 Figure 2c support our results.
"Code is available https https://tet //g/:/gtttttt cwm http://hhshooluu/Trran--CCoo

---

1:000 CArT Joba cart piDcy rancaaoo
(a)
(b)
(c)
Figure Results of the CoRT and the existing TL methods: (a) Estimation errors of logistic regression with varying & and no- Shade areas denote mean squared error (MSE) + standard deviation (SD). (b) Inference results for linear and logistic regression. (c) More scenarios logistic regression. Methods with lower error, shorter length, and near 95% coverage have better performance.
Adaptive Transfer in General Settings: this section, we compare adaptive CoRT and source- detection Trans-GLM (Tian and Feng. 2023). We set K = 10 and simulate source data either in or St. Throughout this simulation, we s = 10, p = 500, and vıs = 100, 0, ..... 10. The target parameter is set to be (0)) = (0.43, 0.5g, 0.64, 0490) and E {20, 40. 80}. For the source in S, we let ((x) ( + h/pRp- For sources in S simulate g(*) by .f8²t 0.5 2/rr()) /p if j +s++11... 1..... 2s} } U rand(s) or B(t) 2hr, otherwise, where {r) (R) are Rademacher variables and rand(s) randomly generated index of size from (2s 1.... pp. Figure presents the le estimation errors of all methods under different h for both linear and logistic regression. Models start with "Pooled", which means that we directly pool all sources without adaptive procedures.
Two insights can be gained in Figure 3. First, by comparing the pooled and adaptive versions, that the adaptive versions always produce lower errors than the non-adaptive ones. This highlights the importance of excluding "outlier" sources before conducting knowledge transfer. Second, our CoRT outperforms Trans GLM in most scenarios. Besides, adaptive CoRT is guaranteed not to be worse than the baseline nasse- but Trans-Gla fails to present such robustness.
h=20
h040
h=00
h-20
hm
ho00
10
C5
nalvo-Lsso
PoRk
Poaled-Trans-B0
Figure 3: The l estimation errors for linear regression (Panel A) and logistic regression (Panel B).
4.2 Real Data Example
In this experiment, we use county-level COVID-19 data to study (1) we can improve predictions of state's mortality rate using data from other states and (2) if we identify states that are similar to

---

others. Details are available in Appendix G. Each time, we treat one state as the target and others as sources. We include two-way interactions of features, resulting total of 561 features.
Predictive analysis. We compare our CoRT to Trans-GLM and other algorithms, including random forests (RF), XGBoost, and support vector machine (SVM), which only randomly split the target data into 80% for training and the remaining for testing. The results are presented in Table 2. We also employed feature extraction methods, including principal component analysis (PCA) for dimension reduction and neural networks. Both have explosive prediction errors and fail to work in our application since the sample size of training data too small, around 50. Notably, CoRT performs the best in more than half of the target states. When CoRT is not the best, it still outperforms its interpretable competitor, Trans-GLM, in most states.
the target data. We
Table 2: The mean squared prediction errors for each target state. The mortality rate multiplied by 10 The bold numbers indicate the lowest prediction errors. Underlined numbers not significantly different based on the t-test.
Model
AL
CA
co
FL
LA
MT
ND
NY
PA
SD
W1
WV
SVM
0.28
4.43
9.26
0.21
0.15
6.91
4.59
0.68
0.24
4.53
0.29
4.26
RF
0.29
3.22
8.94
0.23
0.18
7.75
4.35
0.56
0.21
3.92
0.28
5.15
XGBoost
0.48
6.03
13.21
0.26
0.26
10.43
6.02
0.59
0.28
5.81
0.36
8.21
Trans-GLM
0.30
5.03
9.82
0.21
0.19
6.52
5.25
0.62
0.21
0.28
5.13
CoRT
0.26
4.35
9.02
0.18
0.19
6.14
5.22
0.52
0.19
4.66
0.27
4.69
Network analysis. We analyze the transfer- ability of the source states to the target states. For each target state, we use Algorithm to adaptively estimate S and CoRT to obtain 8(k) Figure 4 shows the transfer network after 200 replications, where presented directed edge indicates the source state gets selected at least 150 times, with thickness reflecting the aver- age magnitude of 12. We observe that (1) CA, MT, and WV have the most connections to source states due to their fewer counties, im- proving predictions with more source states. (2) Geographical proximity influences transfers, neighboring states share similar county charac- teristics. For example, GA is close FL and WV, leading to connections between them.
Storo
Targ::
Figure 4: Transferabilities among the states. The thicker the edge is, the more transferable the source state is.
Discussion
In this paper, we present Co-Regularization transfer framework, which enhances the multi- nouti-source knowledge transfer process in high- digh-dimensions. We show that the CoRT framework outperforms the classical two-step paradigm both theoretically and empirically. Moreover, the model-agnotic nature of CoRT makes it highly adaptable and can be extended to other high-dimensional statistical models, further broadening its applicability across various fields, including social and biological sciences. We also develop mathematical tools for such co-regularization framework, paving the way for theoretically driven transfer learning research.
There are also open problems left for this topic. For example, we notice that CoRT requires retraining when switching new target tasks, resulting in an increased computational burden. Distributed techniques, such as divide-and aonde--ndd-oouuer, can be used to practically improve computational efficiency, The algorithm for wwwid--and-conuerr in the high dimension regime has been well- dell-developed with theory to guarantee its performance. These distributed algorithms can be directly plugged into the CoRT framework in practice when computation expense a concern. Developing an alternative training scheme and leveraging the co-regularization spirit can also be of interest.
10

---

Acknowledgment: Li's research was partially supported by an NSF grant DMS 2514400 and an NIH grant R01GM163244.
References
Bastani, H. (2021). Predicting with proxies: Transfer learning in high dimension. Management Science, 67(5):2964-22) 2984.
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. (2010). theory of learning from different domains. Machine Learning, 79:151-175.
Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Wortman, J. (2007). Learning bounds for domain adaptation. Advances in neural information processing systems, 20.
Breheny, and Huang. J. (2011), Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. The Annals of Applied Stutistics, 5(1):232 253.
Denevi, G., Ciliberto, C., Grazzi, R., and Pontil, M. (2019). peerminggto-learr stochastic gradient descent with biased regularization. In International Conference on Machine Learning, pages 1566-1575. PMLR.
Du, S. S., Koushik, J., Singh, A., and P6czos, B. (2017). Hypothesis transfer learning via transforma- tion functions. Advances in neural information processing systems, 30.
Duan, Y. and Wang, K. (2023). Adaptive and robust multi-task learning. The Annals of Statisties, 51(5):005-20399
Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456):1348-1366
Gross, S. M. and Tibshirani, R. (2016). Data shared lasso: A novel tool discover uplift. Computa- tional statistics & data analysis, 101:226-235.
Gu, T., Han, Y., and Duan, R. (2024). Robust angle-based transfer learning in high dimensions. Journal of the Royal Statistical Society Series Statistical Methodology, page qkae 11
Gu, T., Han, Y., and Duan, R. (2025). Robust angle-based transfer learning in high dimensions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 87(3):223-745.
Gueuning, T. and Claeskens, G. (2016). Confidence intervals for high-dimensional partially linear single-index models. Journal of Multivariate Analysis, 149:13-29.
He, Z., Sun, Y., and Li, R. (2024). Transfusion: Covariate-shif robust transfer learning for high- dimensional regression. In International Conference on Artificial Intelligence and Statistics, pages 703-711. PMLR.
Hoffman, J., Mohri, M., and Zhang, N. (2018). Algorithms and theory for multiple-source adaptation. Advances in neural information processing systems, 31.
Konstantinov, N., Frantar, E., Alistarh, D., and Lampert, C. (2020). On the sample complexity of adversarial multi-source pac learning. In International Conference Machine Learning, pages 5416-5425. PMLR.
Konstantinov, N. and Lampert, C. (2019). Robust learning from untrusted sources. In International conference on machine learning, pages 3488-3498. PMLR.
Kuzborskij, I. and Orabona, F. (2013). Stability and hypothesis transfer learning. In International Conference on Machine Learning, pages 942-950. PMLR.
Kuzborskij, I. and Orabona, F. (2017). Fast rates by transferring from auxiliary hypotheses. Machine Learning, 106: 171-195.
Li, D., Gaynor, S. M., Quick, C., Chen, J. T., Stephenson, B. J., Coull, B. A., and Lin, X. (2021). Identifying us county-level characteristies associated with high covid-19 burden. BMC Public Health, 21(1):1-10.
11

---

Li, S., Cai, T. T., and Li, H. (2022). Transfer learning for high- dith-immensionll linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):149-173.
Li, S., Cai, T. T., and Li, H. (2023). Transfer learning in large-scale gaussian graphical models with false discovery rate control. Journal of the American Statistical Association, 118/443)21111-2188.
Li, S., Zhang, L., Cai, T. T., and Li, H. (2024). Estimation and inference for high-dimensional generalized linear models with knowledge transfer. Journal of the American Statistical Association, (19644)):274-1222
Lin, H. and Reimherr, M. (2024a). On hypothesis transfer learning of functional linear models. In International Conference on Machine Learning, pages 30252-30285, PMLR.
Lin, H. and Reimherr M. (2024b). Smoothness adaptive hypothesis transfer learning. In International Conference on Machine Learning, pages 10286-30316. PMLR.
Liu, S. S. (2024). Unified transfer learning in high-dimensional linear regression. In International Conference on Artificial Intelligence and Statistics, pages 1036-1044. PMLR.
Loh, P.-L. and Wainwright, M. J. (2015). Regularized m-estimators with nonconvexity: Statistical and algorithmic theory for local optima. Journal of Machine Learning Research, 16(19):55--616.
Mansour, Y., Mohri, M., and Rostamizadeh, A. (2008). Domain adaptation with multiple sources. Advances in neural information processing systems, 21.
Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34(3): 1436 1462.
Negahban, S. N., Ravikumar, P., Wainwright, M. J., and Yu, (2012). A unified framework for high- nigh-dimensional analysis of m-estimators with decomposable regularizers. Statistical Science, 27(():538-55 557.
Ollier, E. and Viallon, V. (2017). Regression modelling on stratified data with the lasso. Biometrika, 104(1):83-96.
Pan, S. J. and Yang, Q (2009). A survey transfer learning. IEEE Transactions Knowledge and Data Engineering, 12110)13345-335
Qiao, M. (2018). Do outliers ruin collahoration? In International Conference on Machine Learning, pages 4180-4188 PMLR.
Core Team (2025). R: Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
Rigollet, P. and Tsybakov, A. (2011). Exponential screening and optimal rates of sparse estimation. The Annals of Statistics, 3912)7731-77
Schölkopf. B., Herbrich, R., and Smola, A. J. (2001). A generalized representer theorem. In International conference computational learning theory, pages 416-426. Springer.
Sun, S., Shi, H., and Wu, Y. (2015). A survey of multi-source domain adaptation. Information Fusion, 24:84-92.
Tian, Y. and Feng, Y. (2023). Transfer learning under high-dimennional generalized linear models. Journal of the American Statistical Association, 1186544))28442269.
Tian, Y., Gu, Y., and Feng, Y. (2025). Learning from similar linear representations: Adaptivity, minimaxity, and robustness. Journal of Machine Learning Research, 26(187):1-125.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267-288.
Torrey, I.. and Shavlik, J. (2010). Transfer learning. In Handbook of research on machine learning applications and trends: algorithms, methods, and techniques, pages 242-264, IGI global.
12

---

Van de Geer, S., Bühlmann, P.. Ritov, Y., and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for nigh-dimensional models. The Annals of Statistics, 42(3): 166- 1202.
Wang, X. and Schneider, J. G. (2015). Generalization bounds for transfer learning under model shift. In UAI, pages
Zhang, C.-H. and Zhang, T. (2012). A general theory of concave regularization for high-dimensional sparse estimation problems. Statistical Science, 27(():576-
Zhang, Y. and Yang. Q. (2021). survey on multi-task learning. IEEE transactions on knowledge and data engineering, 34(12)5556-5609.
Zhang, Y. and Zhu, Z. (2022). Transfer learning for high- high-dimensional quantile regression via convolution smoothing. arXiv preprint 12.:2222..
13

---

Supplement to
"Co-Regularization Enhances Knowledge Transfer in High Dimensions"
Appendix organization:
A Notations
14
Two-Step Framework for High-dimensional Models
15
C Regularizer Detail
15
Construction Details for Confidence Interval
16
E Technical Details for Estimation
17
E.1 Assumptions
17
E,2 Proofs for the Upper bound of CoRT
18
E.3 Proof for the Consistency in Detection
21
E,4 Lemma
23
Technical Details for Inference
25
F.1
Assumptions
................................ ....uppioon.....................
25
F.2 Proof of Proposition
26
F3 Proof of Theorem 3
28
F.4
Lemma
30
Additional Experiments and Details
G.I
Additional Simulation for Section 4.1
31
G.2
Computational Complexity Comparison
32
G.3 COVID Data Details
32
A
Notations
In this section, we list the comprehensive notations that are used in the paper and the Appendix.
We denote scalars with unbolded letters (e.g., sample size and dimensionality p), (random) vectors
with boldface lowercase letters (e.g.. y and (3) and matrices with boldface capital letters (c.g., X). Yı
Tk=1 denote the multiple source data and let (x (0) y (()) be the target data. We use T to represent the transpose of vectors matrices, such x and X For p-dimensional vector x (#1. .p)). the lg norm is ||x||, | /2, the Eo norm the number non-zero elements, and the infinity (maximum) norm is | oc max; [z,]] Let [M] denote the cardinality of the set M. and set with superscript denotes its complement, for example, M Let a and b denote the maximum and minimum between and b, respectively. We use letters C' and with different subscriptions (e.g., C1 and c₁) to denote the positive and absolute constants. Let on = Obb,)) denote (a,/b,l I S for some constant e when n is large enough. Let an (b...) and I b, denote P([an/bu /bml < P((a,//u c) - for O. Finally, On x bn means that an/b,, converges to some positive constant.
Let (x((*) yx) x R y, E R Lk"
for < co. Let = op(b,) denote
14

---

B Two-Step Framework for High-dimensional Models
In this section, we give a general deseription of a dominant two-step framework for transfer learning in high-dimensional models advocated by Li et al. (2022); Tian and Feng (2023). To facilitate comparison between this framework and our CoRT, we introduce this framework following the exact settings in Section 2.
Given the known S, the Trans-GLM, (Tian and Feng, 2023) estimates 3 through the following two
steps by setting 3 a (8) 80)) where
1. (Pre-training):
(1S)) arg min
Σ ((B,D(4) + X ||ll
ns + no kESU(0)
2. (Fine-tuning)
8 (0) arg min
- ((()()) 8, D(0) + 2211011
nio
Here, ((3,D(k) is the negative log-likelihood over k-th dataset with parameter B, i.e., 1 ETh (xx T (x) 3) - yi (x] (T)B)) (). mE
(k) (1)T
The idea underpinning Trans-GLM is that for the source domains in S, the population version of the parameter 3 mixture of (8)) which is sparse and thus should be estimated with ét_ regularization. Besides, if g is similar to 3 i.e., learning ,30 be accelerated by using the results of learning 3 the contrast (0)) should be sparse as well.
Under certain assumptions, the ll and l estimation errors are
log p
logp
l estimation error: || -3(00) 112 S
A h
log p
log p
1/2
llogp N
& estimation error: (1) /300 S
h A
+
A
C Regularizer Detail
In Section 2, we claim the regularization function Px, and Px, can be a general class that includes many famous regularizers, including Lasso, SCAD, MCP, etc. In the following, we provide specific conditions that the regularizers need to satisfy.
Condition 1. We assume the regularizers Px, and Px, that used CoRT satisfy the following conditions.
(i) (O) = O and Px(l) is symmetric around 0.
(ii) Pt(t) is differentiable for + 0 and lim,- (t XL.
(iii) Px(t) is non-decreasing function on t (0).
(iv) P is a non-increasing function E (0, oo).
(v) There exists 7 > 0 such that Px(t) + 7,22 is convex.
Remark Conditions (i)-(ūi) are relatively mild and used Zhang and Zhang (2012). Condition (iv) makes sure that the bound of error ||3 B is vanishingly small. To incorporate the nonconvex regularizers, we include convex side constraint that satisfies R > g(33) > |13],, where & lies convex set and is a tuning parameter that can be chosen to be proportional 1/X (Loh and Wainwright, 2015).
2||221
15

---

D Construction Details for Confidence Interval
In this section, provide the detailed process for constructing clement wise CI for ( We begin by defining some notations. For any IR,, we define the k-th diagonal weighted matrix as
w (k)
diag
( 1
(K)T
0),
6a (xNN (R)T
01)-
the k-th weighted covariate
w (k) 0
(k)
and the empirical version as X (k) 0
Then, we define the k-th population weighted covariance matrix
and its empirical version as
Σ = x (K)T "TTT"( (x(
E(A)
7ik
As stated in Section 2.3, construction process based on the following two steps:
Obtain an estimator for the precision matrix O
2. Obtain the unbiased estimator of ( by desparsified technique. In the following, we present the details of each step.
Algorithm 2: Node-wise Regression for Constructing b
1: Input: source data (D(() = (X (k) (k))), k E S}, target data D)()) = (x (0) y())). (()))
2: Output: The relaxed inverse θ.
3: Obtain (00) by CoRT.
4: For each & U {0}, we stack X ((k) horizontally and obtain N x p covariate matrix (i0)
X ((0.1 (S(H)
((0)
5: For each j = 1, , calculate the parameter 7, of the node-wise regression of X (10.1-..181))
300)
over ((0011...18))) via Lasso, i.e. g
9 arg min 7
Σ
x ((0.1..;181))
((00 (111)
117))>11 /11 + Ablbille B
2(ns + no)
kE
6: For each j 1.... . calculate
22 = 10)
(0)
0j
0.j.j
0.j.-
7: Denote 3, = (f 115-11 9sst11 as follows,
(ppp)) for each j = ... ... P. then construct the C
2122
22,1
F2.
θ diag (fo ...., 2)
(5)
Pp.1
- Pp.2
Construction Ô via Node-wise Regression. this part, we show the construction details of é. This problem has been widely studied in the literature, and there are several techniques to construct relaxed inverse of (1)), and our approach is based on node-wise regression.
For k-th weighted covariate, further define x (k) 0.j the j-th column of X (k) 8 and X (k)
0.-j as the matrix without the j-th column. Further, define (*) the j-th row of 2 without the diagonal
(R)
(j.j) elements, and (k)) the diagonal (j-j) elements of ((k)
Given these notations, now present the construction process in Algorithm 2.
16

---

Construction of the Covariance of the Desparsified Estimator. The following algorithm shows the procedure for constructing the covariance matrix of the desparsified estimator b
Algorithm 3: Covariance of Desparsified Estimator
1: Input: source data (D(R) (X (k))), E S). target data DDO)) y(o)).
((0)).
2: Output: The covariance matrix Σ (000-
3: Obtain (00) by CoRT.
4: Obtain the relaxed inverse O via Algorithm 2.
5:
Denote Σ3:0)
Ekefoyus De (() and the covariance matrix of ((0) (:0)
Eg0 = é Σ B
θ
The construction of Ô and Ego originates from Van de Geer et al. (2014), and we nofice that Tian and Feng (2023), who also considers the transfer learning under high-dimensional GLMM, presents a similar procedure. However, we would like to highlight the difference between our CI construction method and that of Tian and Feng (2023). In Tian and Feng (2023), the transfer learning approach is two-step process, and accordingly, the node-wise regression to obtain Fy is also two-step. In contrast, our method only involves a one-shot loss minimization obtain fy, as we apply co-learning procedure.
Remark 5.
With Algorithm 2 and Algorithm 3. the construction process of element wise CI for (00) can be established in Algorithm 4. The general idea is make sure the source tasks in S can be separated from source tasks in S
Algorithm 4: Confidence Interval for the Target parameter
Input: source data {D(k) kes. target data D(0), and confidence level (1 - a).
2: Output: CI ),jj 1, ,PP
3: Obatin the target task estimator (0)
by minimizing the co-learning loss (1).
4: Construct the related inverse é, via Algorithm the Appendix, which leverages the node-wise Lasso technique over the target data (0))
5: Obtain the desparsified estimator (0) via (3).
6: Construct the covariance Σg of the desparsified estimator ((0) via Algorithm 3, which applies covariate reweighting procedure over (D(*)) for k {0) US.
7: The confidence interval of (3 :j =....... i
21 21-0/2 ( dis 610) + 21 21-0/2 Fros )i
n2
Tio
where 21-a/2 is the a/2 2 left tail quantile of standard normal distribution.
E Technical Details for Estimation
We begin theoretical analysis with the estimation of CoRT and the detection algorithm based on CoRT.
E.1
Assumptions
The following assumption is adopted from Assumption Tian and Feng (2023). We refer the readers there for details and the requirements for it to hold.
17

---

Assumption 7. (Identifiaillity of S) Let E( D(g) and E(B(O), ) denote the 1-th fold cross validation errors with ((0)) and ( respectively. Assume for some h and for all k.
sup (((3)0)) 1( ((B(0) D r + +cI(1) (c)
sup |((B(Ok) ( E(3(0) 1/ ) (T S 92 (c).
For l E S", assume X = infg Aminl (E( ((((1=) 3:0) +tx, (xxx)((()ddt dt > 0, r 0(1)
(kk] O(),, and for sufficiently large C₁
[ - -1 1 Ci (vT!0) v /T (iovvv) + (2T
For 8, assume ( ( + { h22 O().
We provide the explicit forms of T ( 1 91 (S), and 02 (8) (C) in the proof of Theorem 2.
(k)
E.2 Proofs for the Upper bound of CoRT
Proof of Theorem I. We start by proving || l2 < 1 with the appropriate choice of À and sufficient
n. Note that the RSC condition has
log P N
& & A = VC, A 2 22ll 72
||1.
(6)
The first-order condition returns
(3) - 20.
(7)
Thereby,
log
(- (P)) - VC, (B). Á) 2 az Å 112
T2
| 1.
(8)
N
Holder inequality and the triangle inequality imply that
(-VPA()) -2, (3), S ( +IVC,)))++++ |
18

---

Lemma 4 of Loh and Wainwright (2015) has || (3)) I XL, so the next step is to bound
(B3]lc
max
(x (3 ys) Fij
n
nk
o' (3 Vi (1))) Iij
max j=1,- xp"
Zij [zaks. ... ,0. Fkiy)
n
(U)
(x (K)T (B(A) -3( x
(T)T ((0)) 1i [ rkis
max
j=1,--
Ro
(x: ( ((0)) y/ [0o IoN
max
j=1,n-p
T1
e (x (R)T 3(h)) yf (2)
1 (x( (0) ((0) - -y; (0) rois
max
Ikíj
max
1=1,
j=
n
kES i=1
2 (xc (R)T ((aa) v6aa Ikis
(b( (O)T ((0))
34 (0)
max
+ max
rosy
KES 1, sP
1, P
x
1 no
max j= 1 P
1 (xx(1)) g(t)) v. (K)
2
max
Σ
/ (x}
(0)T ((00)
zoij
n
Th
n j
1, 7
no
KES
ng log
Ck
n²
(E)
logp
N
where the last inequality is due to the fact that each
(x (M)T 3(6)) - -3²² -bi Ikis
(1)
log p
max
S ck V
1=
Tkk
with probability at least
C1A exp(C2x log Therefore.
(-VP) VC (B) A) S (AL Al1
log P N
022 Å 112
Å 1 S
(AL +
T2
(TA+ LA
Cm A)
2R (T2 + + Gm)
| S
| -
a2 where 6m = max Cb. is at most by carefully choosing T2, L, and Cm- Finally, we would focus on the case
log p 2 N
E, ( 2 a1
(9)
Since Px, (0) + [113113 is convex.
Px,)(3) (A,)(B) (VPx, (VP, A (V) + 13, A)
implying
(V.) A 2 PA) PA(3)
T4 A
(10)
19

---

Combining (7), (9), (10), we have
a logg i & - S (VP) A (VL.,.A)
N
<PA(3) - P3(3) + / 1 (V,,
Thus,
a1
) 2 ق < PX(0) P3(3) + (3)1 |ل|ا| +
log P
| ||2
N
S PA Pa(3) + (IV2, +2R
log
|
n
Px(g) Px( + (4 4
(A(AA)
AL
2XL
= PA( - (3 +
2 (A + #1 ا
By taking μt Ti, we start from the bottom of Page 574 in Loh and Wainwright (2015), Let M be the support of (3, then
- 301-3// & PX( - +1/2
-Px(3) - PA( (3 +
SPA( - M)) 1/22P
-3/2PA(A (2 1/2P
3/22P M) - (A
( (A1))
SAL (3/2 A ll 1/211 (1
where the last inequality follows Lemma (b). Therefore, we have
(201 3n/ /2) Á S AL (3)|A2 la [12-111]
<3AL Á 1 + ALKI
33A $ Á ||2 Ah.
The proof is then concluded by solving quadratic inequality. The & prediction error:
The prediction error is upper-bounded such that
(VL.)) (L..) Âo) (-VC,(88)) VP)(3 Á
<P)()( (A(0)))) + &ol + \V
Next, we have the following bounds:
() () < XL|| and |[V',(((0))]| o0 S A.
Finally, the prediction error
(B(.)) (3) Ao)
SALIA Polli + T - 11 ملل
SAVe ||2 + Å 1
20

---

The result follows by plugging in the & error bound in Theorem such that
D x
slogp
+
log N
Vah
by
logp
N
E.3 Proof for the Consistency in Detection
Proof of Theorem 2. We first obtain the Explicit forms of T _1 11/h), ( and 92 (8)
(C). (C).
Let (*)) be the parameter for the k-th source model. Denote the g the probability limited of (()) for population version loss function, and its corresponding estimator ((0k), The following
forms can be found in Proposition I of Tian and Feng (2023);
log no
k 0
1/4
2
log p.
log
Vh Vsh,
k E S logp n
nno
1/4
h/
log
s' log p WR +
VWWWs
k E S
where Wx 1 v
v / -
The followings list the forms of (()), (r(,, r (2) (C), and
92 (k) (C).
(i) Linear model:
r-(0)
s log
| 1|,,
1 (1156112 v 110" /Vno.
no
(R)
(44)
[(k E 5) + 110o11121 lI E s9)
r((k)
1
g
I E) 11 (K 17)
V no
2
(R)
1
(134
11 [(kes) (1 |2 [ T E (os
V no
(K)
(C) 92 (c) (xxp(-(),, x: + 0:91 (0)()) exp(-<"),.0) (0) (C)
(k)
exp(-no) +exp( exp(((²).
9i
(ii) Logistic model:
(0)
slogp P
1
|13(0)
/12/ /
110
T Utt
r (k)
I Ux
1
[1B (k: e 5) +)+ ( |
I(k E 57)
n10
no
91 (0)() = 92 (C) exp(
(k)
22))
21

---

Let ((B(0k)) D( and (( D(e)) denote the 1-th fold cross validation errors with _(0k) and 3
respectively. Note that they averaged by the testing data size. First, we have the following
Ufc (B0x)) <c(3r))
c (310x)
= c (3100)
((8+5)
WES
RESe
P C (3300))
p C (3300A)
S+1
Σ RES
+ Σ
2
2
2
KES-
Σ KES
P
1 ((3(0)) )(T)) S ((3(0) D(()) <
S+1
S
+
2
BES
Σ kEgc
P
Σ
(R(3(0R) ) S E13(00
((3(0)1(0))
D(0))
S+1
2
BES
S ++1 1
Σ KESc
2
E' Σ
((B(RR) (RR) D(o)) S -1000,0)00)))
2 Pe
+
S+1
kES
SES
1
{1
P (e(3rUN)
) E (0)
( s =0) ))} +
5
[t]
[1]
KES
#ES
2
ΣE P (((3(O))
(0A) D/0)) f ((300 })}
S+1
KES
SES
where the last inequality is due to Lemma 3 and Markov's inequality. Next, we need to prove, for any s,
(i Pe holds with 1( ((300) 1 < 20) k E holds with a high probability.
low
probability.
In other
words,
P (E(B
(ii) p ((BBBUu) ) E(310), D(0)) > 0) k E St, holds with a high probability.
Intuitively, for k E S. 3000 is obtained with a larger sample size than (00), so we expect a lower cross-validation error, i.e., (() (4) {(3(0) D I O. For k E S (30) is obtained with
data deviating from the target data, so we expect a higher cross-validation error, i.e., ((G(0)), D/0)) ((3(0) D(O)) > O.
For (i), using claims in the proof of Theorem 4 in Tian and Feng (2023), we know
|((()100), D(O) - (()(0) D(O)) ) S c ((r +I + n²] n0- 0, sup
for k E U (0). holds with probability at least 1 eexp( (22)) For (ii), from the proof of Theorem in Tian and Feng (2023), we see
inf ((@(0R) D(() )- - ((300) (D(0) > Colo v1)
holds with probability at least - 2 [exp( (xpp(-co) + exp ( for Co > 0 and ( > 0.
Combining (i) and (ii), we have
9
S) <
1
ΣΣ'
2 ecxp( (xxp(-5))=0) + S+1
2
2xxp + +2exp(-(3))
S
KES 1=1
KES s=1
25
=||5| exp( (52p
+
|59 [exp(-Co + 2 cxpp (x)]
S 5+1
22

---

For any 8 > 0, there exist Co > 0 and Ç > 0 such that
2/8| 1iseexp(-(3
8
25
S [2exp(-Co3) + +2exp(-<3) - 7/2
8 2
2
S+1
In summary, for any 5 > 0, there exist Co(o) > 0 and ((6) > 0 such that
P($= 5) 2 6, as an - 00, k E S (0).
E,4 Lemma
Lemma 1. Suppose Assumption 2 holds. Let n > 4R rî log p for some positive constants 01, 71 and R proportional to 1/X and A 3 B, then
logp
2
A 1a
S
1
(1
(A)
logp N
| 1l 2 1.
(11b)
Proof. For any A E RR, we assume A 1 2R since 33 and A lie in the feasible set. We first note that
g(h) (A)
(DDw
(3 + + - VLD (3 A
1b'
(1x
(k) ,g((k) A)) v ('((x! (/())))))))))))))))))
x
(R)T A
bb (xx"),((0) + thi (x) (k) ا)) ((x(")) 4))²
nk
by the mean value theorem, for some the E [0,1]. For our model, let A (AI. Re and (k), E Rº' denote the i-th observation of the k-th data in X. Note that x; many Os as constructed in X. Then have
A k AJ
contains
Σ 36" (xx(1) (9) + txx(x (1) A)) (ef(,,,)) +
E, (A)
n
li=l
RK
1 (xx(")) ,3) + txx (0) A)) (1x((t) Ai)²
(xx!0) .3) + tox(1x 10) A)) ((xx .i)?
1
fl
1/"" (x g 115x(22 (1)T
(1)T (Ai + Ao)) { {x(1iT (Ai 2ol) +...
n
1=1
70
(D)T ( - 1nxx (O) A0) (x
(0) An) 2
x;
1
Eng
A + Au) .... ((k Ao) + Eoo (Ai)
n
nkk
Aoll Th
llgg (80(h + Aollz118(1) Aollz
n
RES
+
no
aul
A 1
Tu
log
A A
23

---

where the last inequality follows the result (Eq. 65) of Loh and Wainwright (2015) that
logp
(B(h)) + +Do) 2 axll0 + 2oll Th
1 (k)
+ Aoll110(AA)
+ +Doll2, V)O + Doll2 S 1
log P
(An) 2 Aul A01.
To
VI 1
Yau
with probability at least 1 - C1 exp(-o2m4)) k E S U {0}. For simplicity, use ar and Th for aik and Tike respectively.
By the mean- mean-geoment geometric mean inequality, we further have
logp (k) + A011110(())
+ Aoll S
ak
118
(k)
+
Aoll2 +
22 log (10(()) + A
2
200: TAk:
log
Dol2 S
00
A 11
TO
log Aol?
2
200 no
Therefore,
(A)
7kk
OKk
(8)
Aoll2 12
2 log (k))
Aoll
no
000
Aoll
log Aoll
n
20k nk
n
20 nu
KES
- KES
a' 1]6(k) + Doll 1
log
(10(h) + +Aolf
+
(66
A 11
l
||0ll
2
2ax
N
2
200
N
where a' ng05/n / and a0 ngao/n n.
Replacing A by Å
B. we have the following results
a'
|1@11
(8
3 12 + 2
(1 (1S) + 1))1 - -/00
2
2
KES
(12)
a'
d a' _ (IS) + 1)
Σ
Á +
2
KES
by taking aú = a'( + 1). With the definitions of a' and agj we choose as = mI5x(11) + 1)/no-
Denote Tk- and a₁
for all k E S,
Σ KES
The logg
Aolii S 72 loog P
Σ (1 11 + ll
2ak
N
2a
N
KES
72
logp
72 loo
2
+
2a N (|8) - ooll
2a
N
(13)
7 log
72 logg
2
||||| A
2a
N
2a N
logg
200
N
by taking To vI8I
1r. Finally,
To
2
log
7 <
To
_2
log P
1.
(14)
200
N
200
N
Adding Equations (12), (13), and (14), we have
E, (
a'
To
2 log Å
1
ag
N
log P A
= a1
T
2
N
where we take on = a//2 and Ti = 0 oo. This establishes (11a). For (11b), by assuming
2 4 log p. then it holds according to Lemma 8 of Loh and Wainwright (2015).
L
24

---

Lemma (Lemmas 4(b) and 5 of Loh and Wainwright (2015)), With the regularization function Px satisfying the conditions (i)-v)).
(a) For any w, we have \L|W S. (.(w) + T/2
w in magnitude. Suppose e > 0 is such (b) Let I be the index set of the s" largest elements of that ( Px(W1.) 2 0, then
cPA( (w Px(w1s) XL (cllwallz l|w1=ll1)
Lemma 3. Let I, = T(e(B((a))
1(1)) < E(3(0), D/ ((3( )) then
8
S+1
S+1
P
1, <
2 Pe
2
where Pe is the upper bound of P(1, = 0).
Proof.
S+1 I
P
< 2
8=1
S+1
S+1 1
Σ 1. <
is - +
Σ 1.
1s 0)
8-1
S S-1 1
=P
Σ
1
+ Pls
2
S
=P
Σ
1 <1 +
Σ
P(4=00)
#=1
8
0 1 = 0 ==1
+ Σ
PTg 0)
<P
1841 =0)
P[.,=0)
543
S+1 Pe-
F Technical Details for Inference
F1
Assumptions
To develop the theoretical results for Algorithm 4, further suppose some additional assumptions. Throughout the paper, the matrix with subscript ( means the weighted matrix by B. Matrix with superscript (A) is for the k-th data, A S {0). Matrix with subscript j denotes the j-th column. For example, (k) Bag denotes the (j.j)-th element of the weighted covariance matrix for the k-th
data, with weight Bg. We also define
arg arg min F. (3" (x (1 = =(=/) (E()) B. - 3.j
-1 (A)
(15)
7
Assumption 8. We assume the following conditions hold
a) supses.j=l
(0)
15(00) (0)
(k))
(k)
- (k) (k)
S h1.
g
-
((0)
-jj-
g(a)
g(0) -
25

---

b) supkes.j=l:p
(4) 3300 jن
-0)
-
::0) Jid
/ (2/0)) ".:
-300) .h- - g00
-x
mos S
sh2
c) s1pp=1:p 1/0 -
(5(0)
(300) J.j - D (K)
((o) -is S h13.
Remark 6. Condition a) implies supNes lop 1b (6) 7; 112 E h1, i.e., the maximum distance between parameters of the node-wise regression from the k-th source data and the target data is bounded by hy (Tian and Feng, 2023). Condition b) bounds the variance difference between the k-th source data and the target data by h2- Condition c) constrains [Tj - -Tj S h3, which is for the logistic regression only. These three conditions quantify the impacts of source data in node-wise regression for Theorem 3.
(0)
Assumption 9. We assume the following conditions hold
(i) SupRESU(o) |xx(*) 1o00 S UU- < cc, supREsu(0) (x 3(h) S U" <
00 a.s.
(ii infRES)(0) Amin E B(0,)))) 2 O.
(iii) (U) sup so 8² (x( ((h) + 2) S M인 < 00 a.s.
(iv) There exist C > and C" > 0 such that ||,, S C and 1o S for 1,, P.
(v) sup, $j S where sj 11y, Ilo V 11 lu and 1/7] 0(1)
(vi)
R1 + hi
log N
+ 1R h + h22 + 0a + no
logn N
+
log p N
(sh)
= =(no 2)
[ + 1 Ri + (shiB1) -1 + + (12 ho)
0(),
>> gio I sC
for a positive constant C.
Remark 7. ltems (i)(i)) are the usual assumptions needed for the high-dimensional GLMs. (ii) makes jj stays away from zero. (iv) avoids the diverging Y for linear model and diverging You for logistic model. sup; Sj S in (v) is for technical simplicity, the same assumption in Tian and Feng (2023). Item (vi) guarantees the consistency of b, which is part of the proof of the asymptotic normality in Theorem 3.
sure that O -1
F.2 Proof of Proposition 1
In this section, we prove Proposition 1, which states that even (0)) is obtained from a co-learning scheme, its desparsified form takes the same form as the single-task training scheme, i.c., the form in Van de Geer et al. (2014).
Proof. Before we start the proof, we reformulate all the datasets in the following way. Without loss of generality, we assume S {1, ... ..K
} in the proof of Proposition 1.
X 11)
0
X (1)
X 0
X (2)
X
E NXP
X
X
(K)
(0)
where p² = p(1+(111 (S)). Besides, recall that for ls E US and ((R), define the weighted version of (4)
(k) g(4) V diag (x)((x (K)T ( ... (x (K)T
((0)) x (A)
26

---

and we can define the weighted version of X*
as
(-1)
X (2)
(())
g(n)
X (2)
0
0
3
X
0
g g
(k)
X
g(k)
X (0)
(
Therefore,
(
(1))
g
11
g
(2)
12 Σ
(2)
T2
8
1
A B
Σ/3
X
x
7
3
N
K (K)
(K))
C
D
0
g g(2)
grkn
n (())
(K)
> ΣMESL(O) n4 (0)
g())
12
g
((0)
3
where
(0)
T2
g
0
S(1)
(2)
mi
g
0
T2 Σ
0
A
B
(R) g
(K)
0
((4)
Besides, = B and D [kESS ny
(0) 3
We also introduce the coordinate-ve wise representation of a function, i.e., for vector R and function 6 R Φ (x) (d(x1), .(zp))
With these notations, without regularization terms as
rewrite that loss function Lo (B (3(),
(8((0)
JAES)
LD (B)
( B00)
((11)
B(0)
-(Y(U) ... Y(OT Y
+ +11
1) X
(3(K
(())
g(KK)
((0)
(300
(
which is in a single-task loss setting.
With this reformulation, we can apply the desparsified form of GLMs (Equation (18) in Van de Geer al. (2014)) directly, which is
b B
03 CD (3 (10())())))) JeEs).
and the desparsified form of (3) is by taking the last elements of b.
The proof of Proposition 1 then completed by using the following equation
A B
A -
B(D CA-
C
D
- -D-CA-1B)-CAA
(D (D-CA--B)-1
and
Y1))
30) (0
0 CD (3 (8 X
- 0 X*
03
Y(k)
B(K
(0)
(()
30
and then taking the last p elements.
27

---

E3 Proof of Theorem 3
The formal version of Theorem 3 is stated as follows.
Theorem 4. Denote R1 (s (slogp/N)) + (logp/N)áhž, Assume Assumption 1-3, 7 and 9 hold and R2 = o(1o ²), for 1, p,, the following hold with probabilities at least 1 Kno
logg N
| ,/1l x Rı + hi
h R2 122 + h3:
logg N
b0o 3as
θ s h1 + 5b
R1
+ h
+ R hi + h22 h3
Moreover, assume (( Dās - -,, = op(1), then
vno(b - )))00)
S 10,,
v (
Proof. First, note that F3 4f3 for linear and logistic regressions due to the property of the weight matrix. We bound
| , -Csh- CC/T311
$ 179 - 7/l11/ + + - 2) < 19, - + 17 7j ?) $ 2 Rı + (shi R1) h + a (h2 + ha)
since 13 is consistent estimator of 1/7 0(1) and 17,11 O(v3). Next, bound
| < ||7, 7
+177512((t) - Tj 2) 1
logr" N
< R + n
+ h R + h22 + hg-
Next, we bound the estimation error of the precision matrix
10, 1
& 300) Ô, - - S |6) { ($/300 θ >
9,) + +((0, -0,) [0) 0
B
28

---

First, notice that
Σ glo3
θ j
_
Σ
3 0c
1
Xki "8" (xx Ba) - -8" (xx (xx())Tp 3101
X 22
n
10
2
(4)
x;
(k) 26" (x: (k)T ((0)) x (A)T
[7/ - 7sl 603
+
Σ
x;
"0 "² (x, (xx (())
(k)T
x; 4, (f0) r5)
(b)
n
1,
i,k
(6)
(6)
15 (xx ( ((0))
[k)T Y x
7
Σ
E
xi
< (xx (K) ((0)
(k)T
x;
(k)T Y
AE (OJUS,
(7)
akk E
xi
(k)
1b"
{x]
(K)
((0)
x
(R)T
V
7
X
[0)
v
x
(0)
(T
((0)
x
(T)
7
7
7j
KE (OJCASA
(8)
(16)
To bound
( E 3300 θ, O5l we notice
(4) x 11)(00) -300 S RI.
logp
(5) S 14, - 7 - 1003 TT 9 R + h
+ R2 h + l22 + h3-
N
1/4
(oog N
(6) < 1002 - 15 21 1 R
+ +h
+ h
R
+ h2 + h3.
(8) x sup KES
Σ (0)
g oo
Yj
S h2.
Therefore,
(logr' $
② j
Σ gco) θ,
()) ()) g(o) ,,)
- 5
R1 + hī
+ R h + th2 + ha
N
since @0111 S st. We bound the second term
(6,, θ, [0] g
θ 2 ||,, |
S 8 R₁ + (shi R1) 2 + h1 + 85 (h2 + 113)-
Finally, we have
$/2
log
6 Σ 3 θ,
11و S h1 + + R1 + hi
+ RR h h2 + h2
N
To show the asymptotic normality, we can decompose b, - -1(00) by
b, - 0/00
e, -- 0 (g. Ba) + no
θ ()T [y(0) vb (x (0) (310))
(1)
(2)
+ θ x (T)T diag (fa5 (i!"))
35" x
(O)
3(0)
n00 ),T0)) (0) (B, - Bo)
1=1
(3)
29

---

where u falls on the line between x (T) 3(0) and x; { 3(0)
Jj + no 3) 11 - 3 11
(1) x 114, - -ryll + [oo
logg
R1 + 11
+ Ri
/1 + 122 + 113 + no
N
(17)
logp
log
x
+
(sh)
N
N
-
o(no"
under Condition 6 (vi).
no (2) - ((0,0))))
(18)
- (3)
1 + O 0 |1B100 - 11
2Λ
+ 2 Rı Rs) hı + s² (h2 hs) R?
(19)
=(no +)
under Condition 6 (vi). Combined with (17), (18), and (19), the result follows.
( xlogg
1
Final result: Denote m223 = logistic models, we have
v h2. Combining the results of linear, Poisson, and + h3
logp") N
)
| Oy1l S Ru + 1
+ h R + m23-
/logp N
$
6 Σ Boo
θ,و h1 + 5ž
Ri + h7
+ R h + m23
noo 66; - -1/00) S,(0,1).
V0 T 8 3m θ,
E.4 Lemma
Lemma 4. Assume Conditions 1-4 and C6 (i-iv). Les X3 x ( (log2) + ( (1)) for any
j ,.
.p. Suppose h1 S sT 1, then with probability at least Kno
log N
1R, 77,112 z hı
+ R2 Rihn
119, - >,,11 S + R₁ +
s
h R + h1
The proof is very similar to Lemma 4 of Tian and Feng (2023). Note that our 7, is denoted as S,5h in Trans-GLM and the details to bound it are the same. The result is determined by the bound of || - -3
||,, i.e., Rg We replace our bound of (| - (0) ||2 to get the desired result.
Lemma S. Assume Assumption 9 (i-iv) and the conditions in Lemma 4. For the linear regression and logistic regression, if conditions a) c) hold, then
t + +hi R7 + h2 + h3.
logp N
1 - 7 1002 T 2 R1 +h
30

---

Proof:
Recall that -6 (x))
x (0) 7,,) [x) /no and -0z [
-x
(0)
/no- We decompose
1oos r - 1oz Tos + TTo T
(1)
(2)
For (1), we can use similar techniques of S.2.4.2 in Tian and Feng (2023). Note that (1) does not have the weight and uses the target data only. Therefore, we can show that
logp
(1) x R1
thi
hi Ri + h2-
For (2), by the assumption,
| - + = 1xx] (0) -x (0) {
[x, x
(0)
/no - (x
((00
-x
gi0)
,
x
xx(
Bion
/noll
10 [0) /,157/ -1-/0) -00 Σ (0)
300) (5)
h3
as assumed in Condition (c) in Assumption 8. Therefore, the result follows.
G Additional Experiments and Details
To implement the methods mentioned in Section 4.1, we use package gimner for naive-Lasso, R package glmtrans for Trans-GLM, and R package ncvreg for our proposed CoRT (Breheny and Huang, 2011; R Core Team, 2025). Loh and Wainwright (2015) shows that the coordinate descent algorithms in Breheny and Huang (201 1) guaranteed to converge stationary point and matches our theoretical development. All the simulations are on a desktop computer running Windows with Intel Core i9 CPU at 5 GHz with 32 GB of RAM.
G.1 Additional Simulation for Section 4.1
In the known S case setting, we only present the & estimation error for logistic regression. Here, we include the AUC score.
call Laap cart DEAC) Ii rancasso TEU
Figure 5: Averaged AUC scores for logistic regression.
In the unknown S case setting, only present the l estimation error. Here, we include the prediction error for linear and logistic regression in Figure 6 for completeness.
h=00
80
89
Model
CoRT
natee-LLasso
Pooled-CoRT
Poolled- Trans-GL
Trane-GL
Figure 6: The MSPE for linear regression (Panel A) and averaged AUC scores for logistic regression (Panel B).
31

---

G.2 Computational Complexity Comparison
We conduct the following two additional experiments investigate the computational cost w.r.t. K and p. with the same settings in Section 4. For our proposal, we consider two regularizers, Lasso and SCAD. In scenario
fix p = 300 and let the number of sources range from 10 30. scenario 2, we fix 50 and let range from 300 to 700. Results are in seconds and averaged from 100 replicates.
Table 3: Results for varying source number K.
K
Trans- Trans-GLM GLM
Naise-Lasso
CoRT (Lasso)
CoRT (SCAD)
10
0.41
0.11
0.24
0.54
15
0.53
0.11
0.40
0.80
20
0.66
0.12
0.67
1.37
25
0.91
0.11
1.01
2.03
30
1.24
0.11
1.51
2.99
Table 4: Results for varying dimension p.
Trans-GLM
Naive-Lasso
CoRT (Lasso)
CoRT (SCAD)
300
0.41
0.11
0.23
0.54
400
0.63
0.10
0.49
0,88
500
0.74
0.08
0.92
1.49
600
0.92
0.08
1.54
2.28
700
1.18
0.09
2.17
3.56
The computation time of our proposed CoRT (Lasso) and CoRT (SCAD) increases as K or p increases. This is because we use the coordinate descent algorithm implement our methods, In either case, the increasing S or results in more parameters to be estimated, as our methods need to find the contrast (k), However, note that for high high-dimensional data with the CoRT still achieves great scalability, despite its running time.
G.3 COVID Data Details
The data that study is synthesized from various government and nonprofit institutions for all 3142 counties in the US. The data are stored from 1/22/20- 12/20/112/2/20 and publicly available We refer interested readers to Li et al. (2021) for more details about the data collection. County-level characteristics include demographic, race, socioeconomic, and medical comorbidities variables.
The data preprocessing leads to 12 target states, including Alabama (AL-67), California (CA-58), Florida (FL-67), Louisiana (LA-64), Montana (MT-56), North Dakota (ND-53), New York (NY-62), Pennsylvania (PA-67), South Dakota (SD-65), Wisconsin (WI-72), and West Virginia (WV-55). This leads source states, including Georgia (GA-159), Illinois (02.-
Colorado Florida
Kansas (KS-105), Kentucky (KY-, Missouri (MO-115). Texas (TX-253). and Virginia (VA-133). The numbers in the brackets denote the number of counties. We pick states with 50 to 75 counties as the target states, while source states have more than 100 counties. These numbers reflect the fact that source sample sizes are typically larger than target sample sizes.
We use the R packages randomForest, xgboost, and e e1071 (R Core Team, 2025) for implementations. We use max 15 and nrounds=50 for XGBoost and default parameters in the other packages.
32

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We accurately reflected the main contributions in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed our limitations in the Discussion section.
33

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't ' acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We clearly stated all assumptions and provide a complete proof for all theorems.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We included the details in the main text and the Appendix for replication.
Guidelines:
The answer NA means that the paper does not include experiments.
34

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Data is publicly available; Link to our codes is given.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/ Sodiides/oodeSumissee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. //ipps.cc/ppliiii wwtp:ccccppppppccccceeeeooppccce http://ggggiiee//deeuuuttiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
35

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We clearly specified all training and testing details. Hyperparameters are defaulted in the package nevreg.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We used standard errors in the simulations and used test in the real data.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provided the details in Appendix G.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
36

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttpp::///uuis.....// cc/public/Etttioc heabbliicEthicshiieeleeneee
Answer: [Yes]
Justification: All aspects of this paper within the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
As theoretical paper, we don't don't see the direct societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: Our paper does not have a risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
37

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cited data and codes used in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. datastts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The new assets in the paper are well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve human subjects.
38

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the core method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy thttps://euurs //tt:rries...///ee ce/ f2c://nneerencee22255///////// for what should or should be described.
39


Reviews:
[{"review_id": "AHzpj3CveX", "reviewer": "Reviewer_8wQE", "summary": "This paper proposes a two-step framework for multi-source transfer learning. The proposed method combines Lasso and Co-Regularized Transfer (CoRT) to prevent overfitting and selects the most suitable source dataset ($\\hat{S}$) among multiple sources, providing predictions optimized for the target data.", "questions": "1. An explanation is needed for what \"E\" and the \"acute\" symbol in line 94 mean.\n2. The effectiveness of the proposed method could be further emphasized by presenting results for each source without the adaptive technique.\n3. In what scenarios is the proposed method more effective (excluding the size of the target data)?\n4. The authors mention that their method would be effective when applied to neural networks. What considerations should be made when applying it to DNNs?\n5. Instead of dropping specific source domains, is there a way to maximize the knowledge that can be obtained from each source domain (e.g., sample-wise)?", "limitations": "1. Some parameters still require tuning.\n2. The observed improvement in target task performance is relatively modest. This may indicate that the framework struggles in cases where source-target alignment is weak.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "$\\textbf{Strengths}$\n\n1. The authors' efforts to provide mathematical interpretations as evidence and theoretical support for the proposed method are noteworthy.\n2. By splitting the target data to calculate errors, the method minimizes overfitting to specific sources in transfer learning.\n3. Compared to conventional ML techniques, it offers automation advantages, such as reducing the need for parameter tuning.\n\n$\\textbf{Weaknesses}$\n\n1. The explanation of the proposed method is somewhat difficult to understand. For example, in line 49 of the introduction, it should be clarified how adaptive identification of outlier sources is achieved and how this information is utilized during training.\n2. The improvement in task performance by the proposed method is somewhat limited.\n3. The analysis of experimental results is not described in depth.\n4. Minor: The text and captions in the figures are too small. Careful review by the authors is needed.", "was_revised": true, "final_justification": "Thank you for your thorough and considerate responses to the reviewer’s comments. I believe this manuscript holds sufficient scholarly merit to be recognized for its research value.", "rebuttal": "Thank you for your comments! Please see our responses in the following.\n\n**1. The explanation of the proposed method is somewhat difficult to understand. For example, in line 49 of the introduction, it should be clarified how adaptive identification of outlier sources is achieved and how this information is utilized during training.**\n\nThanks for the comment. Due to the space limit, we can only briefly mention that it is achieved via the majority vote mechanism in line 51. This information is then utilized by excluded identified outlier sources in the Co-regularization process. We will add additional details once there is more space.\n\n**2. An explanation is needed for what \"E\" and the \"acute\" symbol in line 94 mean.**\n\nE stands for expectation, and $\\psi’$ is the first-order derivative of the cumulant function used in GLMs, see [1].\n\n**3. The effectiveness of the proposed method could be further emphasized by presenting results for each source without the adaptive technique.**\n\nWe believe the reviewer is referring to Figure 2. As we mentioned on line 298: The Model starts with ``pooled'', which means that we directly pool all sources without an adaptive procedure. Therefore, Pooled-CoRT and Pooled-Trans-GLM are the results without the adaptive techniques, i.e., all sources are directly used in the proposed co-regularization scheme. Note that the goal of transfer learning is to estimate the estimation error of the target data. Current results on Figure 2 show the averaged error of all targets, which should have similar patterns to the results of each target.\n\n**4. In what scenarios is the proposed method more effective (excluding the size of the target data)?**\n\nAs Table 1 indicates, to outperform single-task learning (naive-Lasso), either a similar source (small $h$) or a larger transferable source sample size (larger $N$) can lead to smaller errors. Compared to the existing two-step TL, we allow less similar sources to achieve the same improved error bounds.\n\n**5. The authors mention that their method would be effective when applied to neural networks. What considerations should be made when applying it to DNNs?**\n\nIn the discussion, we only mentioned the limitation of DNN when applied to a very small dataset. Small-sized datasets with extremely large-sized covariates are common in certain fields like social and biological sciences, where high-dimensional statistical models (and the TL techniques in high-dimensions) usually outperform DNN-based methods.\n\nHowever, our framework can generalize to other models naturally as the key idea is to simultaneously regularize (1) the distance between source and target parameters and (2) the target parameter itself; see Eq. (1).\n\nTherefore, for more generic cases, we can modify the objective to \n$$\n\\sum\\_{k \\in \\mathcal{S}} \\left( \\text{Loss}( f^{(0)} + \\delta^{(k)}, \\mathcal{D}^{(k)} )  + \\lambda\\_{k} \\\\|\\delta^{(k)} \\\\|\\_{\\mathcal{H}\\_{1}}  \\right) + \\text{Loss}( f^{(0)} , \\mathcal{D}^{(0)} ) + \\lambda\\_{0} \\\\| f^{(0)} \\\\|\\_{\\mathcal{H}\\_{2}}\n$$\nwhere $\\mathcal{H_{1}}$, $\\mathcal{H}_{2}$ represent generic model spaces and $f$ denote the parameters/models of interests. When extended to DNN, we just need to set these $f$ as the neural network. One consideration would be that these DNN needs to be parameterized similarly. \n\n\n**6. Instead of dropping specific source domains, is there a way to maximize the knowledge that can be obtained from each source domain (e.g., sample-wise)?**\n\nOne way to maximize the knowledge from each source domain is to aggregate models from each source domain by different aggregation techniques, like [2] and [3]. Sample-wise approach could be possible, but it is more related to the topic in active learning.  \n\nThat being said, either aggregation or sample-wise knowledge transfer typically comes with extensive computational cost. For example, aggregation-based needs to calculate source models multiple times, and sample-wise typically requires one to score each sample within the source domain. On the other hand, source-wise selection achieves a better trade-off between performance and efficiency.\n\n\n**Reference**\n\n[1] Loh, P. L., & Wainwright, M. J. (2015). Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima. The Journal of Machine Learning Research.\n\n[2] Li, S., Cai, T. T., and Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology.\n\n[3] Lin, Haotian, and Matthew Reimherr. (2024) On Hypothesis Transfer Learning of Functional Linear Models. ICML."}, {"review_id": "0eIq0JPLrN", "reviewer": "Reviewer_CHM6", "summary": "This paper introduces the Co-Regularization Transfer (CoRT) framework for multi-source transfer learning in generalized linear models (GLMs). Instead of the traditional two-step approach of pre-training and fine-tuning, CoRT jointly estimates the target parameter and source deviations via co-regularized optimization. The paper contributions also include i) an adaptive knowledge transfer algorithm using the majority vote to detect outlier source domains; ii) an asymptotic element-wise confidence interval for the estimate of the target domain parameter; and iii) an extensive theoretical analysis proving robustness to heterogeneous source domains.", "questions": "- Please elaborate on runtime complexity and scalability to large K (number of sources) or p (dimensionality).\n- Regarding the derived theoretical guarantees, how do these relate to or differ from discrepancy-based domain adaptation methods in terms of assumptions and guarantees?\n- Are you willing to release the source code implementing your methods and experiments?", "limitations": "The limitations are only marginally addressed in the Discussion section. The paper would benefit from having a dedicated Limitations section.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- The proposed CoRT framework is novel and avoids the bias introduced by the usual pre-training + fine-tuning paradigm.\n- The paper presents solid theoretical foundations and the provided convergence bounds compare favorably with the existing methods.\n- The proposed adaptive algorithm for identifying outlier sources eliminates the need for hyperparameter tuning, which is a significant practical advantage over existing methods.\n- The experimental results both in synthetic and real world data confirm the effectiveness of the method\n\nWeaknesses:\n- Although the authors claim that the methodology can be extended to other types of models, the paper is focused on GLMs, so it becomes unclear how broadly CoRT can generalize beyond this family.\n- There’s no discussion on computational complexity of the derived algorithms.\n- The real world dataset (COVID-19 mortality) is insightful but relatively small.\n- The authors claim that CoRT can identify which specific covariates in the source domain are transferable, enhancing interpretability. While I agree with this observation, it would be nice to see this analysis in the experiments.", "was_revised": true, "final_justification": "The authors agreed on including an experimental analysis of the computational complexity of their method, and I also found their remaining answers to my questions sufficiently convincing. Thus, and also taking into account the other reviews and the authors' responses to them, I keep my recommendation for acceptance.", "rebuttal": "Thank you for your comments! Please see our responses in the following.\n\n**1. Although the authors claim that the methodology can be extended to other types of models, the paper is focused on GLMs, so it becomes unclear how broadly CoRT can generalize beyond this family.**\n\nWe believe the key idea of co-regularization can generalize beyond GLMs naturally, as the key idea is to simultaneously regularize (1) the distance between source and target parameters and (2) the target parameter itself; see Eq. (1).\n\nTherefore, for more generic cases, we can modify the objective to \n$$\n\\sum\\_{k \\in \\mathcal{S}} \\left( \\text{Loss}( f^{(0)} + \\delta^{(k)}, \\mathcal{D}^{(k)} )  + \\lambda\\_{k} \\\\|\\delta^{(k)} \\\\|\\_{\\mathcal{H}\\_{1}}  \\right) + \\text{Loss}( f^{(0)} , \\mathcal{D}^{(0)} ) + \\lambda\\_{0} \\\\| f^{(0)} \\\\|\\_{\\mathcal{H}\\_{2}}\n$$\nwhere $\\mathcal{H_{1}}$, $\\mathcal{H}_{2}$ represent generic hypothesis spaces and $f$ denote the parameters/models of interests. This allows us to incorporate most linear/non-linear models that learned via empirical risk minimization. Besides, the adaptive techniques (Algorithm 2) for excluding outlier sources are also model-agnostics, which thus can be generalized well to other models and used broadly by the community.\n\nWe focused on high-dimensional GLMs for two primary reasons. \n- First, since our motivation is to provide a better knowledge transfer training scheme than the two-step seminal frameworks [1] and [2] paradigm, we follow their setting which allows us to make head-to-head comparison both theoretically and empirically. Besides, the generalization of CoRT is analogous to the generalization of the two-step frameworks, where the techniques of regularizing the bias between source and target parameters in GLMs has already been naturally extended to different parametric/non-parametric models, from graphic models, to autoregressive models, and even neural networks. \n\n- Second, the high-dimensional GLMs has already been a relatively broad model family and has shown its excellent versatility in various scientify fields such as biological and social applications. Therefore, we believe the study of transfer learning in GLMs provides researchers from different fields with powerful tools, while the extension of CoRT to other models can be natural.\n\n\n\n\n**2. There’s no discussion on computational complexity of the derived algorithms. Please elaborate on runtime complexity and scalability to large K (number of sources) or p (dimensionality).**\n\nTo optimize objective Eq. (1), we leverage the package ncvreg [3], which contains a very efficient and robust optimization coordinate-descent algorithm. \n\nWe conduct the following two additional experiments to investigate the computational cost w.r.t. $K$ and $p$, with the same settings in Section 4. For our proposal, we consider two regularizers, Lasso and SCAD.\nIn scenario 1 (Table 1), we fix $p=300$ and let the number of sources range from 10 to 30. In scenario 2 (Table 2), we fix $n_{0} = 50$ and let $p$ range from $300$ to $700$. Results are in seconds and averaged from 100 replicates. \n\n\n| K | Trans-GLM      |  naive-Lasso      | CoRT (Lasso)       | CoRT (SCAD) |\n|----------------|----------------|----------------|----------------|----------------|\n| 10 | 0.41  | 0.11  | 0.24  | 0.54 |\n| 15 | 0.53  | 0.11  | 0.40  | 0.80 |\n| 20 | 0.66  | 0.12  | 0.67  | 1.37 |\n| 25 | 0.91  | 0.11  | 1.01  | 2.03 |\n| 30 | 1.24  | 0.11  | 1.51  | 2.99 |\n\n\n| p | Trans-GLM      |  naive-Lasso      | CoRT (Lasso)       | CoRT (SCAD) |\n|----------------|----------------|----------------|----------------|----------------|\n| 300 | 0.41  | 0.11  | 0.23  | 0.54 |\n| 400 | 0.63  | 0.10  | 0.49  | 0.88 |\n| 500 | 0.74  | 0.08  | 0.92  | 1.49 |\n| 600 | 0.92  | 0.08  | 1.54  | 2.28 |\n| 700 | 1.18  | 0.09  | 2.17  | 3.56 |\n\nThe computation time of our proposed CoRT (Lasso) and CoRT (SCAD) increases as K or p increases. This is because we use the coordinate descent algorithm [3] to implement our methods. In either case, the increasing |S| or p leads to more parameters to be estimated since our methods need to find the contrast $\\delta^{(k)} = \\beta^{k} - \\beta^{0}$. However, we note that for those high-dimensional data where $p \\gg n$, the CoRT still achieve great scalability given its running time.\n\nWe will include the results in the revised version.\n\n\n\n**3. The real-world dataset (COVID-19 mortality) is insightful but relatively small.**\n\nThank you for the acknowledgement. Datasets in biological or social sciences typically come with small sizes (at each site/domain). While DNN-based TL methods typically work well in large datasets, they tend to perform poorly on small datasets (small size but large dimensionality). Our experiments actually illustrate its effectiveness to unique challenges in high dimensions. \n\n\n\n**4. Regarding the derived theoretical guarantees, how do these relate to or differ from discrepancy-based domain adaptation methods in terms of assumptions and guarantees?**\n\nThis is a great question. It can be long to have a detailed discussion, so we try to make it concise but clear. At a high level, the theoretical study of domain adaptation (transfer learning) consists of two lines. \n\nThe first is, as you mentioned, ``discrepancy-based’’ results; see [4,5] and etc. They obtain model-agnostic and divergence-centric bounds which characterize worst-case performance, making bounds tend to be general and overly pessimistic, particularly when data from the target domain is available, making them less instructive.\n\nThe second one (where our work falls in) is based on more refined assumptions than the first line. For example, works in this line typically assume distribution shifts (our work is a combination of model and covariate shifts) and access to unlabeled/labeled data on the target domain. These settings bring more refined control over the discrepancy between domains, allowing researchers to design specific transfer learning algorithms. The statistical properties and guarantees are also more optimistic, as we can conduct inference on the parameters (Section 2.3) and also investigate the minimax optimality.\n\n\n**5. Are you willing to release the source code implementing your methods and experiments?**\n\n Yes, we will release the corresponding codes after acceptance. Thank you very much.\n\n**Reference**\n\n[1] Li, S., Cai, T. T., and Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology.\n\n[2] Tian, Y. and Feng, Y. (2023). Transfer learning under high-dimensional generalized linear models. Journal of the American Statistical Association.\n\n[3] Breheny, P., & Huang, J. (2011). Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. The annals of applied statistics.\n\n[4] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. (2006)  Analysis of representations for domain adaptation. NeurIPS.\n\n[5] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms."}, {"review_id": "85whyfLZv8", "reviewer": "Reviewer_1KFr", "summary": "This paper investigates high dimensional (dimension larger than the number of data in target domain) transfer learning with General Linear Models. Unlike standard two steps approach, pretraining on the source data and fine tuning on the target data, this paper proposed a co-regularization learning scheme where source data act as a regularization for learning in the target domain. Specifically, the paper proposes co-regularization transfer (CoRT), which estimates the target domain parameter $\\beta$ and parameter contrast $\\delta$ simultaneously. Moreover, the authors propose an adaptive algorithm that identifies potential outlier sources to avoid transferring \"bad\" sources. Simulations show the favorable performance of CoRT compared to two-step training schemes.", "questions": "See weakness.", "limitations": "NA", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. The paper is overall well written and easy to follow.\n2. Strong statistical analysis is provided that shows the clear advantage of the proposed CoRT framework vs existing baselines such as Trans-GML.\n\nWeakness:\n\n1. Even though the specific combinations of components are unique, the idea of one step learning for both source and target domain are not exactly brand new in the literature of domain adaptation.\n2. The practicality of the paper is also of concern, as even though simultaneously training on source and target domain would intuitively leads to better estimation error, having access to both source data at the time of learning in target domain is a rather restrictive assumption. As the establishment of two-step training paradigm is partially conditioned on the difficulty of training on both source and target data. For example, the proposed approach would make switching to new target domain extremely expensive, as the source data need to be involved all the time.\n3. There are some minor presentation issues, including really small font size in Figure 1, making them hard to read.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Please see our responses in the following.\n\n**1. The practicality of the paper is of concern, as even though simultaneously training on source and target domain would intuitively leads to better estimation error, having access to both source data at the time of learning in target domain is a rather restrictive assumption. As the establishment of two-step training paradigm is partially conditioned on the difficulty of training on both source and target data. For example, the proposed approach would make switching to new target domain extremely expensive, as the source data need to be involved all the time.**\n\nWe thank the reviewer for the insightful comment. We agree that the proposal CoRT may appear restrictive in some scenarios, but we believe the approach remains both meaningful and practical for the following reasons:\n\n(1) First, we’d like to point out that those seminal works for the two-step paradigm for high-dimensional data, such as Trans-GLM [1], also necessitated access to source datasets during their training phase. In these works, having access to source data is not only common but often necessary to ensure more robust knowledge transfer across domains as they need to control the training on source domains. Therefore, our work actually considers the same setting as those seminal transfer learning works in high-dimensional GLMs. The proposed joint training scheme (co-regularization in our model context) advances the two-step paradigm under the same setting by being more tolerant to less similar sources (better utilities knowledge from these sources.) \n\n(2) Second, having access to both source and target data is common in the field of transfer learning. Methodologically, researchers from core-ML, statistics and deep learning community have make such assumptions to achieve design algorithm that can transfer knowledge better; see [2,3,4,5,6] and more reference therein. Besides, this assumptions is also practically satisfied in some specific studies, such as biological applications. For example, [7] investigates genetic effects on obesity in a multicenter study where data from both Black/African American (target) and White (source) populations are available. In [8], methods for multi-omics data treat one omic as the target while the others serve as sources. \n\n(3) From a theoretical aspect, access the source data ensures the error bounds, such as convergence rates, to incorporate both source and target size. This allows us to provide insight into when transfer learning outperforms the non-transfer case without directly making assumptions on pre-trained models (like those domain adaptation works ranging from 2006 to 2010s). \n\n(4) Finally, although CoRT requires retraining when switching to new target tasks, distributed techniques, like divide-and-conquer, can be used to practically improved the computational efficiency. The algorithm and theory for divide-and-conquer high-dimensional statistical models have been well-developed with theory to guarantee their performance. These distributed algorithms can be directly plug-in into the CoRT framework in practice when computation expense is a concern. However, this is out of the scope of current manuscript.\n\nAt a high level, our vision of the proposed CoRT is that it is a complement to the modern transfer learning community, rather than a replacement of the successful two-step paradigm. We believe our work does provide a new perspective to better leverage the source knowledge. To provide a balanced perspective, we will also dedicate a discussion to the potential limitations and challenges of this method, together with potential distributed style solutions in practice, in the revised version.\n\n\n\n**2. Even though the specific combinations of components are unique, the idea of one step learning for both source and target domain are not exactly brand new in the literature of domain adaptation.**\n\nWe agree with the reviewer that the one step learning for both source and target is not exactly brand new in the community, especially in computer vision field. These one step approaches are also typically require access to both source and target data and retraining when switching to new target task (this actually also reasonalizes our setting better in question 1). \n\nOur contribution, however, is specifically situated within the high-dimensional $(p\\gg n)$ setting, where the research landscape is markedly different.\n\nIn this high-dimensional regime, the community has almost exclusively followed the two-step paradigm since the seminal works of [1,4,9]. This two-step approach has been the default for adapting transfer learning to various high-dimensional models. Our work, CoRT, challenges this status quo by introducing a fundamentally different, one-step paradigm tailored for this setting. The novelty of our work is therefore twofold:\n\n- Methodological contribution: The design of CoRT departs from the dominant paradigm, leverages the source knowledge better, and accommodates less similar source, which are all illustrated by the theories and experiments.\n\n- Technical Contributions: Analyzing a joint-training scheme requires a completely different theoretical approach than analyzing a two-step procedure. We develope new mathematical tools to derive the results for joint training (CoRT), which, we believe, can benefit the community in the future.\n\n\n\n  \n**3. There are some minor presentation issues, including really small font size in Figure 1, making them hard to read.**\n\nThank you for pointing this out. We admit that it is hard to trade off between informativeness and readability with multiple figures are presented. We will revise the figures to improve readability once more spaces are allowed.\n\n\n**Reference**\n\n[1] Tian, Y. and Feng, Y. (2023). Transfer learning under high-dimensional generalized linear models. Journal of the American Statistical Association, 118(544):2684–2697.\n\n[2] Dai, Wenyuan, et al. (2006) Eigentransfer: a unified framework for transfer learning. ICML.\n\n[3] Du, Simon S., et al. (2017) Hypothesis transfer learning via transformation functions. NeurIPS.\n\n[4] Li, S., Cai, T. T., & Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology.\n\n[5] Sun, B., & Saenko, K. (2016, October). Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision.\n\n[6] Ganin, Yaroslav, et al. (2016) Domain-adversarial training of neural networks. Journal of machine learning research.\n\n[7] Li, S., Cai, T., & Duan, R. (2023). Targeting underrepresented populations in precision medicine: A federated transfer learning approach. The annals of applied statistics.\n\n[8] Lac, L., Leung, C. K., & Hu, P. (2024). Computational frameworks integrating deep learning and statistical models in mining multimodal omics data. Journal of Biomedical Informatics.\n\n[9] Bastani, H. (2021). Predicting with proxies: Transfer learning in high dimension. Management Science."}, {"review_id": "IC1VT7BhXo", "reviewer": "Reviewer_MAPn", "summary": "This paper discusses the multi-source transfer learning for high-dimensional generalized linear models (GLMs). The Author proposed a co-regularization process that simultaneously learns source and target hypothesis with target domain and source domains that are similar to the target. Theoretical results shows that the proposed CoRT has a tighter error bound and a narrower confidence interval than target-only Lasso. Empirical studies validated its effectiveness.", "questions": "1. Discussion for more general model is suggested. Will the CoRT still work for more complicated non-linear model? What about the convergence?\n\n2. The idea of Co-Regularization seems to be similar to the multi-task learning. The similarities and differences between them should be discussed. \n\n3. Further, I am curious about whether the training of target model will be disturbed by the gradient of the source models, making it harder to reach the optimal solution.\n\n4. Since all models are known in the simulation experiment, numerical validation of the error bounds and the convergence behavior will more strongly support the main theorems of the paper.\n\nIf the authors can address most of the questions, I am willing to raise the score.", "limitations": "Discussion for more general model is lacked.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The method has clear modeling logic and is easy to follow.\n\n2. The theoretical analysis provide guarantee to the convergence of the optimization process.\n\nWeaknesses\n\n1. The paper only discussed method and theory for the GLMs, while even the small datasets may have more complicated hypothesis space. Thus, further discussion for more general model is necessary. \n\n2. The objective of the CoRT, i.e. Eq. (1), seeks for optimal solution of both source and target model, which is harder than only training the optimal target model. The gradients may cause negative impact to each other.\n\n3. The evaluation experiments are insufficient and further numerical analysis of the method is lacked.", "was_revised": true, "final_justification": "My main methodological concerns have been largely addressed.", "rebuttal": "Thank you for your comments! Please see our responses in the following.\n\n**1. The paper only discussed method and theory for the GLMs, while even the small datasets may have more complicated hypothesis space. Thus, further discussion for more general model is necessary. Discussion for more general model is suggested. Will the CoRT still work for more complicated non-linear model? What about the convergence?**\n\nWe appreciate the insightful comment. \n\nThe key idea of CoRT can actually generalize beyond GLMs naturally. This is because the core of CoRT is to simultaneously regularize the complexity of (1) the bias between source and target models and (2) the target model itself; see Eq. (1). Therefore, for more generic cases, we can modify the objective to \n$$\n\\sum\\_{k \\in \\mathcal{S}} \\left( \\text{Loss}( f^{(0)} + \\delta^{(k)}, \\mathcal{D}^{(k)} )  + \\lambda\\_{k} \\\\|\\delta^{(k)} \\\\|\\_{\\mathcal{H}\\_{1}}  \\right) + \\text{Loss}( f^{(0)} , \\mathcal{D}^{(0)} ) + \\lambda\\_{0} \\\\| f^{(0)} \\\\|\\_{\\mathcal{H}\\_{2}}\n$$\n\nwhere $\\mathcal{H_{1}}$, $\\mathcal{H}_{2}$ represent generic hypothesis spaces and $f$ denote the parameters/models of interests. Therefore, the non-linear models can be incorporated into the CoRT framework naturally by setting the hypothesis spaces as specified function spaces, like RKHS, class of DNN, etc. Besides, the adaptive techniques (Algorithm 2) for excluding outlier sources are also model-agnostic, which thus can be generalized well to other models and used broadly by the community.\n\nWe focused on high-dimensional GLMs for two primary reasons. \n- First, since our motivation is to provide a better knowledge transfer training scheme than the two-step seminal frameworks [1] and [2] paradigm, we follow their setting which allows us to make head-to-head comparison both theoretically and empirically. Besides, the generalization of CoRT is analogous to the generalization of the two-step frameworks, where the techniques of regularizing the bias between source and target parameters in GLMs have already been naturally extended to different parametric/non-parametric models, from graphic models, to autoregressive models, and even neural networks. \n\n- Second, the high-dimensional GLMs has already been a relatively broad model family and has shown its excellent versatility in various scientify fields such as biological and social applications. Therefore, we believe the study of transfer learning in GLMs provides researchers from different fields with powerful tools, while the extension of CoRT to other models can be natural.\n\nFor the convergence results, we believe the form of the error bounds for different models should stay in a similar fashion as long as it applies to the CoRT framework, but their exact form should be case by case since the model assumptions and analysis tools are different. This is consistent with prior literature: when the two-step method was extended from GLMs to nonparametric regression [3], autoregressive models [4], or Fréchet regression [5], the form of the convergence rate was preserved, with only the model-specific rates being updated, e.g., replacing the sparse learning parametric rate by the nonparametric rate in [3]. Therefore, we expect the form of convergence rates for using CoRT on different models will stay similar asymptotically, but just having the components being more model-specified. We believe this can be an interesting future direction. \n\n\n**2. The objective of the CoRT, i.e. Eq. (1), seeks for optimal solution of both source and target model, which is harder than only training the optimal target model. The gradients may cause negative impact to each other. Further, I am curious about whether the training of target model will be disturbed by the gradient of the source models, making it harder to reach the optimal solution.** \n\nWe thank the reviewer for this insightful question. \n\nAs the objective indicates, the training is not to seek optimal solutions for target and source models, i.e., $\\beta^{(0)}$ and $\\beta^{(k)}$. Instead, we reparameterize $\\beta^{(k)}$ as $\\beta^{(0)} +\\delta^{(k)}$, seeking for optimal solution for the target coefficient and the bias terms, i.e., $\\delta^{(k)}$. While simultaneously optimizing them can be challenging, our optimization procedure, based on coordinate descent algorithms (for nonconvex penalized regression [6] via package ncvreg), iteratively updates the target model $\\beta^{(0)}$ and each bias $\\delta^{(k)}$ term. It effectively decouples the optimization steps and ensures a more stable convergence path by isolating the high-dimensional gradient updates. This procedure guarantees convergence to a stationary point [7]. \n\nWe also mentioned this in the appendix, see line 659.\n\n\n**3. The idea of Co-Regularization seems to be similar to the multi-task learning. The similarities and differences between them should be discussed.**\n\nIt is worth noting that multi-task learning is closely related to transfer learning, though their objectives differ. Multi-task learning aims to solve multiple learning tasks simultaneously by leveraging shared structure across tasks, like the Data Shared Lasso framework [8, 9] in high dimensions. In contrast, transfer learning focuses on improving performance on a specific target task by transferring knowledge from source data.\n\nWe believe that ``CoRT is similar to multi-task learning'' is due to the fact that the objective of CoRT, i.e., Eq. (1), looks like the objective in multi-task learning. However, we would like to note that the CoRT objective is actually asymmetric w.r.t. the source and target, as the last two terms put specific emphasis on the target model (thus focus on optimizing the learning over the target domain). This is the core of CoRT, as it adopts the spirit of multi-task learning via a shared objective function across domains while having its own special focus on the target. We will add related discussion in the revised version once more space is allowed.\n\n\n**4. Since all models are known in the simulation experiment, numerical validation of the error bounds and the convergence behavior will more strongly support the main theorems of the paper.**\n\nYes, all the simulation studies, as well as the real data study, were designed to support the theoretical results. The trends in the experiments all illustrate the convergence behavior of the main theorems, and are discussed in the result description in Sections 4 and 5.\n\n**Reference**\n\n[1] Li, S., Cai, T. T., and Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology.\n\n[2] Tian, Y. and Feng, Y. (2023). Transfer learning under high-dimensional generalized linear models. Journal of the American Statistical Association.\n\n[3] Lin, H., & Reimherr, M. (2024). Smoothness adaptive hypothesis transfer learning. ICML.\n\n[4] Ma, M., & Safikhani, A. (2025). Transfer Learning for High-dimensional Reduced Rank Time Series Models. AISTATS\n\n[5] Zhang, K., Zhang, S., Zhou, D., & Zhou, Y. (2025). Wasserstein Transfer Learning. arXiv.\n\n[6] Breheny, P., & Huang, J. (2011). Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. The annals of applied statistics.\n\n[7] Loh, P. L., & Wainwright, M. J. (2015). Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima. The Journal of Machine Learning Research.\n\n[8] Samuel M Gross and Robert Tibshirani. (2016). Data shared lasso: A novel tool to discover uplift. Computational Statistics & Data Analysis.\n\n[9] Edouard Ollier and Vivian Viallon. (2017). Regression modelling on stratified data with the\nlasso. Biometrika."}]

Author rebuttal:

Meta-review: The paper proposes a joint one-step framework for multi-source transfer learning in high-dimensional GLMs. The method jointly estimates the target parameters and the source parameter deviations. Theory is provided for the case of GLMs and they appear to be solid. Experiments on both simulation and real-world COVID data were provided. One main criticism of the paper is that the problem addressed is going against the current trend in pre-training large models and fine-tuning on new domain, which limits the real-world applicability of the problem setup. However, within the considered scope, the paper's contribution is solid.
Decision: Accept (poster)