Transfer Learning for High-dimensional Quantile Regression with Statistical Guarantee
Authors: Sheng Qiao
Yong He
Wenxin Zhou
Venue: Accepted by TMLR ()

Abstract: The task of transfer learning is to improve estimation/inference of a target model by migrating data from closely related source populations. In this article, we propose  transfer learning algorithms for  high-dimensional Quantile Regression (QR) models with the technique of convolution-type smoothing. Given the transferable source populations, we derive $\ell_1/\ell_2$-estimation error bounds for the estimators of the target regression coefficients under mild conditions. Theoretical analysis shows that the upper bounds are improved over those of the classical penalized QR estimator with only the target data, as long as the target and the sources are sufficiently similar to each other. When the set of informative sources is unknown, a transferable source detection algorithm is proposed to detect informative sources from all available sources.  Thorough simulation studies justify our theoretical analysis.

Published in Transactions on Machine Learning Research (01/2024)
Transfer Learning for High-dimensional Quantile Regression with Statistical Guarantee
Sheng Qiao
sqaao edu
Deportment of Mathematics University of California San Diego San Diego, CA 92093, USA
Yong He
heyong Banougg@sdd...m edu.cr
Institute for Financial Studies Shandong University Jinan, 250100, China
Wen-Xin Zhou
wenwninedruic..eu
Department of Information and Decision Sciences University of Mlinois Chicago
Chicago, IL 60607, USA
Reviewed on OpenReview: https: // openreview. net/ forum? id= d3wur ffG4 AAG4
Abstract
The task of transfer learning is to improve estimation/infereneee of target model by mi- grating data from closely related source populations. In this article, we propose transfer learning algorithms for high-dimensional Quantile Regression (QR) models with the tech- nique of convolution-type smoothing. Given the transferable source populations, we derive f//2gesttimation error bounds for the estimators of the larget regression coefficients under mild conditions. Theoretical analysis shows that the upper bounds are improved over those of the classical penalized QR estimator with only the target data, as long as the target and the sources are sufficiently similar to each other. When the set of informative sources is un- known, a transferable source detection algorithm proposed to detect informative sources from all available sources. Thorough simulation studies justify our theoretical analysis.
1 Introduction
Transfer learning Torrey & Shavlik 2010) has been growing popular and drawing increasing attention in machine learning, which achieves great success in wide range of real applications with limited available training data. Transfer learning aims to transfer knowledge from related source tasks/domaing to enhance the learning or performance of the target task/domain, which typically involves two main subproblems. First, some criteria should be come up with to quantify the relatedness/similariie among target and source tasks. Intuitively, a high similarity would enhance the performance, while a low similarity would be harmful for the target task, which is known "negative transfer" in the literature (Torrey & Shavlik 2010). Second, transfer procedure should be carefully designed to transfer the "critical" knowledge from source domains, just like the human intelligence of leveraging prior experiences to tackle novel problems. A well designed transfer algorithm should not only identify the positive transfer sources thereby enlarging their impact, but also avoid the negative transfer in any case. All in all, transfer learning has become an active and promising research area, and substantial contributions has also been made recently to the theoretical guarantee for transfer learning in both supervised, semi-supervised, and unsupervisod scttings, see for example the context of classification by Cai & Wei (2021 Roeve et al. (2021), high-dimensional (generalized) linear regression by
Li et al. (2022: Tian & Feng 2023 Lin & Li 2022), graphical model by Li et. al. (2023): He et. al. (022

---

Published in Transactions on Machine Learning Research (01/2024)
As far as we know, there exist no work on transfer learning for quantile regression and aim to fill this gap in this paper.
Comparison with the existing work and our contribution
A few works explore transfer learning under the high-dimensions setting. Bastani 2021) studied the transfer Icarning problem under a high-dimensional gencralized linear models (GLM) with one single known trans- lerable source data and the dimensionality p is assume to be larger than the sample size of the target dataset Rtarget while smaller than that of the source dataset Asource- A two-step transfer learning algorithm was developed and the l--stimmaiinn error bound was derived when the difference between the target and source coefficient is fo-sparse. More specifically, their estimator requires Prarget Rssurce 2 Os²p²² log" (p/E)/5²) where & denotes parameter which is less than the ln-rrm of the difference vector between the coefficients of the target and source, is the number of features, and s is the sparsity of the difference in coefficients between the larget and source. Li et al. (2022) studied the high-dimensional linear regression problem under some weaker assumptions, where both target and source samples are high- dimensional. Multiple source datasets are available and the transferable set may even be unknown in their paper. With sysprse difference vector between the coefficients of the target and source for q E [0,1) and l--saare target parameter, the fz-estimation error bound was derived and proved to be minimax optimal under some conditions. In the setting where the transferable set is unknown, source detection algorithm was proposed to consistently select the informative sources. Tian & Feng (2023) further investigated multi- source transfer learning on high-dimensional generalized linear models (GLM). They assumed both target and source data to be high-dimeenn digh-dimensional and the disparity in coefficintss between the target and source to be fisparse. Given the informative sources to transfer, the & /f2-csstimation crror was derived and proved to be minimax optimal under mild conditions. [Tian & Feng (2023) also established a transferable source detection algorithm to identify the informative sources. In addition. they constructed the corresponding confidence interval for individual regression parameter. Li et al. (2021) proposed a federated transfer learn- ing approach to consolidate data from different populations and from multiple medical associations. The larget and source data are both high-dimensional in their discussion and they characterized the vector of disparities between the target and source parameters to be lo-sparse. Compared with Tian & Feng (2028 their approach achieves a faster convergence rate under some conditions and has weaker requirements on the level of heterogeneity for data from diverse populations.
Os² log (p)))/3)/ as long
we propose multi-source transfer learning method under high-dinensional quantile regression. To overcome the non- smoothness and non-convexity of the quantile loss, motivated by He et al. (2021) and Tan et al. (2022). we employ the convolution-typee smoothed quantile regression. With the help convolution smoothing, Tan ct al. (2022) proposed a gradient- algorithm that is more scalable to largo-scale problems with either large sample size or high dimensionality compared with other methods for fitting high-dimennsioall quantile regression. Assuming the difference vector between the target and each source coefficients be fi-sparse, we establish the e cistmtioninttion error bounds that are proved to be sharper than the bounds of the classical fi-penalized quantile regression (Belloni & Chernozhukov 2011 under some conditions.
Inspired by the two-step algorithm in Bastani 2021] Li ct al. 2022 and Tian & Feng 2023
In this paper, we propose transfer learning algorithms for quantile regression with high--iiennsoonll data and we assume the difference between target and source coefficients to be fo-sparse or fs-spasse. In the setting where the sources are sufficiently close to the target, our theoretical analysis and simulation results show that the estimation error bound of the target coefficients improved compared to the classical 41-penalized quantile regression model (Belloni & Chernozhukov 2011) using only the target data under mild conditions. To overcome the lack of smoothness and convexity of the check loss, we employed the convolution-tyy smoothed quantile regression and analyzed the (local) restricted strong convexity of the empirical smoothed quantile loss functions in the transferring and debiasing steps. We also extended the source detection algorithm in Tian& & Feng (2023) to the quantile regression selling. Simulation results show that the algorithm works well in discovering useful sources. In contrast to the case with fs-spare difference vector between the target and source coefficients, the algorithm with fo-sparse one learns the source coofficients indopendently, which greatly reduces the communications cost across different sources. Furthermore, the algorithm with fo-sparse difference vector has fewer assumptions on the level of heterogeneity for data from different sources.
2

---

Published in Transactions on Machine Learning Research (01/2024)
The most related work is concurrent paper by Zhang & Zhu 2022). which also considered the smoothed quantile regression models under transfer learning framework. They proposed a smoothed two-step transfer learning algorithm as well as a new source detection method based on the K-means clustering algorithm, which does not need the input of threshold in contrast to the source detection algorithm in Tian & Feng (2023). In addition, they further extended their work to the distributed quantile regression and model averaging setup. However, compared with Zhang Zhu (2022), our work doesn't require the restrictive conditions on the kernels that s1 K (u/h)/h < MM almost everywhere in u. In addition, given that the disparity vector is characterized in fo-norm instead of fi-norm, introduce an algorithm which is motivated from Li et al. (2022) and Li et al. 2021). The & f1/2g-ostimation error bounds are also established and proved to be sharper than the bounds of the classical & f--penalized quantile regression (Belloni & Chernozhukov 2011) under some mild conditions.
Before ending this section, we introduce the notations used throughout the paper. For every integer k 2 1, we use E denote the k-dimensional Euclidean space, and write [A] = fI.....kk k). For 2, Sh-
E R& :||u||2 = } denotes the unit. sphere in R$ For any symmetric, positive semidefinite matrix A R if its vector of eigenvalues is denoted by 7(A) and ordered as 71 (AA 2, ........ %pp (A) 2 0, the operator norm of A is || (A). Morcover, the vector norm induced by A is |ull ||A for any u E R For any real numbers and t, s VV denotes max(s, t) and s At denotes min(s, t). For two sequences {a, (on)x11 and (haInzi, which consist of non-negative numbers, On S b, means that there exists a constant C > 0 such that an S Cbn. an x b, equivalent to On S bn and bn S an. For r, > 0, define the b2-ball and f--one as
Ba(r) {6 E R :| S r) and C1 (5 R :|01|| [|101aa].
2 Methodology
2.1 Problem setup
Given the predictors E Rº and scalar response variable y E IR the 7-th conditional quantile functions of given æ is written as
(T) = iy : (w) 2 T)
where (-) is the conditional distribution function of given Consider the following linear quantile regression model at a given T E (0,
(T) =
where = (8)
B5(T)) e RP is the true quantile regression coefficient.
Let {1), - be a random sample from x). The precoding model assumption is equivalent the following model
//4 x 8 + c and P(G 0/æ,) =
The 4 4-penalized quantile regression estimator Belloni & Chernozhukov 2011 is generally defined as one of the solution to the optimization problem
minimize (3......,))) ST)TER
P+(:: -x]
(1)
1=1
Q(8)
where (u) is defined as (u( = - 1(u 0)}, also referred as the r-quantile check loss function.
Let (:) be the empirical cumulattve distribution function of the residuals (r:(B) Yt x:" (TT))_ i.e.,
f(u: B) = Em 1rr((8) < u} for any u E R. Then the empirical quantile loss Q(3) in [1 can be written
2(3) I pr (u)d B).
As the empirical cumulative distribution function (:::8) discontimuous, the empirical quantile loss is non- differentiable, which brings great challenges to both computation and statistical theory establishment. The
3

---

Published in Transactions on Machine Learning Research (01/2024)
kernel smoothing method (Horowitz) 1998) is commonly utilized to tackle this issue. However, the smoothed loss is still non-convex, thereby we further consider the convolution-type smoothed quantile loss function, which is not only convex but also differentiable and brings great convenience in terms of both computation and theoretical analysis. In the following, we briefly introduce the convolution-type smoothed quantile loss lunction, which was firstly introduced by Tan et al. (2022).
Let K-)) be non-negative kernel function that symmetric around 0 and integrates 1, and h >> 0 be bandwidth. That is
Kn(w) (1/h)K(u/h), K(u) ((w)=f( K(v)du and (u) K(u/h), u E.R
The empirical smoothed loss function can be defined as
1 n Σ In xF with (u) (pr Kn)(w) / (n( u)de,
(g)
where denotes the convolution operator. Therefore, the Er-penalized convolution smoothed estimator is given by
B E arg min {Qn(8)+ )1
BERP
where the smoothing bandwidth I adapts to the sample size and the dimension p while 3 depends on the quantile index T, bandwidth h, and penalty level A.
Remark 2.1. To better understand this smoothing mechanism, compute the smoothed loss la(w) plicitly for several widely used kernel functions. Recall that pr (u) uu/2 (r 1/2)u.
(i) (Gaussian kernel) The Gaussian kernel ((u) = ó(u), where x(-) is the density function of a standard normal distribution. The resulting smoothed loss is (u) (h/2)G(u/h) (T 1/2)ju, where G(u) (2/m)1/2e-w/// /2 + u(1 - (-)--))
(ii) (Uniform kornel) The uniform kernel is K(u) = (1/ S 1), which is the density function of the
uniform distribution on [-1, 1]. the resulting smoothed loss is (u( (h/2)U(u/h) + (T (1-0011///00 1/2)u, where (uu) (12/2+11 1 smoothing in the sense that (u) (L/2) + (r 1/2)u, where for every u E R, Zu denotes a random variable uniformly distributed between u - h and u + h.
< 1) |u| 1 [[u] > is Huber-type loss. Convolution plays role of random
(iii) (Laplacian kernel) The Laplacian kernel is K(u) = e |w| /2112. We have (u) = pr( + he hellw//h/2
(iv) (Logistic kernel) The logistic kernel K(u) = 10((111111 + e uy2 The resulting smoothed loss (u(u) = Tu + h log(11 + w//h).
(v) (Triangular kernel) The triangular kernel is K(u) (1- (u]-lu))(([u/ < 1). The resulting smoothed loss is
4. (u) = (h/2)l.r (u/h) + (r 1/2)u, where lrr(u) = (u² - |u|3 /3 + 1 < 1) + 1[u] 1).
(vi) (Eaancchnikov kernel) The Epancchnikov kernel is K(u) = (3/4)(1 - u. u 1 (/ul - 1). The resulting smoothed loss is 4, (u) = (h/))E(u/h) + (r 1/22,,, where E(u) = (3u² /4 u² + 3/8)1 < 1) + |u] ( ([u) 1).
One can easily check that all the empirical smoothed loss functions above are convex. See Figure for a visualization of Horowitz's and convolution smoothing methods.
In the following, consider the multi-source transfer learning scenario, where we have larget data set (x("),, ()) and K. source data sets with the k-th source denoted as (k),,, , (*)), where x(k e R= (M) R for = O....,K. K. The i-th row of (k) and the i-th element of (M) are denoted as x; (k) and yi (k) respectively. The goal is to transfer useful information from the source datasets to improve the estimation
4

---

Published in Transactions on Machine Learning Research (01/2024)
Gaussian
07
Unform
Laplacian
0.6
Logistic
Tiangular
Epanechnikov
Quantile
0.5
Loss Function
0.4
0.3
02
011
0.0
-1.00
-0.75
-0.50
-0.25
0.00
0.25
0.50
0.75
1.00
Value
Figure 1: Plots of standard quantile loss, Horowitz's smoothed quantile loss (Horowitz) 1998
and some
proposed convolution-tyy smoothed quantile loss with different widely used kernel functions.
accuracy of the target parameters. Denote the true target parameter as 8" = w (0), We assume the responses in the target and source data all follow the linear quantile regression model, that is,
(N) y = (o) )T (k) te, and P(e < 0f0; (x) = k == == ==.....,K
We build our quantile regression transfer learning procedure in the high-dimensional regime with a sparsity assumption. In other words, assume the dimension p is much larger than the sample size n, for all k while the target model is s-sparse, which satisfies ||
Define the k-th contrast 8(k) f (1k) and [| lg is referred to as the transferring level of source k in the litcrature, where (0, 1}. Define the level-m transferring set Am = (k 1| S m) as the set of sources which has transferring level lower than m. Denote n EtEA... 7k. 0g = / (n..., + no) for {0} U. and KAA.. = Amm
As stated in the introduction, we will consider two types of transforring level, corresponding q E (0, respectively. In the of q= 0, the transferring set corresponds the source data whose contrast. vectors have at most m nonzero elements. In the case of g 1, all the coefficients of the contrast vectors can be nonzero, but their absolute magnitude decays at relatively rapid rate. It will be seen later that as long as m is relatively small, the source data in An can be useful in improving the estimation accuracy of S In addition, the logic of the algorithm with f--normed A.. and the algorithm with fg-normed Am are quite different and we will elaborate on these two different algorithms in the following sections.
2.2 The proposed algorithm with an -nomm constrained transferring set
In this sention, we propose the transfer learning algorithm with 4-norm constrained transferring set, which is motivated by Tian & Feng 2023). This algorithm involves two steps. The first step of our algorithm is transfer the information from useful sources by pooling all the data transferable Ao to obtain primal estimator. We also call it the transferring step. To be more precise, define total smoothed loss function for the target and source datasets in the transferable set Am, i.c.,
Am and target set
Qn(w) n1AAm. + vi0 KEA...
IN (B)
)

---

Published in Transactions on Machine Learning Research (01/2024)
where
(u(u) (pr (u)(u)= / (u u)dv.
Then for the transferring step, we aim to find the minimizer to the following optimization problem with respect w R
minimize (Qn(w) + Awl1w
We denote the minimizer as &mm,, i.e, a-m = arg minw (Q((w) + {ullw|l> By selecting an appropriate bandwidth h, the iteratively rewcighted E1-penalized SQR estimator proposed by TTan et al. (2022) shares the same upper bounds for both & and & errors the fi-QR estimator, as indicated by Belloni & Cher- nozhukov (2011). Furthermore, they introduced coordinate descent and LDMM-based algorithms for solving l-ppnnaiieed quantile regression, which are computationally efficient especially for large-scale problems.
Denote the true parameter in the first step as wAm, and wmm has the following explicit form:
Am = 3 + 8dm.
where 8.hm = EKEA,, 8 and on = =m/(nAA + no). For the second step (the debiasing step), we correct the bias, 8A.m,, based on the estimator wrm acquired in the transferring step. The smoothed loss function for the target data with respect to ò is defined as
1 no
9 (w 6)
Σ
l (vi
(0)
(2::0)) + -)).
The error of the debiasing step is under control for a relatively small m, since 8Am. is ly-sparse high- dimensional vector.
We call this algorithm Oracle f--Trans-SOL SQR as we first assume that all useful sources are known as a priori.
Algorithm ! formally presents the Oracle l Ttrans-SQU algorithm.
Algorithm 1: Oracle f S--rran-SQRR
Input: Target data (X(1), y))), source data ((X(A), y ( k==1 penalty parameters d and As, transferring set Am- The estimator B.
Output:
1 Transferring step:
wm + arg min (a((w) + Aullw|l1
Debiasing step:
8mm +- arg min (0.0) (a (a0m + +5) Ao|10111
min
return 3 = Am + 8Am.
If Am is unknown, then we need detection algorithm to find useful transferable sets in practice. We propose transferrable source detection algorithm which is inspired from the Algorithm 2 in Tian & Feng (2028) Firstly, partition the target data into subsets. Secondly, fit the penalized smoothed quantile regression on each combination of (q 1) target subsets and calculate the loss on the remaining target subset. In the following, consider the average cross-validation loss i Run the transferring step on each combination of
(g 1) target subsets and each source data, and evaluate the loss function the remaining target subset. Similarly compute the average cross-validation loss ((( for each source. Thirdly, calculate the difference
between (100) and f.(k) for each k and compare it with predefined threshold. Finally select the sources

---

Published in Transactions on Machine Learning Research (01/2024)
Algorithm 2: Transferable Source Detection
Input: Target data (x ()),, all source data {(x (k),
() 1== threshold Co, penalty
parameters (fx00la) DA
Ik=0lam Saoolam where is the number of folds chosen.
Output: The set of transferable sources Á.
Randomly divide (x("), ()) into equal-sized sets ((X(0) )
for = to do [[0]a]] + fit the penalized quantile regression {(X (0] penalty parameter (())aa
(0))))) (X(1))) (D]aa)) with
((k)aa run the transferring step in Algorithm with
4
() and penalty parameter (A0(a) for all & + 0. Calculate the loss function [. (g(1)ol) (x(0)) y (ole)) for k 1,..., K.
((x(0))),(( (000) (X("))al y (0a))) U (x (M) y
(R)
f >a= 1/ (
- f.0 (((00)1)/4
(B(k)]al) - L6 ///g-///-000////////00//0) 1).
A (k + 0: 10) 1000 - Colô v 0.01)).
8 return Á.
whose difference is less than the threshold and include them in the set A. The detailed transferable source detection procedure is summarized in Algorithm
With the transferrable source detection algorithm, we propose a feasible Algorithm 3 in practice, in which we first detect useful source datasets A by Algorithm and then run Algorithm using datasets ((x(1), (k), ( REE(OUUU
Algorithm 3: Trans-SQR
Input: Target data (x (0) (0)), ), all source data ((x()), ( k= a threshold Co and penalty
parameters ([)())()) -
Output: The estimator g.
Run Algorithm 2 (Transferable Source Detection Algorithm) and output Á.
Run Algorithm (Oracle Trans-SQR) using data ((x(1), (b)) REfOUA
return 3.
2.3
The proposed algorithm with an norm constrained transferring set
In this section we consider a more strict transferable set A', {k 118(^)1lo S m), where the fi-nrrm discussed in Section 2.2 is replaced by norm., Compared with the formm, the theoretical analysis of the transfer lcarning procedure under f--norm is free of the restrictive Assumption 3.4 below, which requires "sufficient" similarity between the target covariance matrix and transferable However, as formor is not additive, it is not easy to combine target and source data to estimate a primary estimator for the true target parameter. Instead, we correct each souree data independently and incorporate the corrected source and target data to make predictions. Certain adjustments need to be made on the proposed transfer learning procedure in Algorithm [
covariance matrices.
This lo-morm constrained transfer algorithm is inspired by the idea Li et al. (021)). Unlike the transferring step in Algorithm D the first step of the algorithm in this section train each source separately to get primal estimators of (R), I E {1...., K),, where the smoothed loss function for each source is
me
(1) (W) = n.. Σ
In (vi
(2x()))T

---

Published in Transactions on Machine Learning Research (01/2024)
In the second step, the debiasing step in Algorithm| 1 adjust for the differences ((h) for all & using the target data, which is obtained via.
((h) arg min of + Aollōl11
8
where the smoothed loss function with respect to ò is defined as
(0( + -0) 6) n1
1 oo (v, (0) - (x)) (()())))))))))))))))))))))))))) + +)).
Then threshold for each (()) is computed by only keeping the largest v no/ log elements of 8(k) and letting all the other elements be zero. In the third step, with the estimated "bias" from the second step, the corrected source data has the following form:
{( (x((), + x()
R
k=1
Then, we combine all the corrected sources and target data to estimate the parameter 3 which is of our interest. The above algorithm estimate the source parameters and the contrast vectors individually, while the Oracle E--TransSQRR proposed in Section 2.2 pooled analysis is conducted with data from larget and sources, which relies on the homogeneous designs of the covariance matrices among target and source data.
Algorithm 4: Oracle Trans-SQR with f--norm constrained transferring set
Input: Target data (x (0), y source data ((x(1) (k), y (1))) penalty parameters Awr As and Äg, transferring set A'n- Let n no n.A.
Output: The estimator 9. For each & E A'm
(1) + arg min @n(w)) + A / |el
w
For each & A'ms
8(k) + arg min (110) (2)t0) 8) + As|1o|11
Threshold (() via 5(4) H
no/ log ((t)), where (b) is formed by setting all but the largest k
elements of b to zero.
4 Joint estimation using source and target data:
B - arg min
(y.
(0)
(x. (3)
g
i=1
13k
(a (=(())((B - 8(0)) + As118111 }
Lm (y)
(k)
kE
i=1
return g.
3 Statistical theory
In this section, establish theoretical guarantees the algorithms in the above section.
8

---

Published in Transactions on Machine Learning Research (01/2024)
Assumption 3.1. There exists f 2 f > 0 such that the conditional density of c(h) given x(8) satisfies I S [e00) (0) S I almost surely (over (x() for all k = o,...... Moreover, there exists to >> 0 such that (aoojatt - Saa (v)l I f0/u- 1 u| for all u, R almost surely (over x(()), and 2 12120) for k ==00 ...... IC, where ( denote the covariance matrix of x*(), and
inf IE ( (t(2()))Te) (( (1) ()) 21
Assumption 3.2. The kernel function R - [0. 00) is symmetric,thal is, K(u) = K(-u), and satisfies that 1_ (u)dd and __ u²K(u)) < oc. For a 1,2,.... let Kig = J- (al"((u)du be the a-th absolute moment of (-)). Assume SuPEER ((u) S R for some R E (0.
Assumption 3.3. For A = O,..., K, [() E|x(() (()()))))()))))))))))))))))))))) (x)()) is positive definite and z(k) = (2(())-1/22k) E R" is sub-exponential: there exist constants uo, cu 2 1 such that P([(=(#))* 2 sol|u|le t) S cue for all u E RP and 2 0. For convenience, we assume co = 1, and write of maxigisp B(zz).
Under Assumption 3.3 the a-th (a 2 3) absolute moments of all the one-dimensi marginals of are uniformly bounded: Pa == suPueso-1 (x)) I alvß- In particular, μr S 12
Meanwhile, for every 8 E (0, 1), define
M]6 inf n > 0: E ((z()))T 21 (1(2(t))* m)]
5 for all S
Since (x u for any v g--1, 7s is well-defined for each 8, and depends implicitly on the distribution of (k))
Assumption 3.4. Denote
Σ = Σ / (1)
K
- + tww))d
k=
(00) / V2Q(0 (1)- + tw )dt,
where (2) - t) tw) EfSejo (lw 118)) x (xx) (z.())
Define
C1 = sup
oSkgK
Let C1 be bounded, that is C1 < oc.
imposes the Lipschitz continuity on the conditional density (.)= Assumption 3.2 holds for most commonly used kernel functions, for instance, uniform kernel, Gaussian kernel, etc.
Assumption 3.1
Assumption 3.3 assumes sub-exponential condition the random covariates characterized by well- behaved covariance structure. In particular, μ4 can be regarded as the uniform kurtosis parameter.
Assumption 3.4 restricts the difference between the target covariance matrix and transferable source co- variance matrix in some sense, which guarantecs the estimator at the transforring stop is close to the truc parameter Bt. This assumption is commonly used in other transfer learning works, Tian & Feng (2023):
et al. ) Zhang & Zhu (2022); Huang et al. (022).
Formally, consider the parameter space
(s, = f (w())) JEAn 113" < 8, sup ( -8 S <m}
k
3.1 Estimation with an normr constrained transferring set
Proposition 3.1. (Local Restricted Strong Convexity) Assune Assumptions 3.1 3.3 hold. Let = w- n 12AA. + no and k min/ag11 K(u) > O. If (r. n, n, satisfies
max(4m1/r7; 320071
1/22////
1/2 1/2
f/lo and nh 2 2 / logr,
9

---

Published in Transactions on Machine Learning Research (01/2024)
for any w" + Br(r and w w(()) B1 Bi(d),
log p log (w w²) 2 Φ1 / 02
Q
|| |||.,
(4)
nh
with probability at least 1 - (pn) where Φ1 = f/10 and d2 > 0 is a constant depending only on (x, f).
Proposition 3.2. Assume Assumptions 3.1 3.3 hold. Let v = B1 B2, 01 ---K-- f/25 and 0 = C (20g).
If max 8worp4 1/2 s g - f/1o with 01/4 defined in 3 and nog 2 ff 2n / then for any a E By
(81) (32)
( (12) -VQ. (32))" -32) 2 2allll C
logp + log no
||
nog
with probability at least 1 (pma) 1 where Cz, > 0 is a constant depending only on (a. D) By the arithmetic mean-geonee geaneeeemettic mean inequality
logp + log no
|1l11|/1|1 < Φ
C
logp + log ng
C..fN
||0|
nog
2
nog
have
of
log +
log 1o
(B1) ())(()))) (V2) (B1 B2)
|001
||01
2
nog
with probability at least (pma)
In the debiasing step, we need another restricted strong convexity condition with both || |l and
|| 11
the lower bound. Proposition 3.2 provides that kind condition.
Finally, with the above cstablishments of restricted strong convexity, we able to obtain the main result for the two-step transfer learning algorithm quantile regression.
Theorem 3.1. Assume Assumptions 3.1 3.4 hold. Suppose S s/log(p)/no. no 2 Csa log and TAA,, Z no where C > 0 is constant. Also let
log(p)/(nA... + no) S h < min (f (s (w1
slog(p)/1g S 5 (log(v)/no) 1/4
We take Au Co (nam noo) As Co log(()))) where Co and C's are sufficiently large con- stants, then
log
1/4
log
1/4
log
1/4
113 - 19111 S vm
+
Vs
(5)
n10
no
n1 + 710
log
log p
113 88 S
$1
+
v /sm + m,
(6)
n1dm. +
12.m
+
n2p
with probability at least 1-p-" 1-p-1.
Remark 3.1. In the trivial case where An is an empty set, the upper bound in 5 is Op( ( sog(() When An non-empty, the upper bound in (5) is sharper than vs /slog(p)/ne and the upper bound in (6) is sharper than s //log(p)/no if n.A.. Z no and m < s(log())/no)1/2
The above thcorem gives the convergence rate of the Trans-SQR estimator under 2 As the above remarks stated, if the total sample size of the Iransferable sources significantly larger than the larget sample size, the Trans-SQU estimator could even achieve a sharper convergence rate with some proper choices of
the transferable level of the contrasts and the smoothing bandwidth in the debiasing step. As some previous works show, our theorem shares similar estimation error bounds as the results in Tian & Feng (0233 and
Li et al. 2022)
10

---

Published in Transactions on Machine Learning Research (01/2024)
3.2 Estimation with an fy-norm constrained transferring set
Assumption 3.5. For O,..., K, the covariate vector (x() compactly supported with
(k)
((1(0))-1/22)) 00,
sup Ely
and ||o I B almost surely for some B 2 1, where E(A) is positive definite. Without loss of generality, assume B 1. In addition, M4a supuss ( ul" oo for 1....,
Remark 3.2. Assumption 3.5 is a stronger version of Assumption 3.3 Note that quantile regression has B-
where (x) is the smoothed empirical quantile loss and K2 (u) (u/h)((u/h)) Unlike the generalized lincar regression, there is smoothing bandwidth in the denominator. We import Assumption 3.5 to provide convenience for bounding the difference (B)
= Hessian matrix [[m) EI/
(B*).
Proposition 3.3. (RSC in Step 2) Assume Assumptions
3.1
3.2
hold. Let v = B1 82. If
max(4mm/4,3200r/l 32voru4 /23 S g S f/lo and nog 2 fffff-/jj / then for any v E By(r),
(B)) 2 (B2) (V))
(52) ( (Br -82) 2 0.1x fllells.
with probability at least (pma)-1
Theorem 3.2. Assume Assumptions
3.1
3.2
3.5
hold. Let
log(p)/no S h S (2lo((( (8!/2 1/2)
(s m) log(p)/ng g ((s + m) log(p) 1/4
m mlog(p)/n w (m (mlog(p)/m) 1/4
where n no + nAA Meanwhile, suppose S 8, ny no and no Cs² logp, where C > 0 is constant.
We take X Cw log(p)))n As Cs /lg(())//o and As (ta/lo((p)/m where Cwn, Cs and C3 are sufficiently large constants, then
slog
smn log p rhp
|| 1 S
(7)
n
log
m log
||33 - 1/ S $1
+ 8
(8)
with probability at least p-1
The above theorem gives the convergence rate of the Trans-SQR estimator under & f1/62-errors where the contrast vectors are characterized in terms of the fo-no... If the sample size of the target data is large enough and the total sample size of the transferable sources is significantly larger than the target sample size, the Trans-SQR estimator could achieve a sharp convergence rate with some proper choices of the transferable level the contrasts.
Remark 3.3. As mentioned above, Assumption 3.4 is make sure that the estimation error in the trans- ferring step is small enough. However, Theorem 3.2 does not require Assumption 3.4 because Algorithm 4 learns the parameter (A) independently in Step and reduces the bias in Step 2. For Step 1, the upper bound of the difference wetween the estimator (k) and true parameter (k(k) can be controlled by the sample size of the each source data and the to transferable level m. For Step 2, the estimated could also be closed enough to the true difference between the target and source parameter by having an appropriate target sample size. Therefore, if both the target and source sample sizes are large enough, the error of Algorithm would be well controlled without Assumption 3.4
11

---

Published in Transactions on Machine Learning Research (01/2024)
4 Numerical studies
In this section, evaluate the performance of our proposed algorithms via numerical experiments. The methods in the following section include Smoothed Quantile Regression (SQR) target data, the Oracle- Trans-SQR, An---an------- Trans-SQR and the Waive-Trans-SQOt, which naively assumes A... = L,.... I.. in the Oracle Trans-SQR. The purpose of including the Naive-Trans-Iee is to understand the overall informative level of the auxiliary samples.
4.1 Transfer learning an nnnrree Am
We consider p= 500, no = 200, and .,,...., 710 = 150. The covariates from target x; are i.i.d. Gaussian with mean zero and covariance matrix E with En 0.5li -'1 for all = 1.....77 no and E are i..d. Gaussian with mean zero and variance one for all i. For k E Am. x ~ ~OO, E + eE"), where ~~ (N Ip)
(0)
For the target, the true parameter 3", we set 8 = 5, B3 = 0.5 for j E (1....,8) and By = 0 otherwise. Denote R (k)
p independent Rademacher variables. R(() independent with R(k)) for any k + K. For any source data k in Am. we let the true paramcter (R) =3* (m/p)R (k), where m E {5, 103. For any source data K' not in Am, the true parameter w(k') = 8 + (2m/p) We train the four methods with 100 reproductions and record their average fz-estimation errors under different settings r. Figure shows the changes of the estimation errors along with the amount of the transferable sources.
We observe from Figure 2 that the Oracle- Trans-SRR has the best performance among all the methods and An-TTans--Se has almost the same performance as the ferable source detection algorithm still works under the smoothed quantile regression models. Meanwhile, compared with SQR on target, the estimation errors of the Oracle-Trame Trans-SQR and Am--Taans-SO are always smaller, which means that the source data which share some similarities in f--norm with the target data could improve the estimation. Another observation that the performance of Am--Tras-SQQ consistently improves as more and more souree data are transfcrable. This matches the thcorctical ez-sstmmation error bounds which become sharper as n.m.. grows.
which indicates that the trans-
4.2 Transfer learning on an fo-normed Am
We consider p == 500, 200, and assume that there are 2, 6, 8, 10 transferable sources with the sample sizes 400. The covariates from target x:!0 are i.i.d. Gaussian with mean zero and covariance matrix Σ with Ejy' = 0.5 The covariates from source x! are also i.i.d. Gaussian with mcan zero, but with covariance matrix Σ + EET, where e ~ (.,,,,
For the target, the true parameter 3 we set 8 = 5, B = for j E s}, and 8, = 0 otherwise. For the source, their true parameter (*) is generated from w (k) 83 A1(j E M), where M is a random subset of [p] with |M m. We take m (2,4), and A 2. Figure and 5 show the fzessimaaion errors in different settings of m.
From the results, Trans-SQR with fo-norm constrained transferring set has better performances than SQR only on larget and SQR on all sources and target. Meanwhile, when the target data sample size no becomes larger, the performance of Trans-SQR increases quickly, which accords with our results that the estimation error is depend on the target sample size. There are considcrable decreases in estimation errors of Trans- SQR when the transferable level increases or A increases, which corresponds to the difference on components between target and source populations.
5
Conclusion
This paper studies transfer learning for high-diieensional quantile regression models, employing convolution- type smoothing techniques. The proposed algorithms focus on leveraging 21/oo-oormed transferable source populations to improve estimation accuracy of the target regression coefficients. We derive error bounds for the estimators in terms of f//2nnorms for the algorithms. Theoretical analysis reveals that these error bounds surpass those of the classical penalized quantile regression estimator, which only utilizes the target
12

---

Published in Transactions on Machine Learning Research (01/2024)
10
10
0.-
0.4
0.5
0.4
0.3
02
0.1
0.0
08
02-
Oracle-Tara-500
Rabee-Tans-OOR
Figure 2: lg estimation errors of several methods under quantile levels r = 0.5,, over 100 repetitions, where Gracle-Trans-Soe Algorithm
data, provided that the target and source populations exhibit sufficient similarity. Furthermore, we propose a transfcrable source detection algorithm to identify informative sources from the available sources whon the set of informative sources is unknown. Numerical experiments validate our theoretical results.
Acknowledgments
Wen-Xin Zhou is supported by the NSF grant NSF DMS-2401268. Yong He is supported by NSF China (12171282) and Qilu Young Scholars Program of Shandong University.
13

---

Published in Transactions on Machine Learning Research (01/2024)
m 10
0.8
03
02
0.6
0.
GouDS
10
+ Oracle Tonn SQR
Apr Tinre SOR
OR target
Moive Tranc SDR
Taase_
Figure 3: l estimation errors of several methods for Gaussian and t2 errors, over 100 repetitions.
14

---

Published in Transactions on Machine Learning Research (01/2024)
m
N0 0.2
SQB al
SQnt on target
Figure 4: l estimation errors of several methods with to constraints for to errors, over 100 repetitions.
15

---

Published in Transactions on Machine Learning Research (01/2024)
20
1.0
0.5
SOR
SCR
Figure 5: & estimation errors of several methods with by constraints for Gaussian errors, over 100 repetitions.
16

---

Published in Transactions on Machine Learning Research (01/2024)
References
Hamsa Bastani. Predicting with proxies: Transfer learning in high dimension. Management Science, (5(; 2964-2984, 2021.
Alexandre Belloni and Victor Chernozhukov. 4 fi-penalized quantile regression in high-dimensional sparse models. The Annals of Statistics, 39(1):82 130, 2011.
Olivier Bousquet. Concentration inequalities for sub-additive functions using the entropy method. In Stochas- tic inequolities and applications, pp. 213 247. Springer, 2003.
T Tony Cai and Hongji Wei. Transfer learning for nonparametric classification: Minimax rate and adaptive classifier. The Annals of Statistics, 49(1):100-128, 2021.
Xuming He, Xiaoou Pan, Kean Ming Tan, and Wen-Xin Zhou. Smoothed quantile regression with largo-scale inference. Journal of Economelrics, 2021.
Yong He, Qiushi Li, Qinqin Hu, and Lci Liu. Transfer learning in high-dimensional semiparametric graphical models with application to brain connectivity analysis. Statistics in medicine, 41(21):4112-44 4129, 2022.
Joel L Horowitz. Bootstrap methods for median regression models. Econometrica, pp. 1327 1351, 1998.
Jiayu Huang, Mingqiu Wang, and Yuanshan Wu. Transfer learning with high-dimensional quantile regression.
arXiv preprint arxi:: 2211.14578, 2022.
Michel Ledoux and Michel Talagrand. Probobility in Banach Spaces: isoperimetry and processes, volume 23. Springer Science Business Media, 1991.
Sai Li, Tianxi Cai, and Rui Duan. Targeting underrepresented populations in precision medicine a federated transfer learning approach. Xix preprint arXiu:2108.. 12112, 2021.
Sai Li, T Tony Cai, and Hongzhe Li. Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statisticol Society. Series B, Statistical Method- ologv, (411):149-173, 2022.
Sai Li, T Tony Cai, and Hongzhe Li. Transfer learning in large-scale gaussian graphical models with false discovery rate control. Journal of the American Statisticnl Association, (1(4443)277121188))) 2023.
Lu Lin and Weiyu Li. A correlation-ratio transfer learning and variational stein's 's paradox. arXiv preprinl arXiv:2206.0600 2022.
Po-Ling Loh and Martin Wainwright. Regularized m-estimators with nonconvexity: Statistical and algo- rithmic theory for local optima. Journal of Machine Learning Reseorch, 16:559-614 616, 2015.
Henry WJ Reeve, Timothy I Cannings, and Richard J Samworth. Adaptive transfer learning. The Annals of Statistics, 49(6:33618 3649, 2021.
Kean Ming Tan, Lan Wang, and Wen-Xin Zhou. High-dimensional quantile regression: Convolution smooth- ing and concave regularization. Journal of the Royal Statistical Society Series B, 84(1):205 233, 2022.
Ye Tian and Yang Feng. Transfer learning under high-dimensional generalized linear models. Journal of the American Statistical Association, 118(544))2684-2697, 2023.
Lisa Torrey and Jude Shavlik. In handbook of research on machine learning applications and trends: algo- rithms, methods, and techniques: algorithms, methods, and techniques. IGI global, pp. 242-2642244 264, 2010.
Roman Vershynin. Fimg--dimensional probability: An introduction with applications in dala science, vol- ume 47. Cambridge university press, 2018.
Xiao-Tong Yuan, Ping Li, and Tong Zhang. Gradient hard thresholding pursuit. Journal of Machine Learning Research, 18(166):1-45 2018.
Yijiao Zhang and Zhongyi Zhu. Transfer learning for high-dimensional quantile regression via convolution smnothing. Xiv preprint arXiv:2212.00. 2022.
17

---

Published in Transactions on Machine Learning Research (01/2024)
A Appendix: Proofs of the main results
A.1 Technical Lemmas
For E 12, suppose A w* Define
(A) Qn(w) Qu(w²) (w -w)- (A) Qu(w) - Qui(w"),
and their population counterparts (A) (A)R((A) and Da(A) (A)r(A)), where wt is the true parameter of the transferring step in the algorithm.
Lemma A.1. Let 8 be the true target parameter, then ||-- - 11 and Σ E
Cpm, where C₁ sup || |
((k) are given in Assumption 3.4
Note that w" has the explicit form, w* = 3 0 Lemma A.1 gives an upper bound of the distance between the true 8* and the true estimate in transferring step. In other words, the f--mrrm of 8* is controlled by m.
Lemma A.2. Define Tे TB() RP, where (B) VQ)(()) (B). Assumptions 3.1 3.3 ensure that for any t> 0,
A S sa (T( +
2t
max T, nAA + no
t
n1A.. + no
with probability at least 1- 2pe where C (T 1) and maxigiso Ojj-
In both transferring and debiased steps, we need to restrict the regularization parameters X (or Ag) to be no smaller than 2 /l oo (or 2|]=)))>>)- This Lemma helps to specify the choice of the parameters.
Lemma A.3. Define bj -|==1//VQ which quantifies the smoothing bias, then for some 0
h
6% S lokz 2
where to the Lipschitz constant the density felm(.) Lemma A.4. For r,l > 0, define
1
((r,1)=
sup
Kn - v.) Rn(x]3 vi)
For any > O, with probability at least 1 - et
logp + f1(2
+ log nh.
t + log
((r,1) S TV
+
n
A.2 Proof of Proposition 3.1
Define the Taylor error
T(w, = - Qn(w) - Qu(w")) - (VQ)(w")) "ww w²).
In the following proofs, consider the subset of w, w" By(rr Cx:(1), and . w + Bi Bd).).
18

---

Published in Transactions on Machine Learning Research (01/2024)
It follows from second-order Taylor expansion that
w(w,w")
(w w') V2''
Qn (lw (1 f)w")) (w w")
Σ
Σ
Kntti (k) - (x (k) (T (K) w 1,th0 )w)) (1x) ((ac:())rr ) ww))²
A(n + nn)
kEAmU10) i=1
E
Σ
KnfG - (1h) (z. - w²)
w(+))}
2(nAm + no)
kEAmutO) i=1
-(wz ((m) w")),
for some 4. E [0, 1]. For each i and k, define the event Fi,k.
Fix = [ s h/4) n {1(2) (( w³)l s س| - h/(r)) n -w" s h1/4),
-
for all w' By(r). Thus
T(w,w") 2 2(m + no)h
K
Σ Σ
(at (((()))T( w²))² F
(9)
kE s=1
where x = minj ((u). For a truncation level R > 0, define functions
u22 lu| S R
(n(u)= = (R - (u|)2
R |u| S R.
2
0
|u| > R.
By this construction, (p(()) S u 1(lu] S R), Pen(cuu) ( and yn is R-Lipschitz.
In addition, define the trapezoidal function
1
lu| S b
vn((u) = 2 - |u|
N < |u| < R
0
|u| > R.
and note that UR is (2/R)-Lipschitz and vu((u) < 1flul < R}.
With the two new-defined function and the notation A w w", n2 T + no, we have established the lower bound of
7(w,w")
n
K
Σ
Hllel $ hh/44p (all):)/(2r) ((=())=AA)'n, (lz w())))
2nh
kEAm 1=1
|
Σ
1l6l hh/4)/h///m (1Aa)//aaaa
w()))
(10)
nh
R.
Dolwow')
In the following proofs, we bound EDo(w, w²) and Dolw, - (DDow,.")) respectively, First, we show that
BDo(w, w") 0.21
(11)
19

---

Published in Transactions on Machine Learning Research (01/2024)
Note that
P/ 1/2 f/ (0)
[[[[|eal s [ < [[ S 1/44]/z0
[
h
h/4
[el (1) - ce)
h/4
lgh?
16
Hence we obtain
(2)
h
1hh
2
2
16
Provided h S f/4o= S f/lo, we have
[[a S 1/4
7fh
16
Meanwhile
E(1((e)) s b/4 [01]
1he [2
//1// h/4
h//4
(ee - Sej (0)lat
implies
9fh
B[[ [/4)aa/
S
16
Then
(DDlw,ww")
E [<cl (+1 (9n/(22) (10p/((11111///mm) ((//))))A/|AA))))))),) (w
F
nh
7f
(P)(22)) (x (d"A///Alll)/)aa("""" -w(+))
2
16
7L
F
( [[=T\/||||>> 1 A/ S A/(44)) (wo"
16
7L
18 (<>A>//1
16
T
A <>"A\/l1
2 f1 ( -0))>>>110) +//8|}1/}}}
7L
1-B
( bx) 11 > n/(4r))] > - - /1=" - 118)
16
By the definition of n7. as long as o<rs h/(4m//))
(<<A\\/AA)))) 2 f1 A/IIIII) > | _
sup AEBe(
Moreover, w" E (h)) + (d). Hence
|| - (()))) - || - wل(lس|| || S 1/2 d.
Under Assumption 3.3 with to 2 1, the tail bounds of sub-exponential z = Σ 1/2 implys that
- ((k)) > h/8) < 2 2cxp
{
h
80071 1/2 d
20

---

Published in Transactions on Machine Learning Research (01/2024)
1/2 1/2 Let 2 32v071 dui
It then follows from numerical calculation that
1 - E (/>A\\|)))>))> (1=1=1/1111111 (1
- (12""(""""'"""" [ h/(4r)) -w())) h18) 2 20.49
holds uniformly over A Bx(r) Cz().. Putting together the pieces yields
BDolw, w*) 0.21f.
Next we find lower bound of Dolw, w*) - BEDo(w, w*) over w w + Br:(r) n Cx((). Define
1)r,1))= =
sup
{ (-Dolw,w") + EDo(lw,
WEW
Write Dolw,w") nn1 (kEAmU E Wi (w, where
(ww(www") h h 1 S h/4)ph ((2; ( A/lll)ls)))))) /h/4 ((x) ( (()))
satisfies < wi, w*) < h/(4r)² since ( < (R/2)² and vn(u) [0. 1]. Moreover,
Ew (w) ) E 21les h/44 ((x) )=A// (T==))"((w= - w())
9f
Raphy ((o) ( //A/|||(ss))
16h
9f
(K) ((=+)))////I)))))" = 9f
[((x;
16h
16h
Using Bousquet's version of Talagrand's incquality yields that, for any 2 > 0,
hi
3/2
1/2
J2 +
h 2
(2r,1) < ())(r.)) + (E 1/2
1/2
2r
4
nh
(4r)2 3n
1
hz
3
241
1/2
jz +
h
[E + BB((r1) +
+
4-
4
nh.
(4r)2
3n
5 BB(
3v2u1 1/2
fz
13 h2
(12)
4
nh
(4r)²n 3
holds with probability at least 1- To bound the expectation Es(r, 1), using Rademacher symmetrization and the connection between Gaussian and Rademacher complexities, Lemma 5.5 in Ledoux & Talagrand 1991h, we have
EQ(r, I) 2 E
sup (WXw)))((r.))
Guow
(13)
where
Gwww"
22 e < h/4) A/1lA))))))))))))))) th/a ((a" (k(1)))
nh Σ
REA..
and e; are independent standard normal variables. Note that Gw a Gaussian process conditioned on (k) (k) [ for & Am U (0)- For (w, and (wo, w write A w" A' w' wnx and (w a
21

---

Published in Transactions on Machine Learning Research (01/2024)
X 1[kl I h/4), then
Gw.w Goo wi-
Guo. Gw W'A GWit'+ Gwiw
nh ((x k,
1
Σ
/////1 ((2;) (== - -w(t))
((o (()))}}
+ nh CexeWin/a( (w w((l)) ((=")
1
Σ k,
Pk/(2)) (|/|\
Note that Pn and the are Lipschitz contimuous, and pπ(④) - (R/2)? Let E" be the conditional expectation
a given {(v (k) (x ) Consequently,
EE* (Gw... Gwr S
8
h
Σ
X (x
((x) w'))
(nh)²
4r
Ae ((0)
Σ
(a( (k) ( w¹))
(14)
4r
kEAAm Ufd} ==1
and
E.
w0 (wi'wo+
(Gw
(15)
h
2
nk
Σ
<
(nh)²
2r
k{A 1=1
Tik
Σ
X ( - A1/IA/
(16)
2r
kEAm
Motivated by the last two incqualities, we have
E* (Gwww - Gwr,arr))
Σ
Σ
(+) w'))²
21
kEAm ((0) i=1
Σ
Σx xa (x) ( A1/I)//1
2r
BEAmU(0) i=1
Define another Gaussian process Zu,wr
Bww
1
Σ
T1
(x (W) wh))
21/2,2
kEAmU i=1
+ 21/2rn
1
Σ
Σ
dl (x
(2))" ( A/I|
kEAm U(0) t=1
such that E (Gw.w (wr,w")) < E (Z Eus w )z, where {ef and {el') are dependent copies of
{e,). Applying Theorem 7.2.11 in Vershynin 2018), obtain
E. sup Garw-
S R sup Zesswr)) w.
(17)
22

---

Published in Transactions on Machine Learning Research (01/2024)
To bound the supremum of Zw using the cone constraint and I - w 1 S d, we have
E'
sup ww.
2d
E
1
exxax (K)
2r
w _,
1=1 REAm U{0)
221
E
1
nk
4 X (K)
(18)
2r
n2
Am
(0)
Thus, by 13
17 and
18
have
EΩ(r.) S V π
Σ
Σ i=1
x
(K)
E
Σ
x
(19)
BE UEA
RE
U{0)
i=1
It remains to find the bound the two lo--norn terms on the right-hamd hand side of n 1T-1 _kEAm)u() E exxra
19
Note that the variable
(k)
is zero-mean for j ,,...,..
)]
de xe(xcart (8)
{
E &
(K) ex
E exp
S
E
exp
f=1
x²x (10 (k)
exp
2n²
f=1
X2
The
(k)
exp
Xs
(1
2n
Thus, |n- ExEAm E exeaiy
(k)
is sub-Gaussian with parameter n-
Σx
Σ Th
x
( (2)
Applying Lemma 15 in Loh & Wainwright 2015
we have
E
x
e
max
Σ> (2
(2) 22
olggp
n
g......p
1=1
CV Togg
x2 (1
12
2=1
implying that
n4
(k) 1
log
Σ nik
X (x (K)
5 0
E
n
n
1=1
Σ k Σ T (3( (1))
log P
71.
log
9
< C0
(20)
16
Similarly,
E
nk
X
(K)
co
log n
9
(21)
16
Finally, we take r r= 1/(44//41))))) d = h( 1/2 11/2 //22/(33m)) and 2 = + logp, combining
19
20
21
with
Bousquet's version incquality 12
we conclude that
2(r,1) < 0.01 elly (1+ logp) (nh)
23

---

Published in Transactions on Machine Learning Research (01/2024)
with probability at least 1 p='e- for any t t> 0, as long
nh 2 12 / log(p).
This, together with 9
10 and 11 we have
T(w, w") 2 |
0.211 - (0.01f+
1+ ++-log
nh
logp
0.2f cW
(22)
nhh
with probability at least p e-
It remains to extend this bound to that is uniform in the ratio argument. Consider the inequality
|| which do via pecling
E.
+ log
T(w,w))
(23)
10
| Σ
nh
Since
71
1/2
|
||||
20c"
K
+
nh
log
5.
we define the set
; = fo Rº 111 1/222 S || S S 57 1/2221
for each & = 1, 1,..., N = [loge (cv //c/////////// nh/logg)) to let
|
A : 1/2 <
SC
CUNTONN
C
A Σ
By the union bound, we then have
AA E R. 71
1/2
S
11A11/1AA| SC s.L.
K.
T(w,
2c" ||ll
I log
10
|
nh
N
P
3A E
O₂ s.t.
K.
T(w,w") > 6012-11/24 V
11/22
+ logg
10
|
n/
N
K K-f
T(w, w*) )² > >"" 11/2gh
t + logp
P
sup
V
k=1
||||//I|
1= 1/2gt
10
A
nh
N
et N P-
Taking t = log (c///// nh/ log + yields that with probability at least 1 - p p-
K f
1 T(w <
2c"||A111
logp + log(logg(()) (c nh/ log +
10
|
nh
Multiplying by || 33 on both sides yiclds
log + loggooggoooooooooooooooo flog, (c nh/ logp))
T(w, w') N
10
nh
where c" > O is constant depending only (1s. D)
24

---

Published in Transactions on Machine Learning Research (01/2024)
A.3 Proof of Proposition 3.2
The Taylor error around B2 in the direction 81 B2 is given by
T(31. B2) (0) (B1) Q (32) (8:)) (3: -2).
For given kernel function (-) and bandwidth > O, the smoothed quantile loss Q can be written as (nog) Di= Pr (=( {(u (a) ( y. ) }du. Therefore
T(31,82)= 2no i=1
Kg - (0- (aa - -B))3
Kg - -1)
31) --0 -
2no
for some E [0, 1], For each i, define the event Ei,
= [|ell S 9/4) n ( y² B2)l S 911 B2l1x/((r)) n (1(x 8a)1 s 9/4),
for all B1 B₂ (r))) Thus
no
T(31,
(0) 15 1e.
(24)
2nag
where 5 = =B1- .2. For truncation level R 0, define functions
u2
lu|
S R
R 2
(R(U)=
(R
|u])
[u| S R. 2
0
|u| R.
By this construction, (p((u) S u22 1[u] S R), Pen(cc)= = pu(()) and yn is R-Lipschitz. In addition, we define the trapezoidal function
1
lu| S 2'
(n((
2 - 2
|u|
g < uu| - R.
0
|u| > R
and note that vhn is (2/RR--ipschitz and (u) < L| S R).
From these two new-defined function,
24 implies
Ro
T(81. 82) 2
K 2nog
|
1 1|/ s g/4) (////////////////(((11/4 ((² (( (z= -B*)
|
1(el g/4))///4/////////////////////// (1 (100)) - -B¹))
(25)
2
nog
Do(B..82)
In the following proofs, we bound EDo(S1, B2) and Do B2) - EDv(31, B2), respectively. Note that
クい fe (0)
g
B[1fle/ s g/4
E[1 f[1fle] < g/4}/= (0) ~/ f( (0)
/
1//4
Meta (1) - (0)
g/4
25

---

Published in Transactions on Machine Learning Research (01/2024)
Then,
9 2 fz (0) - [[1[|e 5 59/4)
lug
_ 16
g x| 1/0 N 2 f
log²
16
Provided 9 f/lo f/lu, have
B[][|e] S 5//43/=0 [0)
7fg
16
Meanwhile
/PvA - (fe (0)dt
00/2
E fla S g/4)|== [0 - 12 f2 -
implies
9f9
B[1[/e| s 9/4//=//11
16
Then
EDo 12)
1
no.
E
[ (le) 9/4)
(1/ "/) (( 831]
no9
7
x 8/ (111812)0,,,000 -81- 87)]]
2
E
(2)((2))
16
7f
2
1 -
E
(x 8/110 2 1/10|11 g/(4))
16
- E
(a (a"8//11 8/1181 2) 1
(4) t/81/e
2
7L T6
1 - E (
//110|=11 > 9/(4r)} - 1 -B)) g/8) 1/2 //8
By the definition of n6: as long as 0 < - g/( (4m//44)
E (E
sup
> 8/(4r)) 5 4
Moreover, 81 8" + Bx(r/2). Under Assumption 3.3 with vo the tail bounds of sub-exponential
= --11/2 implys that
(B 8a) > g/8) < 2 exp {
40or
Let g Sugry
1/2 and then it follows from numerical calculation that
{
1 - E a 0/11|1]))))) 2 1 5/|||||s] > g/(4r)) - / (|2 (B) g/8) 9/8)| 1/2
2 0.23
holds uniformly over A Bx(r) Cs((). Putting together the pieces yields
EDo(B1, > 0.1f.
(26)
Next we find a lower bound of Do(BI, (()B,,B))-00 (2)33)--- DD--ED((91,330 over A(r, = ((B1. 31 E 8" + BB(rr/)) 82 E
81 CC()) supp((:)) C S}. Define
2(r,1)
sup
{-Do (2) + (D,,
(8) (2)AA(()))))
26

---

Published in Transactions on Machine Learning Research (01/2024)
Write B2) = (1/no) E= (B1, where
wi(B1. [2) = و f(ca = 9/4//// (= 4//4 ((x) ((x))) (T -3)-18))
satisfies 0 S w,(B1,82) gg((4) since pл(u) - (R/2)² and (T(4) E [0, 1]. Moreover,
(u2 E 2 21[l S g/ 9/44/// ( ) /1/1////112))3/ ((x) 00 (B: -99)
< 9f Ep/ (10)) 10 /
16g
9f
" 9f
$
16g
16g
Using Bousquet's version of Talagrand's inequality yields that, for any 2 > 0,
1)r,1) < ( + (EQ(r,1))')
1
g2
3/2244
1/2
V
fz
+
2r
110
nog
(4r)
3no
< (E + E
1
3/2uu
1/2
fz +
9 2
4r n10
4
nog
(4r)
3no
El( 3v/211 1/2
fz
13
gz
(27)
nog
3
(4r)
holds with probability at least 1- To bound the expectation Est(r 1), using Rademacher symmetrization and the connection between Gaussian and Rademacher complexities, Lemma 5.5 in Ledoux & Talagrand 1991h, we have
EQ(r,) 5 22
2 E sup (B1.)))
Gp
(28)
where GBL.P2 = (nog)- no e S gg/4)/9//(er) ((a5 (0) (1 (()" 3²)) and e; are inde-
p, is Gaussian process conditioned on {w) (0) (0)
pendent standard normal variables. Note that Go, and G = 0. For (2) and (8i, B2), write ō = Bi B2, 8' Bi Bg and Xi = 1(eal S g/4]. then
GB GB
= Gg 02
G G M 48 + G3 G39
1
exi (2)) (ax ///1///02))))))))/))))) x)) / (
(0) (()) -3-) - /////(( (1) -81)}
=
nog
1 no
ex /
( -()) ((x) (0)
- (
- (fg((2)) (1x (0) 5' /10 (mx) }s>}}
nog
Note that fo and UR are Lipschitz continuous, and pR(()) < (R/2)². Let E be the conditional expectation given ((v) (0) (0)
2 )e Consequently,
E` (Ga.s, - GB), 2 (nog)
8
Xi
(( (0) ( -1))²
(
4r
i=1
Xi (()(()) -11))²
(29)
4 n
and
no
Ex (G Gg
(xx )0 (
(nog)²
2r
1
ng
xi (m (1) ( 8' 8//110'1=)))
(30)
2 i=1
27

---

Published in Transactions on Machine Learning Research (01/2024)
Motivated by the last two incqualities, we have
E" (G Gp) )
2 (1 ( - + 2 i=1
Xi ((a) (0) ( 8'/11''(1=))=
Define another Gaussian process B2 as
Ro
(0) -8)) V2rna
c (=x 3:)/11011=
m2
No
1h
(x)) (0) (B= -B')s +
xxx (0)
( 3:)/11011s
V2
such that E* (G s; S E* (Zp. - Ep, My )2, where {e(} and {e/} two dependent copies
of {ee). The second equlity holds since supp(Bi), supp((r) S 8. Applying Theorem 7.2.11 in Vershynin (2018), we obtain
E' sup Gp 81.82
S E' sup 2p,
(31)
81.82
To bound the supremum of Lg,0,, using the cone constraint and B1 8 + By(r/2), we have
/2
1/2 21
E sup Eg
3,082
<
4r
E
1.
2r
4 x
1=1
1=1
$
V2
9fg 8
221
E
1
ng
X
00)
(32)
4r
16
2r
n1
where S= Ess -Exssz)). Thus, by
28
31
and
32
we have
ER(r, S
3
fgs
+
E
no
xx
(0)
(33)
8r
no
It remains to find the bound of the second term the right-hand side of In 1 Eie e xax zero-mean for j ,.....P. P.
33
Note that the variable
E
exp
1 2
Σ
X
(0)
)
<
II exp
Xe
(0) Xe
no
14a
i=1
1AO
X² (z:1g) )2
exp
2m
1=1
X2
xi (x) (0)
exp
22m
im1
Thus, In In'' Eie
X
(0)
is sub-Gaussian with parameter ny""
Ee x2 0 ) Applying Lemma 15 in
Loh & Wainwright (2015)
for some universal constant ea > 0, we have
np
ng
E
X
00)
(0)
max j Fh....p
x2 (0) y²
logg
T1o
/ llg
nh
Σ
x² 10)
T1g
28

---

Published in Transactions on Machine Learning Research (01/2024)
implying that
E
1
No
c
X
00)
c1
log
E
ng
X² (z
120
no
n1
llog
Σ
i= X1 (1)))
C1
B
n1
n10
log
9f9
S c1001
11
Therefore,
nl
i
(0)
0z
fgg logg
(34)
no
4
740
i=1
Plug this bound to 33
we obtain
igs +
301
fglogp
EΩ(r, 1) S Vx
(35)
8r
4r
Finally, we take r = min(g/ (4////44/////// g/ (8vo)12 )} and t log p. combining
35
with Bousquet's version
inequality 277 we conclude that
15
fgs
15c1o
fg fologg
3 3/2
1/2
fz +
13 g2
0(r π
32r
n1
16r
n1
nog
3
(4r)
< 0.02 + dll// (1+ log p) (nog)
with probability least p --t as long as nug 2 f1 nh M This, together with
24
25
and
26
we have
T(B1,
R
1100
0.1f
0..2f tcly
+ log
2
nog
E 181
0.08f
-cly
t log nog
(36)
2
with probability at least 1 1- 1-pp-ee-
It remains to extend this bound to one that is uniform in the ratio argument. Consider the inequality
| which do via pecling
K.
+ log
T(8i 12)
|801
2c"
25
nog
R.f.
10111
+ log
T(B1, (2) 2
||8 |
25
|18 M
nog
5 f
2c |0]
++ log
T(B1. (82) <
25
||8|/3
| >
nog
For positive integers k 1,...,N = [log (cv /og// log p)
define the set O₁ {8 E I2P m 11/2g--1 <
||||1/||||| I -1/22b), so that
SERP: 1/2 <
|
K f
Bog
:= 5' CUN
50c V
1 log
29

---

Published in Transactions on Machine Learning Research (01/2024)
Then
P
36 E (6 E RD: : 71
-1/2
<
||8||//|0|||> S { s.I.
R f
2c" |6|1
t log
T(31. 32)
25
|
001 |
nog
P
38 On s.t.
K.f
T(3,,
"2/i"1/2"
+ log
25
|
nog
sup
K K.
T(B,,
c 1/222
t + log
k=1
|
1/2kk
25
|81
nog
z [logylev nog/ log 1p-
Taking& L = log(logg (c/ /og/ logp))
yields that with probability least - 1-p -
f K.f f
2c" 2c"11011 | xx
logp + (eo log + u
T(31,
<
25
|
nog
Multiplying by | on both sides yields
K.
log log (log, (cv nog/ log p)} +
T(31,82) 2
|1813
10
25
nog
where d" is constant depending only (1s, D)
A.4 Proof of Theorem
Transferring step: Let wn be the true parameter of the transferring step and S be the active set of 3 with cardinality The symmetric Bregman divergence between Ammm and w" is defined as
(vQn(0Am) - vQ((w))) "aar))"""(a))) (wa - 22)
(37)
Let Å = Am wt By optimality, there exists a subgradient é E 0E arw (wwi)), such that
(QAm + +1 =0.
Then 37
is equivalent to
- -(VQ)(=")"'AAAa)) -
(38)
Note that
(T' - ad ) < ||00|| - |m ||| = ||زا||| + || ||| ق + will2
||l||| + ||w@=|11 | + wااl: - || + wš-1|1
||Ás||ı - -1| || + 2|1
30

---

Published in Transactions on Machine Learning Research (01/2024)
Then,
(VQN((Am) - (QN(ww)) ")=²A"
= =wO' - -mnm) + (VQx(w")- - (QN(w")) 'w" - a.m) + (VQn(w*)) (w-- - -Am)
S a ((A5/|) || || 21128=1111 |[V(n(W") vQu |عاا ||ا|
||
+ | QO |12 1|ا
(39)
Conditioned on the event {L 2 2 / bs0) 39
becomes
(VQn(OAm)) - VQ)(w)) v (لأ || + 21|w3=111) + A 112111 + &
Lemma A.1
implies |k03ol11 111 S mmm, so we have
(QQ)())))) - VQ)(w))) 'A باله - 2 2 مالl + 22 f
Since (VQn(OAm)- - VQN(W))) ( 2 0, Å satisfies the constraint || || - 3|| + +40m - -2- 1 from which it follows that
||ااا 1| < 4 || + 40 + 2X ( 1x-
(40)
Now we claim that when du 2 TT loo with probability at least 1 p-, it holds that
| |s: S 802Cim
log + log(nA... no)
3 71
1/2 82/2 26 + 2
Cidum
(41)
Φ1
1m + n2g
Φ
Φ1
If the claim does not hold, consider C = (A 1.55 0.5A~ any & E (0, 1),
0}. For
1
1 -سا / St. 33 (اااقاااا + + +7
2 A |||
$ Aw fu
+ 2\ Cim by /
which implies that t E C. We could find some satisfying that | 1 1 1 and
802Cim
logp + log(nAm + 3/w7 1/2 /2/1/2 + 26
C1
|| Y
+ 2
Φ1
nAA.. +
Φ1
Denote A +. and F(A) = Qu(w" + +1) (w +~ + +1ll1 - -||w*|11 by convexity,
Since F(0) = 0 and F(I) 0.
F(I) F(t +1 (1- stF(A) 00
However,
F(A) DA(A) \e/W"111 + ww' رl
= -R((A) + (VQN(w))) ")²²²)" A ااا|ا l|1 + -wl|o* + +1ll1
-Ra(A)) (Nullo~11 - \w| + ر|| (VQ)(w") - VQN(w)))
-(VQN(w')) "²)).
31

---

Published in Transactions on Machine Learning Research (01/2024)
Then by Proposition 3.1 and
39
log log n2
_ اااقااسسة + - =
nh Cim by | 1>: v(A) +1
13 d22
||
logp log n nh
2 2011 1/2
| 2 االهااله - -m -6,11
Note that |اsl| & 3//2113112 2 2 2/2 | 3 and 40 Therefore, when
(nr. + no)h 16 の5 (log log n) max(16s7 4x 2 2b
have Φ2 / (logp + log ( (nd)://n 1/2 b S 01 Then follows that,
logp + log Cim + Nwn 1/2 by || 21m
F(A) 2 2 1 4
nh
00
which contradicts with F(Ā) < 0. Therefore the claim holds.
It remains to control the probability of the event {A 2 | and the probability of the local RSC condition. By Lemma A.2 we pick
4 4log(2p)
+ max(1 - T,T) 2 (2p)
A = 2 ( [T(1 - [)(1-))+ Ch2) nm no
nA n1
so that {Aw 2 2 1loc From Lomma A.3
we have by s Ch Now with probability at least 1 (pn)=1,
logp log n Am no)
slogp
log
||ا| $ m
+
+ h22 +
m.
n4Am + n1
n1 + n10
11 + n2
We then let 12 < 8//2> Xw,, that
logp + log(n + no)
s slogp
logp
| S 2 mmy
my
+ V
+
Vm.
(42)
nd n2
ndm 120
12 + no
with probability at least (mm) -1.
Note that
|| S + 4Cmm
2x_ 62 ارل
s <4v5|| 1|2 + 40 + 2x_ 69 1
(AVan) 1/2 1oxe2 h | + 401m,
which encloses
s slog(p) + log(n),. + +1))
log +
log
|ال S m
+ +
V + m,
n1 + 740
n1 + n10
11,4m. + Y10
with probability at least 1- 1pm))~
Debiasing step: Denote 5" 8r w" 8Am B a. and Jmn 8mhm 8"
32

---

Published in Transactions on Machine Learning Research (01/2024)
Similar to 39
we have
(V0 + (VA - )²)
- (110 - - 111 - |13 س- 111) + | vo
(8))
|13 - 111
||
+
1211/00000 (B*) ||B 1B-1l
3
As|1B* @mmm
|||
2 As - @m ||| + -bgll3 -B"11
2
3
3
As|1B* w
111 +
Asl ||| 2 2sl1 -- |/1 + b/g +b/l/ | 8 11>1s
2
3
Am + 2 |1ll
3
3 1-11 As|1B - - |11 + -
(43)
2
2
On the other hand,
(VQ)0) + 84m)
T(3 3²)
S Ao \ amm 1|1 - ||3 ش- 111) + A8 || - - + +bg13 -9011x
2
S As 65-1 |1| + 15 25sn1 11 As \s 255"m |l As Ass1 asyt ||
do
+ 2 | - 8111, + b -1
$ As (1183 0z Am |11 - ||As azn1 11) Ao (1185- az 1|1 + +113s- azstm 111)
+ 21.11 -5 111 +
2/1 110 1111, + +bb -B'1ls
A
2
< Ao - Asll 1s
As-11 + -228118s= - wssell + 2Ap As |l
+ 2 211 8111, + 111l3 11
A6
3
1
As|1B3 Aslli
\o
11ss-- -Bs- + 2m + 2AslI |11 + +b]113 B
2
2
3
1
Asl1@
Aslli
1185 As + 22scim + b - 18
2
2
llg(() + s log(nA.. + nno)
logp + n. + n10
log
+
2Ao
ml
+ + nl.m. + ni0
Vsm + m)
(44)
n.A.. + 710
Thus
1/2 Vs|lB* - -111= + 26,x5'113 "13"" - 1111"110000 -111 118- B111 S 471 1/2
<log(p) + slog(nAm +m)
log
logp
+ m
+ + 12Am +no
Vsm + m
(45)
1mm + no
11 + -2
Set 9/8cc) for some > 0 and R (47) 1/2 V5 + 265 )r + 4C,5, if m I CV8 for some positive
constant C and RAA.. + nn 2 s logp. p. Denote 0(r, Rt) = B and B = (1 n))8²* + n18, where " sup(u E 10.1]: 8 + u( B*) E 8 + Or. R)). If â & O(r, R), then 17 E (0. 1) and / falls onto the
boundary of @(r, R);
Rr; otherwise 9.
Combining
43 and Proposition
3.2
we have
Q1 ||3 1-1 02
log + log no ||
1r1 S
3
AsOm 3 >dlla|11 + +b1 -88 1811.
3
(46)
2
nog
2
33

---

Published in Transactions on Machine Learning Research (01/2024)
Besides,
43
implics
2b
||3 amm1 - 33 +
1/3 18" 1ss
As
As result,
||3 - 18 511 - ammml ||| + 110 -8
26
S 40m + 4||
1 3 381100
Let
01
-bg225 then 460 becomes
logp + log no
ail1B 11 202
(16Cm² 16 Áll 26 2/1 1311
nng
3A8Cm 3 ر|| I|1 + 26 3011s
h+2
logp
alB
26y113
g 11x
9
(m² ف l + ASm + Assl
a
nog
b
log P
|13
1188-81113
3011
1|
9
m + ||||
Asm + 1o11
Nog
Thus,
logp
| 1/ S
(m -1(111)) A v Adll + +bg-
nog
Let As Colol(p)mmo and g (TO)) then
logp"
3/8
log
1/4
log
1/4
'logp
1/2
|| 15
||
vm
V /|7||1
no
no
10p
n2
log p
3/8
logp
3/8
log P
m
+ 8
71
no
72
+ 770
log
3/8
log
1/4
logp
1/4
Vsm
vm
70
12.m
+ no
log
1/4
logp
1/4
logg
1/4
log p
1/8
/s
+
(sm)
1/4
77g
TiAm.. + no
11
+ 12g
If no > log e falls in the interior of ((r, R),, so we must have E O(r, R). Consequently, A 9 satisfies the claimed bound,
log no
1/4
log
1/4
log 11 + my
1/4
11 1l Vm
+
Vs
(47)
no
In addition, if m < 8 >//og(p)/mm) the above upper bound is sharper than /slog(p)/ng Then by 45 we have
logp
log
|| 8" S SV
+
/ - m.
nA +
12 no
A.5 Proof of Proposition 3.3
The method is similar the proof of Proposition
3.2 At first, the divergence is given by
D(B1, B2) (VO (8)) -QS"0 (B2) -22)-
(B1
34

---

Published in Transactions on Machine Learning Research (01/2024)
For given kernel function K()) and bandwidth 0, the smoothed quantile loss Q8 be written - Pr (r(u)((( (x) yT y, ) //gdd.. Therefore
no
D(31.82) 2
K
((x) ((x yT -B2)) 1
nog
where the event &, is defined by,
& = Hlal S g/44 ( B2) s 911|3 B2l1z/(2r) n ( 80) s 9/4].
for all $1 B2 Br(r). For truncation level R > 0, define functions pR(u) and (T(()) previous proof. By this construction, (p(()) S u² x R}, PeR(cu)) = pR((),, VR is R-Lipschitz, vR is (2/ss-Lipschitz and (a) < 1[lu= - R}. From these two new-defined function, we have
8g Heel s g//4 9//44///10111//mm ( - 3²))
D(81, B2) 2
| |
nog
Hea
/9/4//9//22)
((28) -
(48)
Do(81.82)
where 8 -1 B2. Finally, with similar proof as Proposition 3.2 if miin///44m/a)) ./ (3200//// and
nog 2 ff f log then
Do(81, 0.1f.
with probability at least 1 (pma)-1 Therefore,
D(31, 2 0.1 fl|B B2113.
A.6 Proof Theorem 3.2
For step 1, the parameter (k) most s + sparse. Therefore, similarly as Theorem in Tan et al. 2022),
have
(s + m) logp [k))() (() (s + m)
logp (())
[10(0) (1)) <
k E A',
(k)
with probability at least 1 - -p provided that the bandwidth satisfies
m)
logr
of + m) logr
S min( (8 x()))
max
(b)
f2
(k)
where 0 max12/2p Oj are the diagonal elements of Σ.
For step 2, denote (bb) = 3 (h), g(k) = (k) and ((k) 8(k) - 8(h), For each k A'm
(V0 + 8(()) - (O(") + (()))) T ((())
S As (115%11 - -1 -118(*111) (11) + (VQ 10) (A) + 8*) - (2.0 -8() + ( (8") VQ, (8) ~ |,(0 + Σ Σ 1/1122Q (8 1|
(49)
||
35

---

Published in Transactions on Machine Learning Research (01/2024)
Then by Lemma A.4 with 2 2ogg p and for each & A'm: we let rh v(s m) log(p)/ (s m) / (loo(p)//// (1)
and 4.
(8) We have when As 2 2lo
(VQ,) (x) ) (() + 8(4))) ( ((())
$ 16(118 - 111( + c
8 + m
logr p
log
log
V m
log p
||
n (k)
no
no
(2]
Ao
2 | + 1,110(101>>
Since
| |||
S. - (1(8(k)) -8(h)) and ||055 [1
|8 1I = =11(((() -8(4)) sc 11a.
when C. 22, obtain
= (3 2 + C. (.))1((1)))) -8())) -8()))) + +,,1100001
3
(6)) + ))(h))² ((()) ))()) + 80)) (
S 1/2 3
(2 + Co |10000112 + 15]10(011)
By Proposition 3.3, the RSC of (VQ), (6)1)) + 8(0)) - vQ. (w)(1) + =0(4)) )(((), we have
0.1 f| I m²22 ( [ |10000112 (.,,)>000)11 13116(5111>
with probability at least 1 - (pmo)- Therefore, if we let g² S 1/
|
m mlgg
n0
By Lemma 17 in Yuan et al. 20188 and the condition mn S / noo/ log we have
m logg and 5(()) - 8000 $ my
logp
(0(0) 5(h)
no
no
For step 3, let ((0) =0) 0, then the loss function in step 3 could be written as:
(v) (X (K) (T -3-(())) = Σ
(k)
200 - g(())
no + T.A Ke (U)UAA 1=1
KE REFOPLLAS
The symmetric Bregman divergence is defined as
Σ
(VQ 5(0) - (Q(00 -8(0)) 8*).
AE(O)UAA
36

---

Published in Transactions on Machine Learning Research (01/2024)
To simplify the notations, define (g) EkE(OJUAL, VQ (B 8(4)). Similarly as above, we have an oracle inequality for B,
(B) - (81) (3
A (1131 - 111111)+ + E (VOL) + -(1)) (w)))) 3)
RE(O)UA
Σ (w (w)) (z)) -(3"-8)) -B) Σ (VOL (((00))))("""""
"(0 - A)
RE(OLUAS
kE
$ >>0(|10111
(1 +
E
(VQ (w) (K) + -(()) - (W) (w) "(3" - 3)
REfOPLLY
Σ
(w)))) (w |1l |13- - $11
kE (O)OLAA
11 (w)(
Σ
E 1/2 (Q(h (8)
||1 118- -1
AEÇOJUAR
bg
For the second term above, by Lemma A.4
(W + 8(4)
8(8))
( (k)) "(8" ()
logp
log P
logp
log p
C
+
Vmn
|| - 38
n4p
no + n.A
no + TAA
no
C%
If we set A3 2 T ooo and C% S Ag/2, then
Rr (3) - v (3" 8r)
Ag + 0¹ (2)1((8* -B) + b 65118r - -B11s
2
?d + C: (r:)118- 1180- -1112 + +b - B11
3
5 1/2
Under the RSC of (Q) v R.. (3*) T 8- B*), we have
(v v Ru (8))) (a= - 3r) 2 ci 2 - -B11
with probability as least (pn)-1, where na + nA and ci positive constant. The proof of the
RSC in step 3 is similar to Proposition 3.3 Thus,
|| - -11 S 81/2
3 2 ( C/ 777 -1/2 + bb
Through a similar proof as Lemma
A.2
we obtain Ao Vlogl)//n If log(p)/n S w² < st/2 have
logg
sm logg and || -Ô11 < log + sy
log
||8r -B11 S
n
n1
n10
with probability at least 1 p1
37

---

Published in Transactions on Machine Learning Research (01/2024)
B Proof of Lemmas
B.1 Proof of Lemma A.1
Define (8) for all 0 < k < K as the true parameters of each local source model, then note that (w())
and VQ(W") Σx o VQ(H) = O. So we have
VQ(W))- VQ(B)) VQ(0)) ax (w(()(((()))) =0
VQ(W)) VQ((*) =
0k (Q( VQ(0))
Note that (w())) = =)0 0, so
Æ a 0 (VQ(()))))) (w)) - (B))) = Σ
x
a (w)( - (B
k=
k=1
By the second-order Taylor expansions and Assumption 3.4
K Σ ak { ( ((1-t)@* + "dt'd((w" - -3") Σ ak /
+ tw (N) - -3"
km0
Iw0- - Bn s
a | |1-15(0011 1|w - 1-
By the definition of the parameter space
m(s = (8r, (w)00)) 113" s, sup [|)(0)) 30112 S m m) kedm
We have (R) 11 S m. Let C₁ sup, ||51 Then Lemma A.1 proved.
B.2 Proof of Lemma A.2 For the transferring steps,
V@n(w) RAA + T4)
11g
K
(xr
(k)
Tw
Vi
(K)
(k)
k==
v2Qu(w) = RAn + no
K
K
(x
(k)yTT
ys
(2)
2
(x.
(k)
h
1=
1=1
Let c((h) = R1 (1((==())) ///b T, then VQu(w) (nA... + +no) -1 =k=
(k) xx
(k)
and
|| nAm.. ng k=0 1=1
x
(k)
BiS! (k) (x,))))
x.
The upper bound of |l involves two quantitics that are related to
(w (()) - -E) (0)) and E
(zo (zo(t) (w - (ki) - (A))
E
R2
h
h
38

---

Published in Transactions on Machine Learning Research (01/2024)
For the first term, by change of variable and integration by parts, we obtain
E R2 (al w(h)) ) 20( (2(-n//))ss
h
h
(
2
(50)
By the fact that (0) = T, we have
Frjor (-vh) (r) - (cla(t)dt
(r-heee=00) + / oh (Sepa(1) - fela(
(51)
Morcover, it can be shown that
OK = wK (K(D) I.d K(w)(1 K(u))du > 0 and ax S 11,
(52)
where K1 =
f |u| K du.
Substituting
51 into 50 and by 52 we obtain
(e(())T((u w w()) - (K)
000 (())R(v)dd 20fe)(( / 00
R-2
= 2F
(u))
h
-00
vh
+
(fepo(t) (())
J0
2axhfeea(00) + 1nhh 1 (²K(())(()dd
S + lo8gh?
where the first inequality holds using the Lipschitz condition on fej in Assumption 3.1 and the last inequality holds by Assumption 3.2 Through a similar calculation, the Lipschitz condition on fele cnsures that
(x (x())) (w (*)) -2) (2((3)) S
to high?
E
2
Hence
B(E(*) (k))2 = Eg (x (1)g2 E((E! (k) ( (+)
zij x15))
2
w
B(g2 (+) = E
K
-
T
m
E
K2
T
y
E
x T
t+2
h
Sr+Hangh2
2r- 1p kgha
S [ Ch?
where C= (T + 1)lon2. Then, by Assumption 4.3, have
B(E(AA) Jij (10))2 7(1 T)ais Caigh?
39

---

Published in Transactions on Machine Learning Research (01/2024)
Also by Assumption 3.3 and | v max(1 T,,),, for d -3,,.... ,,.....
E(Is) (( (k) 2ig (4) 1 max(1 - TT, 2 (1018 (2 TE((E) ((z)))) (0
S max(1 T, (7,))-2(7(11 7) + Ch²)
d! 2 [ r) + +h²) max(1 T, 7) T)
Thus it follows from Bernstein's inequality and union bound that for every / 2 0,
| & sa soy (r(1 - cha)
21 ndm. + no
+ max(1 - TT,T) nMm + 120
t
with probability at least 1 - 2pe
For the dobiasing step, through the similar proof we could get same results with different sample size and smoothing bandwidth.
B.3 Proof of Lemma
A.3
Note that
| 25-1/200 1(h""\]
- y)th) (g)())
-
(x()) ( 1)
Σ 1-1 1/2
IE
K
h
k=
s7(1) )
sup
K
- -
C (8)) T
h=
1g
K2h
2
B.4 Proof of Lemma A.4
For h: -,...,P, ,,, define that
du (r,1)= = sup vEBe(r)n3,())
- (1 E) (n - -i) Kn(-es)) Ei) )
n
where v = - gr. Note that (br,1) < (We)))) + Ego #:) In the following, we bound (x( and Egu, u,), respectively.
Let be any positive constant such that o² 2 SuppeRg(A)BR;()) Eg. (3t. x.). By the bounded design, we note that sup, I (yi, æx)] < [osl < 1. Applying Theorem 7.3 in Bousquet 2003 Bousquet's 's version of Talagrand's inequality, obtain that for any O,
2z (x(r,)) (xx(r,)) v 2B0x,rr 0)}
(53)
n 3n
40

---

Published in Transactions on Machine Learning Research (01/2024)
holds with probability at least 1 - c- For the second moment (10 by change of variable and Minkowski's integral inequality we derive that.
2i) =
{Kn -Rx())))) Sel (t)dt
{Kn(u)- - Knnu --,-x, x) fertor (x - u)du
hE
(R(v) - R(v - x//)/1))² Ses)a (e (x) - wh)dw
( K(u wxx u///ud de
$ fh E
1it (a b)²
1/2
1/-0
00 K2 w
S fh 1F
(m) v)²
dw
-
S Rfh Rffh-B((xk x) S R.hh112
It remains to bound (r)(r,)) Note that A x;) go & ( x4) < (R/h)laTT xTool, for any v, v'. Hence using Rademacher symmetrization and Talagrand's contraction principle, we have
(r,,(r, 1) $ 2E
sup (rBn()))
Ci (Ui, x;:)
< 4RE
sup (r
4 h
(54)
nh
where en are independent Rademacher variables. Applying Hoeffding's moment inequality,
1/2
/2 2 log(2))
Eg
c
max Isk
zik
(55)
n
where Eg denotes the expectation over {ei
By
obtain 54 and
55
/
log(2p)
Eth(r,
< 4
h
Taking t log p in 53 we have that
1 logg. p + f//3
t + logp
+ logp
(x(r, () S h
(56)
nh
holds with probability least - -6
Next we find an union upper bound of Egg.A & (vi, x:)l. Similarly as the method to bound the second moment, we derive that
Ego, (Vi, mx)) hE
(K(v) K(v x///h)] Se,1o, (xf - vh)du
< fEE
Ja loT m
K(v - wxx v/ dw du
|0 v) [.''
fE
K(v way dw
R fB aaoo 5 Rfrr.
41

---

Published in Transactions on Machine Learning Research (01/2024)
Finally taking the union bound, we obtain that with probability at least 1 en
((r 1) - 1 logp f1/2 ery
t logp
+ t log
n
nh
n
C Empirical results on the similarity of target and source data
This section presents the empirical results of the relationship between target and source data with different degrees of similarity to the target. For the similarity defined in 6-norm of the contrast of each source, we consider p 500 and the sample size of the target and each source is 400. For the transferable source k, we let w = 3" + (m/p)R,.) (M) to satisfy the transferring level ||8(0)/12 = ||](*) 80111 S m, where 3 is the target parameter, R,²²) is a vector of p independent Rademacher variables, and m = 10. For any source data 1'' that is not transferable, we let (k') 3 + (2m/p)R (K), All the other scttings are the same as the numerical studies in section
Figure 6 and illustrate that when the contrast is relatively small in 4--ormm there is significant overlap between the target and source data. When the contrast is relatively large, the source data would have much more frequency at the two tails, which may cause the negative transfer if those sources are used in transfer learning.
target transferable source
target transferblle source
Desquenc
75-
Troquencyy
25
12
30
10
(a) [[6(k) |/2 < 10
(b) 1|0( S 10
tarpet
35
trrett
frequency
35
15
30
10
10.0
-7.5
5.0
2.5
0.8
5.0
7.5
10.0
10.0
-75
5.0
-25
0.0
25
5.0
7.5
35.0
Veluam
(c)
16(k'111 >
10
(d)
| 8
111
10
Figure 6: The predictors are from I-distributions with 4 degrees of freedom.
For the similarity defined in fo-norm of the contrast of cach source, we consider p 500 and the sample size
of the target and each source 400. For the transferable source k, (k) is generated from w = 3 2- e M), where M is random subset of (p] with |M] = 2. For the source k' that is not transferable, (k') is generated from w/
(kk) = 33 + 2 1(j E M'), where M' is random subset of jp] with |M' = 4.
Figure and 9 demonstrate that with a relatively small contrast in norm,, most of the target and source data also overlap. However, casy to observe that the source data has distribution with relatively long tail. Conversely, when the contrast becomes larger, the distribution of the source data at the tail becomes more distinet from the distribution of the target.
42

---

Published in Transactions on Machine Learning Research (01/2024)
Gaussian
Gaussian
sagett
30
30
25
Frequency
20
-10.0 -7.5 -50
10.0
-100
-75
-5.0
-2.5
0.0
(a) [|8(6) |/1 S 10
(b) 110(5) & 10
Caussian
Caussian
target
trrget
source
source
Preeuercy
20
20
23
10-
10
-10.0 -7.5
-5.0
-2.
0.0 Values
255
500
75
10.0
-10.0
-75
-5.0
-2.5
0.0
2.5
5.0
7.5
30.0
Minuss
(c) 160h
||1 >
(d)
| |8(K
/112
10
Figure 7: The prodictors are from Gaussian distributions.
40-
monchattle COATOA
40-
mandrable courte
10
10-
-00
-25
-10
10
-10
-30
300
33
(a) (|6(0) 1o 2
(b) 118000 llo 2
targst
tarpet
30
12-
valans
salue
(c) ||8
l1oo >
(d) ||8
llo >
Figure& 8: The predictors are from t-distributions with 4 degrees of freedom.
43

---

Published in Transactions on Machine Learning Research (01/2024)
transterate source
transtrretee
-00
-1
-10
20
12
20
20
-1
-10
-5
20
12
10
(a) [|800] |lo 2
(b) || llo S 2
Goussian
Caresian
Iarge
taree
40
30-
30
10-
-1
Valaes
values
(c) [18(4)) 11oo |lo >
(d) [16(k)) 116 >
Figure 9: The predictors are from Gaussian distributions.
44


Reviews:
[{"review_id": "BcS3LuOZD9", "reviewer": "Reviewer_CHYg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths \n\n1) The proposed algorithms are intuitive and seem practical, leveraging smoothing techniques to handle non-smooth loss and using sparsity assumptions.\n2) The theoretical analysis providing error bounds under mild conditions is solid. The bounds quantitatively show potential improvements over not using transfer learning.\n\nWeakness:\n\nThe relaxation of not needing the condition that $sup|_h|≤1 K(u/h)/h < M_k$ seems like incremental relaxation over previous works  Zhang & Zhu (2022)\n\nRefs: \n\n[1] Transfer Learning for High-dimensional Quantile Regression via Convolution Smoothing", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Paper contributions are as follows : \n\n1. Proposes transfer learning algorithms for high-dimensional quantile regression using convolution-type smoothing. Assumes sparsity on the contrast between target and source coefficients and shows the estimation error bounds can improve over using only the target data.  \n\n2. Adapts the transferable source detection algorithm to the quantile regression setting to identify useful sources when the informative sources are unknown. Provides theoretical analysis and simulation studies to validate the performance improvements.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Provide some analysis or empirical results on sensitivity to key assumptions like similarity of source and target distributions. Relaxing assumptions even mildly would broaden applicability.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wlgqMsAVRi", "reviewer": "Reviewer_yBAM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The method of determining the transferable set, detecting the deviation of parameters, and the joint optimization step looks reasonable and novel. The authors provided experimental results only for synthetic data. It would be nice if some experiments for real data have been provided, but it should be fine because the graphs show the algorithms are working as expected.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces an algorithm aimed at enhancing quantile regression through transfer learning. Building upon the optimization method for quantile regression, the authors propose a pipeline of leveraging similar datasets to benefit from utilizing extensive data. Dataset similarity is gauged by a straightforward criterion, employing the q-norm of the deviation between optimal parameters with q = 0 or 1.\n\nThe proposed pipeline includes the transfering, debiasing, and determining the transferable source. While the algorithm itself proposes a standard modules applicable to various algorithms beyond quantile regression, the accompanying theories are tailored specifically to quantile regression.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I suggest minor changes:\n\n1. Use a consistent inner product notation: use either bra-ket or transpose.\n\n2. Include reference for negative transfer on the 7th line in the Introduction.\n\n3. In the last equation of the Introduction (in page 3), $\\Sigma$ should be $A$ for consistency of the notations.", "broader_impact_concerns": "Nothing to note.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CA2eEJbAXy", "reviewer": "Reviewer_JxJL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The problem setting of this paper is clearly described, and is a natural area of interest for machine learning researchers. The related literature is also described quite meticulously, and the place of the present paper within this context is for the most part clear. The basic formulation is also easy to understand, and the basic algorithmic strategies undertaken are intuitive.\n\nOn the other hand, I found certain parts of the overall logic underlying this paper quite puzzling, and while at a macroscopic level the paper is well-organized and clear, a more microscopic look at it yields all sorts of notational issues; the formal material is really quite sloppy in my opinion. Since the paper basically just re-appropriates existing algorithms and analytical machinery to the quantile regression setting, a certain degree of effort needs to be expended to ensure the reader understands what the results in this paper teach us that was not already known before.\n\nI will try my best to elaborate on the points which are lacking in a constructive but concise fashion below, using some concrete examples from the text.\n\n- One major point I tripped up on was all the talk related to *convexity*. The authors say that using the traditional \"kernel smoothing method\" (p.3), the loss is \"still non-convex,\" and use this to motivate their \"convolution-type\" smoother, which they say is \"not only convex but also differentiable.\" Maybe I'm missing something basic here, but the pinball loss $\\\\rho\\_{\\\\tau}(\\\\cdot)$ is convex, correct? Is it *strong* convexity (i.e., a quadratic growth) the authors want? In any case, the logic here seems quite faulty. The loss is pretty nice in that it is Lipschitz, though. If *smoothness* (i.e., a Lipschitz gradient) is the critical point, then yes, the pinball loss here is not smooth, but no gradient-based optimizers are actually studied here, so how this technical backdrop actually links to the methods studied here is not clear to me.\n\n- If the authors want to focus on convexity, they should be more explicit about what properties their proposed objective has, and when. Is the \"empirical smoothed loss function\" defined at the bottom of page 3 convex for any function $K\\_{h}(\\\\cdot)$ that is non-negative, integrable, and symmetric about zero?\n\n- Several quantities appear before they are defined. Here are a few examples.\n  - The \"contrast\" $\\\\boldsymbol{\\\\delta}^{\\\\ast}$ (p.1).\n  - Matrix $\\\\Sigma$ used in defining the ball and cone on page 2 (really, these aren't needed until much later...).\n\n- The same notation is used for different objects in various locations. This should really be avoided. Here are some examples.\n  - $K$ is both the number of source data sets (p.4) and the kernel function.\n  - The notation $\\\\hat{Q}\\_{h}(\\\\cdot)$ is used with different meanings (e.g., p.3 and p.4). This is a critical quantity; such ambiguity severely reduces clarity and is really just sloppy exposition. It appears again on p.6 with a different meaning; the one on p.6 really should be $\\\\hat{Q}\\_{h}^{(k)}(\\\\cdot)$, I think this is obvious.\n  - In Assumption 3.2, $u$ is used as a label on the upper bound $\\\\kappa\\_{u}$ and also the variable being maxed over on the left-hand side of the inequality; more semantic inconsistency.\n\n- I find it rather hard to parse the main theorems, as well as the related propositions. Here are a few things I tripped up on.\n  - In Prop 3.1 and 3.2, are the quantities $r$, $l$, $\\\\boldsymbol{\\\\beta}\\_{1}$, and $\\\\boldsymbol{\\\\beta}\\_{2}$ arbitrary, up to the inequalities holding? Nothing is said about them. Even more frustrating is that real-valued $l$ appears alongside $f\\_{l}$, where the subscript $l$ is just a label representing \"lower\" I assume (from Assumption 3.1). Again, same symbol, totally different meanings. Really must be avoided.\n  - What is $\\\\alpha\\_{1}$ in Prop 3.2?\n  - Most critically, what is $\\\\hat{\\\\boldsymbol{\\\\beta}}$ in Theorems 3.1 and 3.2? From the context, one would like to assume this corresponds precisely to the outputs of Algorithms 3 and 4 respectively; is this so? Even if this is the case, it feels like there is a conceptual gap in that the Theorem 3.2 benefits not just from the stronger \"closeness\" assumption used in defining $\\\\mathcal{A}\\_{m}^{\\\\prime}$, but that it is also an \"oracle\" (i.e., this index of good sources is known).\n  \n- Early in the paper, the authors compare their work with existing papers at a detailed technical level, but I think this could use some additional exposition. Here are two points:\n  - First, in comparing the $s^{2}$ versus $s^{3}$ factors (paragraph 2, page 2), the authors say that \"this is due to the bandwidth involved in the smoothing method.\" What does this mean? The main results in this paper assume the bandwidth is optimally set, correct? Naively, I think some readers will assume that the difference is due to the fact that bias is introduced by using the smoothed objective instead of the traditional quadratic regression objective; is this not the case? One expects that there is a tradeoff - a benefit due to smoothness, and a takeway due to bias - and that these are balanced in the analysis, but this is unclear at present.\n  - Also on page 2, the authors say that compared with Zhang and Zhu 2022, they do not require a particular \"restrictive condition\" on the kernel function; why is this? What differs here? It isn't just due to the stronger $\\\\ell\\_{0}$-norm contrast assumptions, I assume?\n\nMinor issues:\n- Quotation marks are vertical, but really should be slanted.\n- \"the logistic of the algorithm...\" (p.4), maybe should be \"logic\"?\n- Asterisk is bold in second-last paragraph of page 8.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The learning problem of interest here is quantile regression (high-dimensional, in a sparsity scenario), and the main research question of interest is pinning down sufficient conditions for \"domain transfer\" to be beneficial, i.e., to provide stronger guarantees (error bounds) than are possible with just the \"target\" data alone.\n\nThe main substantive content of this paper is as follows:\n\n- Concrete algorithms for transfer learning in the context of (sparse) quantile regression. The main procedures are Algorithms 3 and 4. The key underlying difference is in the assumed \"closeness\" between the sources and the target; the latter assumes an $\\\\ell\\_{0}$-norm bound rather than the more traditional $\\\\ell\\_{1}$-norm bound. The procedures themselves are quite clear and based on similar strategies in the literature, essentially finding good solutions on the source domains, and then since domain \"closeness\" is characterized here by the difference in underlying weight vectors, this difference is then estimated (using target data for comparison), and finally used to correct the learned source weights before integrating into the final target estimator.\n\n- High-probability (target domain) error bounds for estimators based on the aforementioned algorithms (I'm not certain about this; see my later comments).\n\n- Some basic simulated numerical tests, comparing their proposed procedure with some natural benchmarks, in particular looking at the weak/strong scenarios of interest ($\\\\ell\\_{1}$ versus $\\\\ell\\_{0}$ bounds on the index of \"good\" sources, denoted $\\\\mathcal{A}\\_{m}$).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "Please see the comments above. Taking everything together, I feel like the authors have made an honest effort, but the presentation is way below the standard that one would expect. TMLR does not require the results to be \"significant,\" but at this point it is not really even clear how the material here differs from Zhang and Zhu 2022, aside from treating the \"lucky\" situation in which we have a $\\\\ell_{0}$ bound on the contrast (plus oracle information?). My personal opinion is that clearing all this up is quite close to what I would call a \"major revision.\"", "broader_impact_concerns": "Not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have put an honest effort into revising the paper to address the concerns raised by reviewers, most of which are related to the presentation. The presentation issues have made it difficult to verify all of the claims in the paper, although no major issues related to correctness of the results have been raised by reviewers.

After revision to improve the presentation, all three reviewers are in favour of accepting the paper.
Decision: Accept (poster)