Active representation learning for general task space with applications in robotics
Authors: Yifang Chen
Yingbing Huang
Simon Shaolei Du
Kevin Jamieson
Guanya Shi
Venue: NeurIPS 2023 poster (2023)
Keywords: active learning
representation learning
robotics
theory

Abstract: Representation learning based on multi-task pretraining has become a powerful approach in many domains. In particular, task-aware representation learning aims to learn an optimal representation for a specific target task by sampling data from a set of source tasks, while task-agnostic representation learning seeks to learn a universal representation for a class of tasks.  In this paper, we propose a general and versatile algorithmic and theoretic framework for \emph{active representation learning}, where the learner optimally chooses which source tasks to sample from. This framework, along with a tractable meta algorithm, allows most arbitrary target and source task spaces (from discrete to continuous), covers both task-aware and task-agnostic settings, and is compatible with deep representation learning practices. 
We provide several instantiations under this framework, from bilinear and feature-based nonlinear to general nonlinear cases. In the bilinear case, by leveraging the non-uniform spectrum of the task representation and the calibrated source-target relevance, we prove that the sample complexity to achieve $\varepsilon$-excess risk on target scales with $(k^*)^2 ||v^*||_2^2 \varepsilon^{-2}$
 where $k^*$ is the effective dimension of the target and $||v^*||_2^2 \in (0,1]$ represents the connection between source and target space. Compared to the passive one, this can save up to $\frac{1}{d_W}$ of sample complexity, where $d_W$ is the task space dimension. 
Finally, we demonstrate different instantiations of our meta algorithm in synthetic datasets and robotics problems, from pendulum simulations to real-world drone flight datasets. On average, our algorithms outperform baselines by 20%-70%.

Active Representation Learning for General Task Space with Applications in Robotics
Yifang Chen', Yingbing Huang?, Simon S. Dul Kevin Jamieson 1+ Guanya Shi
Paul G. Allen School Computer Science & Engineering University of Washington, Seattle,W. WA
(yifange, sadu, jamieson, guanyas) washington edu
2 University of Illinois Urbana-Champaige Champaign, IL {yh21)ei1linois edu
Robotics Institute, Carnegie Mellon University, Pittsburgh, PA gguanyas @ @undrew.emu.edu Equal advising
Abstract
Representation learning based on multi-task pretraining has become powerful approach in many domains. In particular, task-aware representation learning aims learn an optimal representation for specific target task by sampling data from a set of source tasks, while task- agnostic representation learning seeks to learn a universal representation for a class of tasks. In this paper, we propose a general and versatile algorithmic and theoretic framework for active representation learning, where the learner optimally chooses which source tasks to sample from. This framework, along with a tractable meta algorithm, allows most arbitrary target and source task spaces (from discrete to continuous), covers both task-aware and task-agnostic settings, and is compatible with deep representation learning practices. We provide several instantiations under this framework, from bilinear and naature-based nonlinear to general nonlinear cases. In the bilinear case, by leveraging the non-uni non-uniform spectrum of the task representation and the calibrated source-target relevance, prove that the sample complexity to achieve e-excess risk on target scales with (k* )+ [|3s-2 where A:* is the effective dimension of the target and ||00| (0, 1] represents the connection between source and target space. Compared to the passive one, this can save up to I of sample complexity, where dw is the task space dimension. Finally, we demonstrate different instantiations of meta algorithm in synthetic datasets and roboties problems, from pendulum simulations to real-world drone flight datasets. On average, our algorithms outperform baselines by 20% 70%.
dw
Introduction
Recently, few-shot machine learning has enjoyed significant attention and has become increasingly critical due to its ability to derive meaningful insights for target tasks that have minimal data, scenario commonly encountered in real-world applications. This issue especially prevalent in
robotics where data collection and training data is prohibitive to collect or even ren-duurodec (e.g.. drone flying with complex aerodynamics [T] legged robots on challenging terrains (2). One
'Code pulhtttps:/ggithuut http:///gg.tttaoo///////aa/aaaaa
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

promising approach to leveraging the copious amount of data from a variety of other sources is multi-task learning, which is based on a key observation that different tasks may share common low-dimensional representation. This process starts by pretraining a representation on source tasks and then fine-tuning the learned representation using a limited amount of target data (
In conventional supervised learning tasks, accessing a large amount of source data for multi-task representation learning may be easy, but processing and training on all that data can be costly. In real-world physical systems like robotics, this challenge is further amplified by two factors: (1) switching between different tasks or environments often significantly more expensive (e.g., reset giant wind tunnels for drones (7); (2) there are infinitely many environments to select from (i.e., environmental conditions are continuous physical parameters like wind speed). Therefore, it is crucial to minimize not only the number of samples, but the number of sampled source tasks, while still achieving the desired performance on the target task. Intuitively, not all source tasks are equally informative for learning universally good representation or target-specific representation. This is because source tasks can have a large degree of redundancy or be scarce in other parts of the task space. In line with this observation, Chen et al. S provided the first provable active representation learning method that improves training efficiency and reduces the cost of processing source data by prioritizing certain tasks during training with theoretical guarantees. On the other hand, many existing works [9]13] prove that it is statistically possible to learn a universally good representation by randomly sampling source tasks (i.e., the passive learning setting).
The previous theoretical work of g on active multi-task representation learning has three main limitations. First, it only focuses on finite number of discrete tasks, treating each source inddpne- dently, and therefore fails to leverage the connection between each task. This could be sub-optimal in many real-world systems like robotics for two reasons: (1) there are often infinitely many sources to sample from (e.g., wind speed for drones); (2) task spaces are often highly correlated (e.g., perturbing the wind speed will not drastically change the aerodynamics). In our paper, by considering a more general setting where tasks are parameterized
a vector space W,, we can more effectively leverage similarities between tasks compared to treating them as simply discrete and different. Secondly, the previous work only considers a single target, while we propose an algorithm that works for an arbitrary target space and distribution. This is particularly useful when the testing scenario is time-variant. Thirdly, we also consider the task-agnostic setting by selecting (kk) representative tasks among the dw high dimension task space, where k < dw is the dimension of the shared representation. Although this result does not improve the total source sample complexity compared to the passive learning result in the bilinear setting [12, it reduces the number of tasks used in the training and therefore implicitly facilitates the training process.
In addition to those theoretical contributions, we extend our proposed algorithmic framework beyond a pure bilinear representation function, including the known nonlinear feature operator with unknown linear representation (c.g., random features with unknown coefficients), and the totally unknown nonlinear representation (c.g., deep neural network representation). While some prior works have considered nonlincar representations 9 TO 14 [3 in passive learning. the studies in active learning are relatively limited [8]. All of these works only consider non-linearity regarding the input, rather non-linearity and show its
than the task parameter. In this paper, we model mask-parameter-wee effectiveness in experiments. Note that it particularly matters for task selections because the mapping from the representation space task parameters to is no longer linear.
See more related works and how our problem scope is different from theirs in Appendix A.
1.1 Summery of contributions
We propose the first generic active representation learning framework that admits any arbitrary source and target task space. This result greatly generalizes previous works where tasks lie in the discrete space and only single target is allowed. To show its flexibility, we also provide discussions on how our framework can accommodate various supervised training oracles and optimal design oracles. (S)ction
We provide theoretical guarantees under a benign setting, where inputs are i.i.d. and unit ball is contained in the overall task space, as a compliment to the previous work where tasks lie on the vertices of the whole space. In the target-aware setting, identify an e-good model our method requires a sample complexity of O(dx (k*) 2 1 min(k" "+2", x²))=² where ke is the effective dimension of the target, is the conditional number of representation matrix, and
2

---

||0112 E (0, 1] represents the connection between source and target space that will be specified in the main paper. Compared passive learning, our result saves up to factor of k² in the sample complexity when targets are uniformly spread over the k-dim space and up to dw are highly concentrated. Our results further indicate the necessity of considering the continuous space by showing that directly applying the previous algorithm onto some discretized sources in the continuous space (e.g., orthonormal basis) can lead to worse result. Finally, ignoring the tasks
dw _ when targets used in the warm-up phases, in which only a few samples are required, both the target-aware and the target-agnostic cases can save up to O + k) number of tasks compared the passive one which usually requires dw number of tasks. (Section !
We provide comprehensive experimental results under different instantiations beyond the benign theoretical setting, studying synthetic and real-world scenarios: 1) For the synthetic data setting in a continuous space, we provide results for pure linear, known nonlinear feature operator dx and unknown nonlinear representation Øx Our target-aware active learning (AL) approach shows up to a significant budget saving (up to 68%) compared to the passive approach and the target-agnostic AL approach also shows an advantage in the first two cases. 2) In a pendulum simulation with continuous task space, we provide the results for known nonlinear feature operator ạ and t'w and show that our target-aware AL approach has up to 20% loss reduction compared the passive one, which also translates to better nonlinear control performance. 3) Finally, in the real-world drone dataset with discrete task space, we provide results for unknown linear and nonlinear representation ox and show that our target-aware AL approach converges much faster than the passive one. (Section 5)
Preliminary
Multi-task (or multi-environments). Each task or environment is parameterized by a known vector E RRdw. We denote the source and target task parameter space as Woouree C Rd Weaget C Rd These spaces need not be the same (e.g., they could be different sub-spaces). In the discrete case, set as one-hot encoded vector and therefore have in total dw number of candidate tasks while in the continuous space, there exist infinitely many tasks. For convenience, we also use w as the subscript to index certain tasks. In addition, denote the task distribution for the sources and targets.
Vsuuce E A(Wsunce)) Marree A A(Wargge)) to
Data generation. Let X E Rdx be the input space. We first assume there exists some known feature/augmentationn operator txx - Rdex 2dw sow W - Rdow 2dw that can be some non- linear operator that lifts w, x to some higher dimensional space (e.g., random Fourier features (15)). Notice that the existence of non-identical & indicates the features not pairwise independent and the design space of Waouree is not benign (e.g., non-convex), which adds extra difficulty to this problem.
Then we assume there exists some unknown underlying representation function øx (x() - R. which maps the augmented input space (x)) to a shared representation space R E IR* where k < d I dew and its task counterparts ow +WV) R. which maps parameterized task space to the feature space. Here the representation functions are restricted to be in some function classes $ e.g., linear functions, deep neural networks, etc.
In this paper, we further assume that Φw is a linear function Bw E Rxddd To be more specific, for any fixed task w, assume each sample (Ir, y) Vie satisfies
y = dxx (vx (x)) T Bw (w) E {~N(0,0")
For convenience, denote Z,, the collection of few sampled data (arhe ) (x v0-) plu- We note that when vxx, v/w is identity and éx is linear, this is reduced to standard linear setting in many previous papers 9 12 8.
The task diversity assumption. There exists some distribution p E A(Wsource) that Amin (Bweww ww (w) BTT) > O, which suggests the source tasks are diverse enough to learn the representation.
Data collection protocol. We assume there exists some i.i.d. data sampling oracle given the environment and the budget. To learn a proper representation, we are allowed access to an unlimited Touure number of data from source tasks during the learning process by using such an oracle. Then at
3

---

the end of the algorithm, we are given few-shot of mix target data Zuaget = {Zul wnrMargs which is used for fine- the-tuning based on learned representation Φx. Denote Teurget as the number of data points in Zurget.
When the target task is not a singleton, we additionally assume few-shot of known environment target data Zurget := {Zw: w WoeWoyss where |Wurgell = dim (Wunget) and Warges {arg maxweWim. EWagss Amin (Win Again denote Rarget as the number of data points in Zrarget. we have tearget R3 Teurggt
Data collection protocol for target-aware setting.
2/3
<
Tisource.
Remark 2.1. Here Warreer represents vectors that can cover every directions of Warget space. This extra Zurger requirement comes from the non- non-linearity of le loss and the need to learn the relationship between sources and targets. We want to emphasize that such an assumption implicitly exists in previous active representation learning [ since Zurret = Zurget in their single target setting. Nevertheless, in passive learning setting, only mixed Zuage is required since source selection process involves. Whether such a requirement is necessary for awgee active learning remains an open problem.
Other notations.
Let e; to one-hot vector with at i-th coordinates and e; 2
2.1 Goals
Expected excess risk. For any target task space Warget and its distribution Marget over the space, as well as a few-shot examples as stated in section 2 with our estimated Φx
our goal is to minimize the expected excess risk
ER (2xxx Prarget) Ervg
B(z,g) B
11xx (xx( Davg - y112
where Mavg arg min E(z.g)e
11xx (x)) 2lا2, which average model estimation that captures the data behavior under the expected target distribution. Note that the Wusgett Mearget are given in advance in the target-aware setting.
The number of tasks. Another side goal is to save the number of long-term tasks we are going to sample during the learning process. Since uniform exploration over di gsouree dimension is unavoidable during the warm-up stage, we define long-term task number as
Kuo w E
Wssurce
nhw 2 ii( 9)
where a is some arbitrary exponent and E is the target accuracy and nw is number of samples sampled from task as defined above.
3 A general framework
Our algorithm iteratively estimates the shared representation èx Bw and the next target relevant source tasks which the learner should sample from by solving several optimal design oracles
(f
g(f
min (EW(W, (ouree))
Amax
This exploration and exploitation (target-aware exploration here) trade-off is inspired by the classical e-greedy strategy, but the key difficulty in our work combine that with multi-task representation learning and different optimal design problems. The algorithm can be generally divided into three parts, and some parts can be skipped depending on the structure and the goal of the problem.
Coarse exploration: The learner uniformly explores all the directions of the Wsouree (denoted by distribution go) in order find initial k-dimension subspace V that well spans the representation space (i.e., Bw BT, < Bw B < cBw Bw for some arbitrary constant e S dew, ). To give an intuitive example, suppose Bw E 2x2 has the first half column equals é1 and the second hard equals e2. Then instead of uniformly choosing {es (ea)velagwy task, we only
need explore over two tasks [[[]
1. 0, 12)
0. O
1,

---

Algorithm Active multi-task representation learning (general templates)
1: Inputs: Candidate source set Wanurce- Classes of candidate representation function 4x @w and the known feature operator e'x, sow.
2: [TTarge-aware only] Inputs: Target set Wurget and distribution Marget- Few-shot sample Zurget defined in the preliminary.
3: Stage I: Coarse exploration. (Warm-up stage)
4: Set initial sampling distribution go =((w,, Idow where is defined in Eqn. ]
5: Set 7Ao poly(d k) + poly(daw: k).
Collect nogo(w) data for each task de- Offfine ol [Z h (l) txx) and Bw <-
noted as (loowloo())) and update ox f
O (1 go (o()) Φx)
6: for =
7: Stage 2: Fine target-agnostic exploration (Directly choose of 2u when k: == ((dw))
8: Compute the exploration sampling distribution of = s Bw o 1bw. Ts)
9: n² = poly(dex: k)e," Collect n² q' (w) data for each task denoted as [Zw 1owhoope holg))))) and update Φx + O offline (1 (Z h (10) x/x) and Bw + O (1 (wl)(())) Zuagar ox)
10:
('Target-aware only] Stage 3: Fine target-aware exploration
11:
Compute the exploitation sampling distribution oe = g Bw o thw, Eregu) where Eregu is the regularized version of Bw (Eunreyy wows BW after clipping out insignificant eigenvalues. Set n² ~ poly(dwxx k),,² Collect nigz (w) data for each task denoted as {Z Jwl (w)) and update éx + OXX offine a 21 w/ol (w) and (5))))) 1xx).
13: end for Return Φx
We want to highlight that the sample complexity of this warm-up stage only scales with døx k and the spectrum-related parameters of Bw (i.e., K ((
)), not the desired accuracy E. Fine target-agnostic exploration: The learner iteratively updates the estimation of V and uniformly explore for Ö(c, times this instead of dew subspace, denoted by distribution Q1-
(Note this E comes from the exploration part in e-greedy, which is (n/) 3) Such reduction not only saves the cost of maintaining large amount of physical environment real-world experiments but also simplifies the non-convex multi-task optimization problem. Of course, when A: = (dd ) we can always uniformly explore the whole (dea space denoted in the algorithm. Note that theoretically, Q1 only needs to be computed once as shown in In practice, to further improve the accuracy while saving the task number, the @i can be updated only when a significant change from the previous one happens, which is adopted in our experiments as shown in appendix E.I
Fine target-aware exploration. In the task-awareness setting, the learner estimates the most- target-related sources parameterized by {w} based on the current representation estimation and allocates more budget on those, denoted by distribution Q2. By definition, 92 should be more sparse than QI and thus allowing the final sample complexity only scales with ke which measures the effective dimension the source space that is target-relevant.
Computational oracle for optimal design problem. Depending. (1ww (w) w the learner should choose proper offline optimal design algorithms to solve A). Here propose several common choices. 1). When Wsoure contains ball, can approx- imate the solution via an eigendecompositiio--basee closed-form solution with an efficient projection as detailed in Section 4 2) When Wanuce is some other convex geometry, we can approximate the result via the Frank- Wolfe type algorithms [16], which avoids explicitly looping over the infinite task space. 3) For other even harder geometry, can discretization adaptive sampling-based based approximation [17). In our experiments, we adopt the latter one and found out that its running time cost is almost neglectable in our pendulum simulator experiment in Section where the v'w is polynomial augmentation.
the geometry of
Offline optimization oracle O. Offline Although we are the continuous setting, the sampling distribution go Q1, 92 is sparse. Therefore, algorithm allows any proper passive multi-task
5

---

learning algorithm, either theoretical or heuristic one, to plugin the offtine- Some common choices include gradient-baser joint training approaches[ [8 the general non-convex ERM and other more carefully designed algorithms [22 22 We implement the first one in our experiments (Section 5) to tackle the nonlinear 4 Φx and give more detailed descriptions of the latter two in Section and Appendix B. to tackle the bilinear model.
4 A theoretical analysis under the benign W setting
4.1 Assumptions
Assumption 4.1 (Geometry the task space). We assume the source task space Wssurce is a unit
ball Bow (1) that spon over the first drouce 2 1 dw without loss of generality, while the target task 2
space Warget C Rdw can be any arbitrary Bdw dd (1).
Under this assumption, let Bay denote the first dippece columns of Bw, which stands for the source-related part of Bw. And B
Then we assume the bilinear model where ox Bx E xd and vix, vww = I. Therefore, døx = dx, dew = dw. Moreover the model satisfies the following assumptions
Assumption 4.2 (Benign Bx, Bw). Bx is an orthonormal matrix. Each column of By has
magnitude (1) and m ( (Buuuee) > L. Suppose we know x(B T (Bw ( and =dw.g=1.
g I miin (B BW , T (Bir Trivially, (
Finally, the following assumption is required since we are using a training algorithm in 12 and might be able to relax to sub-gaussian by using other suboptimal oracles. Assumption 4.3 (Isotropic Gaussian Input). For each task w, its input satisfies Fi,00 (0 Ia).
4.2 Algorithm
Here we provide the target-aware theory and postpone the target-agnostic the Appendix. its analysis is covered by the target-aware setting.
since
This target-aware grgorithm@follows the 3-stage which corresponds to sampling distribution qu- Q1: Z2 with explicit solutions. Notice that calculating qi once is enough for theoretical guarantees.
offline 2 and use the simple ERM methods for OW based on the learned B. For the coarse exploration and fine target- agnostic exploration stage, the main purpose is to have a universal good estimation in all directions of Bx. (i.e., upper bound the (B Bx)) Therefore we choose the alternating minimization (MLLAM) proposed [1]. On the other hand, for the fine target-aware exploration, we mainly care about final transfer learning performance on learned representation. Therefore, we use a non-convex ERM from 99. We defer the details and its theoretical guarantees for O into Appendix B.I
We use existing passive multi-task training algorithms as oracles for O-X offline 1 O.X
Note the major disadvantage from p comes from its sample complexity scaling with a number of training source tasks, which will not be problem here since in O-X offline. since only k k* & dw number of tasks are used. The major benefit of using non-convex ERM comes from its generality that works even for the non-linear setting and is not tied with specific algorithm. That say, as long as there exists other theoretical or heuristic oracles O offline, 1 O offline, 2 giving a similar guarantee, stage always works.
4.3 Results
Theorem 4.1 (Informal). By running Algo.
2 in order to let ER(2xx, Viarget) S e² with probability 1 - 8, the number of source samples Thyouree is at most
δ ((kdx + log(1/6)) (1=*)2 minnk" , (2(W)) )} max [W low-order)
Here k* rank rank([Engg (Beng~wans Bw BTy)) represents the effective dimension of larget and
W = arg min l|ul|2 8.1 Baguce w = /.y Aas where U, A 4- Eig (Ewagretang W Bw).
w Wanne

---

Algorithm 2 Target-aware algorithm for benign source space
1: Inputs: Target probability ö, R. Some constant 31, $2, 93. Others same Algo.
2: Set go as (o()))= dw Vt E dw, andqu(w) = 0 otherwise
3: Set no B [b2dxk² + d'v VK log(1/6)) Collect nodo (w) data for each task de-
noted as Jwlg()))
4: Update Bx + offline
ho) and Bw & OW (1Zw w wloo( Bx)
5: Compute @) as 41 (vi) = E. vi E ki and go(w) = 0 otherwise. Here uy is the i-th vector of V,
where U, D. V SVD(BWrrg
6: for = 1,2,,,,....
7: Set ni = B2e, *
(k3 (k3d + Collect niqi (w) data for each task de-
noted {2w) w/1: (w)+f-
8: Update Bx + Oxx Oflline Zw holmn (w))) Biqune + O (Zw hllen (w) Bx)
and B - Ow anfine (Zuggt Bx)
9: Find a set of target-aware tasks parameterized by W3 with each column as
w/
Wi(0) = Projw w
|
dx
where w/ arg min W s.t. w
WJ w ua di AYA 2 8(kdw)
V n1
Burggt BWJ wo Wij wo) )
where U,A +- Eig Ereg Miragge
10: Compute of as (w) ",)( Yu col(W,) and of (w) otherwise
11: Assign n² total sampling budget as # col(W,) max, [1)(())
12: Collect ni (w)) nágg (w) data for each task denoted [Zw) wi)g)(w))(
13:
Update the model, note that both data collected from stage 2 and stage 3 are used.
Bx
O offline 3 Zww 3 wlen (w))EU and ga(w))tt)
14:
end for Return Bx
As long as the number of target samples satisfies
Tharget 2 Φ((k log(1/6))s=
Miarget H (E (1*) Vi (dtoo 3d )))
Comparison with passive learning. By choosing {e= hi (apaa as a fixed source set, we reduce the problem to discrete setting and compare it with the passive learning. In 19) the authors get Nioa as R dw [[mngroune Bw Bw worn Byy
most
-2 We first consider the cases their paper that the target task uniformly spread [ Eano w B = +
2 (Baans)
When the task representation is well-conditioned 22.n =kdxxe-2) while the active one (kdd 2 2-) (See Lemma B.8 details), which suggests as long as dw > 1.2,, our active learning algorithm gain advantage even in a relatively uniform spread data and representation conditions.
(Buourc)) PW
dx. We have a passive one as K
Otherwise, consider the extreme case that o (Btuice)) 1. We have passive one o(d while the active one ((I-dxx-2), Notice here we require dw > 1.3
Both of them indicate the necessity of considering the continuous case with large dw even if everything uniformly spread. On the other hand, whether we can achieve the same result as the passive one when dw < k3 remains to be explored in the future.

---

We then consider the single target wy case.
With well-conditioned Bw, the passive one now has sample complexity O(k2 while the active gives a strictly improvement O x =
With ill-conditioned Bw where miin (Bw)
and max; || 1, that is, only a particular direction in source space contributes to the target. The Passive one now has sample complexity O (kd xkdxdwe- while our active one only has kdxe= which demonstrates the benefits of algorithm in unevenly distributed source space.
Comparison with previous active learning. By using the same discrete reduction and set single target wo. we compare our result with the current state-of-art active representation algorithm in [23] They achieves (kdx [ f)), where arg min, |||||| s.t Bw By On the other hand, our active one gives (kdx [|w" l12e
where w" = arg min, ||m|| s.t By = Wn, strictly better than the discrete one. This again indicates the separation between continuous and discrete cases where in fixed discrete sets, the L₁ norm regularization is strictly better than Lg.
wn. which is
Furthermore, when fixed discrete set is given, which is exactly the setting in 23 Their algorithm can be seen as a computationally efficient reduction under ours.
B.5
Save task number. When ignoring the short-term initial warm-up stage, we only require main- taining Ok + log(Nonak)) number of source tasks, where the first term comes from Φi in the larget-agnostic stage and the second term comes from d2 in the target-aware stage.
5
Experiment
In this section, we provide experimental results under different instantiations of the Algorithm and all of them show the effectiveness of our strategy both in target-aware and target-agnostic settings.
5.1 Settings
Datasets and problem definition. Our results cover the different combinations of tix, Φx, t'w as shown in Table T Here we provide brief introduction for the three datasets and postpone the details into Appendix
identity v'w
nonlinear vow
identity vxx and linear σx
synthetic, drone synthetic
NA
nonlinear d/x and linear éx
pendulum simulator
identity dhx and nonlinear 6x
synthetic, drone
NA
Table 1: Summary of different instantiations
Synthetic data. We generate data that strictly adhere to our data-generating assumptions and use the same architecture for learning and predicting. When Φx is nonlinear, we use a neural network Φx generate data and use slightly larger neural net for learning. The goal for synthetic data is to better illustrate our algorithm as well as serve as the first step to extend our algorithm on various existing datasets.
Pendulum simulator. To demonstrate our algorithm in the continuous space. we adopt the multi- environment pendulum model in (24) and the goal to learn a w-dependent residual dynamics model [ar, w) R where is the pendulum state and w E IR including external wind, gravity and damping coefficients. fzz w) is highly nonlinear with respective to & and w. Therefore we use known non-linear feature operators vxx, vww In other words, this setting be regarded as a misspecified linear model. is also worth noting that due to the non-invertibility of thw, the explicit selection of source via closed form is challenging. Instead, we resort an adaptive sampling-based method discussed Section 3] Specifically, we uniformly sample w from the source space, select the best w', and then uniformly sample around this w at finer grain. Our findings indicate that about 5 iterations are sufficient to approximate the most relevant source.
Real-world drone flight dataset [7]. The Neural-Fly dataset [ includes real flight trajectories using two different drones in various wind conditions. The objective is to learn the residual
Github Link: http://gthhiocommmooooo/ggg////a Robotics

---

aerodynamics model fzz w) E R where z E R!' is the drone state (including velocity, attitude, and motor speed) and the environment condition (including drone types and wind conditions). We collect 6 different w and treat each dimension of f'(rr, w) separate task. Therefore w is reformulated as a one-hot encoded vector TR.1.
For each dataset/problem, we can choose different targets. For simplicity, in the following subsection, we present results for one target task for each problem with 10 random seeds regarding random data generation and training, and put more results Appendix [E] In all the experiments, we use gradient-descent joint training oracle, which is a standard approach in representation learning.
5.2 Results
Those results encapsulate the effectiveness of active learning in terms of budget utilization and test loss reduction. In the drone dataset, we further demonstrate its ability in identifying relevant source tasks (see Figure 2). We note that in two robotics problems (pendulum simulation and real-world drone dataset), the active learning objective is to leamn better dynamics model. However, in the pendulum simulation, we deploy a model-based nonlinear controller which translates better dynamics modeling to enhanced control performance (see Figure and Appendix E.2
Target-aware AL
Target-agnostic AL
identity t/x and linear dx
38.7% 7%
51.6%
nonlinear dix and linear Øx
38.7%
45.2%
identity thx and non-linear Øx
32.0%
68.0%
Table 2: Results synthetic data. Using the test loss of the final output model from passive learning as a baseline, we show the ratio between the budget required by aagel-awarr/urget-gnntttt active learning to achieve similar loss and the budget required by passive learning.
0.08
0.12
AL.aaret aware passive
0.06
centrol ener
0.10
0.04
0.02
dratogy
passiva
0.04
0.00
number
20000
source
40000 60000
simulstion timett)
The test loss of the estimated model f. The passive strategy suffers from negative transfer while the active strategy steadily decreases. Right: The control error using final output f. Here we use a model- nonlinear policy π (2x, /). The model learned from active strategy leads to better control performance.
Figure 1: Results on pendulum simulator for a specific target. Left:
8 0.79
1.4
Strategy
AL target aware passive
0.20
0.15
8.6
0.00
drans hype
drive typs 10.2
drone sype, 50
drane, typet A O.
arose type 6.30
drane, Sipe 2
truse hrpe, A.
drume Type 4.20 trone type
drone typu,
number saurce samples
Figure 2: Results on the real drone dataset 7 with target drone strone_tpe__.. type_A__pa A_30__ Source data includes two drone types and B, six wind speeds from 0 50, and three directions x- x-y-z. We present results for linear Φx here and postpone the non-linear éx case in Appendix E.3 Left: The test loss of the estimated bilinear model f. The passive strategy converges slower than the active strategy. Right: Top 10 the most similar source tasks. Given the target environment, the algorithm successfully finds the other drone_ _tpe__ environments as relevant sources. See more explanations in Appendix E.3

---

References
[1] Guanya Shi, Xichen Shi, Michael O' O'Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural lander: Stable drone landing control using learned dynamics. In 2019 International Conference on Robotics and Automation (ICRA), pages 9784-9790. IEEE, 2019.
[2] Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. Learning quadrupedal locomotion challenging terrain. Science robotics, 5(47))eabc5986, 2020.
[3] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. arXiv preprint wxXi::2103.00020 2021.
[4] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 140.55000..411,, 2020.
[5] Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022.
[6] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35:
23716-23736, 2022.
[7] Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural-ft fly enables rapid learning for agile flight in strong winds. Science Robotics, 7(66))eabm6597, 2022.
[8] Yifang Chen, Kevin Jamieson, and Simon Du Active multi-task representation learning. In International Conference on Machine Learning, pages 3271-3298. PMLR, 2022.
[9] Simon S. Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei. Few-shot learning via learning the representation, provably, 2021.
[10] Nilesh Tripuraneni, Michael I. Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity, 2020.
[11] Nilesh Tripuraneni, Chi Jin, and Michael Jordan. Provable meta-learning of linear repre- sentations. In International Conference Machine Learning, pages 10434-10443. PMLR, 2021.
[12] Kiran Koshy Thekumparampil, Prateek Jain, Praneeth Netrapalli, and Sewoong Oh. Sample efficient linear meta-learning by alternating minimization. arXiv preprint arX.v:2105.08306, 2021.
[13] Ziping Xu and Ambuj Tewari. Representation learning beyond linear prediction functions. Advances in Neural Information Processing Systems, 34:4792-4800 2021.
[14] Liam Collins, Aryan Mokhtari, Sewoong Oh, and Sanjay Shakkottai. Maml and anil provably learn representations. In International Conference on Machine Learning, pages 4238-4316 PMLR, 2022.
[15] Ali Rahimi and Benjamin Recht. Uniform approximation of functions with random bases. In 2008 46th annual allerton conference on communication, control, and computing, pages 555-561, IEEE, 2008
[16] Michael Todd. Minimum-volume ellipsoids: Theory and algorithms. SIAM, 2016.
[17] Youhei Akimoto, Yuichi Nagata, Isao Ono, and Shigenobu Kobayashi. Theoretical foundation for ema-es from information geometry perspective. Algorithmica, 64:698-716, 2012.
10

---

[18] Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint 1909:9999000000 2019.
[19] Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint 1eXiv:1803.02999 2(3):4, 2018.
[20] Antreas Antoniou, Harrison Edwards, and Amos Storkey. How to train your maml. arXiv preprint arXiv:1810.09500., 2018.
[21] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44 (9):5149-5169, 2021.
[22] Shuxiao Chen, Koby Crammer, Hangfeng He, Dan Roth, and Weijie J. Su. Weighted training for cross-task learning, 2021.
[23] Yiping Wang, Yifang Chen, Kevin Jamieson, and Simon Shaolei Du. Improved active multi-tast representation learning via lasso. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardd, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 35548 35578. PMLR, 23-29 Jul 2023. URL https: http::::/ppoceeedngne mml.. press/ #202/wang223 html
[24] Guanya Shi, Kamyar Azizzadenesheli, Michael O'Connell, Soon-Jo Chung, and Yisong Yue. Meta-adaptive nonlinear control: Theory and algorithms. Advances in Neural Information Processing Systems, 34: 34:10013-10025, 2021.
[25] Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In International Conference on Machine Learning, pages 9120-9132. PMLR, 2020.
[26] Rahul Ramesh and Pratik Chaudhari. Model zoo: A growing" brain" that learns continually. arXiv preprint 2xxiv:2106.0000 106.03027,72... 2021.
[27] Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Processing Systems, 34:27503-27516, 2021.
[28] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. International conference on machine learning, pages 1126-1133 PMLR, 2017.
[29] Hae Beom Lee, Hayeon Lee, Donghyun Na, Saehoon Kim, Minseop Park, Eunho Yang, and Sung Ju Hwang. Learning to balance: Bayesian meta-learning for imbalanced and out-of- distribution tasks. arXiv preprint arXiv: 1ww.:/995......... 12917,20111
[30] Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic meta- learning. Advances in Neural Information Processing Systems, 33:188600 1860-1887111 2020.
[31] Xingcheng Yao, Yanan Zheng, Xiaocong Yang, and Zhilin Yang. Nlp from scratch without large-scale pretraining: simple and efficient framework. In International Conference on Machine Learning, pages
PMLR, 2022.
[32] Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling. arXiv preprint arXiv:2302.0311 139,, 2023.
[33] Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304. 14108, 2023.


Reviews:
[{"review_id": "UbBPjvuN8h", "reviewer": "Reviewer_LcWv", "summary": "Which source task to sample from is an important issue in multi-task learning. This paper proposes a general and versatile algorithm and framework for active representation learning. The proposed algorithm can be adapted to arbitrary target and source task space, and can cover task-aware and task-agnostic settings. The experiments demonstrate the efficacy of the proposed algorithm on various datasets.", "questions": "None.", "limitations": "No limitations are addressed.", "rating": 1, "confidence": 1, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- Interesting and important topic.", "weaknesses": "- Incomplete contents.\n\n- Poor writing and structure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "KxLX5GvmcG", "reviewer": "Reviewer_DHY3", "summary": "The paper introduces and studies a theoretic and algorithmic framework for actively and optimally selecting the source tasks in multi-task learning, accommodating task-aware and task-agnostic settings. \nThe authors present cohesive theoretical guarantees, particularly demonstrated through the pure bilinear setting with the task or sample complexity being improved. They further empirically validate their meta-algorithm with task-parameter-wise non-linearity settings, particularly with emphasis on applications in robotics, significantly outperforming baselines.", "questions": "See some questions in the *weakness* sections.\n\n* What is the trade-off and limitations of the proposed algorithm compared to previous methods?\n* When it comes to a single target task -- does the proposed method outperform the baseline?\n* Given that the model applies to multiple target tasks, how does it compare to previous works when several target tasks are negatively correlated or have detrimental effects on each other? \n", "limitations": "The paper does not explicitly describe the limitations of the approach.\nPlease provide the limitations and the trade-off of the method.", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The problem of active representation learning is interesting and important.\nThe proposed framework is novel and effective, demonstrated with both theoretical and empirical evidence.\nThe theoretical and experimental results are cohesive. ", "weaknesses": "- While the method and algorithms seem sound, their explanation could be better with more interpretation and transitions.\n- The experiment section (in the main manuscript) is not well-presented, e.g., lack of baselines/setting description as well as the interpretation and justifications of results.\n- To my best assessment, the experiments are not systematically and thoroughly conducted.\n**Minor:**\n- The paper misses the conclusion, thorough literature review, and limitation parts", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1: What is the trade-off and limitations of the proposed algorithm compared to previous methods?\n\nPlease refer to our response to all reviewers. In particular, in the “limitation” section we discuss its limitations in both assumptions and the solutions.\n\n### Q2: When it comes to a single target task -- does the proposed method outperform the baseline?\n\nIn most continuous infinite parameter space: Our method is strictly better than reducing to discrete setting + previous methods [1][2], as discussed in line 163-171 and is also strictly better than the passive method (random sampling) as discussed in line 156-162. We shouw it by proving rigorous theoretical analysis.\n\nIn the discrete parameter space, previous works can be seen as a computationally efficient solution under our general framework. That is, the solution q from  [2]\n\n$v = argmin_v ||v||_1 \\quad s.t. \\hat{B}_W^\\text{source} v =\\hat{B}_W^\\text{target} w_0$\n\n$q = v/||v||_1$\n\ncan be seen as an efficient approximate solution of our proposed in line 12,\n\n$\nargmin_q  (\\hat{B}_W^\\text{target} w_0)^\\top (\\hat{B}_W^\\text{source} Q (\\hat{B}_W^\\text{source})^\\top)^{-1}  (\\hat{B}_W^\\text{target} w_0)\n$,\nwhere $Q = diag(q)$ and $q$ is the distribution over all source tasks.\n\nThat is to say, we expect that, under this specific setting, our algorithm should be reduced to previous work. In fact, this is exactly what we do in our experiment: drone, where tasks are discrete, as stated line 757 -762.\n\n\n### Q3: Given that the model applies to multiple target tasks, how does it compare to previous works when several target tasks are negatively correlated or have detrimental effects on each other?\n\nThis is beyond the scope of our paper. Our problem assumes that all the tasks (including target tasks) share the same representation as stated in eqn.(1), so they should *not* have detrimental effects on each other. It is a widely-used assumption in various kinds of pretraining/representation learning papers (see our “literature review ” section in the responses to all). In fact, we believe negatively learning correlated-targets simultaneously is an ill-posed problem, since in that case, collecting data and learning independently for each single target is intuitively the best solution.\n\nOn the other hand, there are some existing works that consider negatively correlated *source* tasks as we mentioned in the literature review. In that case, they are mainly focused on rule-outing the harmful tasks instead of saving computational cost by selecting more useful tasks. Those works mostly have different assumptions and are beyond the scope of this paper.\n\n### Weakness: While the method and algorithms seem sound, their explanation could be better with more interpretation and transitions. The experiment section (in the main manuscript) is not well-presented, e.g., lack of baselines/setting description as well as the interpretation and justifications of results. To my best assessment, the experiments are not systematically and thoroughly conducted. Minor: The paper misses the conclusion, thorough literature review, and limitation parts\n\nWe are sorry for the confusion in presenting the method and algorithms and will try to add more explanation. For the experiment, due to the page limits, we selected the most important part in the main manuscript and left details and discussions in the appendix. We will leverage the extra page to add more intuitions and illustrations for the experimental results. For the limitation and literature part, please refer to our response to all.\n"}, {"review_id": "fWXMxytJjF", "reviewer": "Reviewer_hbuX", "summary": "The paper studies active representation learning under the assumption that the source and target tasks are parameterized by a vector space (instead of a discrete set of tasks). The proposed algorithm works by first finding a subspace of the task space that spans the representation space and then iterates between refining the subspace and estimating for the target. The theoretical analysis shows that under certain assumptions, their active approach can achieve a better sample complexity than a passive one.", "questions": "Why do the Pendulum metrics only include test loss and control error? I would expect to see a measure of success or some proxy for it (e.g., average reward).", "limitations": "N/A", "rating": 6, "confidence": 1, "soundness": 2, "presentation": 4, "contribution": 4, "strengths": "- The paper explores an understudied problem statement in active learning and theoretically demonstrates its importance in comparison to less expressive active learning problem statements, which could have a large impact on future work on the applicability of active learning. \n- The paper is organized and the theory sections are well described.", "weaknesses": "- The problem statement and motivation were not described precisely or concisely. E.g., it's not clear to me why robotics is the only target for this work. The methods section was robotics-agnostic, but the introduction uses robotics as the motivating example and the paper is titled as a robotics paper.\n- The experiment section lacks a comparison to baselines. E.g., a natural baseline would be to compare against an active learning approach that does assume a discrete source and task space. This would help show that the problem statement introduced is important empirically as well as theoretically. For this reason, I am recommending weak reject.\n\n- Nit: The methods section also appeared to be accomplishing a lot of things at once---demonstrating the necessity of the continuous task space, showing the improvement in sample complexity over the passive approach, and minimizing expected excess risk---and it's not clear to the readers how these contributions relate or should be prioritized in our minds.\n- Nit: the text on the LHS of figure 1 is cut off.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1: The problem statement and motivation were not described precisely or concisely. E.g., it's not clear to me why robotics is the only target for this work. The methods section was robotics-agnostic, but the introduction uses robotics as the motivating example and the paper is titled as a robotics paper.\n\nSince one of our contributions is to provide a general framework, it is true that this problem may work for other tasks beyond robotics as long as the task satisfies our problem statement. But we want to emphasize that one of the key improvements is to show the effectiveness of our proposed algorithm in the continuous and infinite parameter space, which is not covered by previous work. Therefore, robotics (or cyber physical systems in general) is one of the most natural applications under such a setting.\n\nThere are two specific reasons to choose robotics : 1) Robotics naturally need to deal with continuous physical environments like the wind speed, mass of pendulum, so we don’t need to “make up” those tasks. 2) Reducing the number of sources in robotics is a real demand, because switching between tasks/environments is much more expensive than collecting data in a fixed task/environment, and there are infinitely many tasks/environments to select from.\n\nWe also notice that some other domains may satisfy these properties, such as some particular vision or graphics tasks. We are excited to implement our framework in other domains in the future.\n\n### Q2: The experiment section lacks a comparison to baselines. E.g., a natural baseline would be to compare against an active learning approach that does assume a discrete source and task space. This would help show that the problem statement introduced is important empirically as well as theoretically. For this reason, I am recommending weak reject.\n\nWe apologize for this confusion. As we discussed in the responses to all, the directly relevant previous work in this area is very limited. Before we submit this paper, only [1] considers this problem in a discrete setting. Later there is a follow-up paper [2] with improved results.\n\nFirstly, because those previous works are designed for discrete tasks, they are strictly worse than our algorithm in most standard scenarios. We show it with rigourious theoretical analysis and discuss this comparison in Line 263-268. \n\nSecondly, under the discrete space setting (where each $w$ is a one-hot vector) with a single target, previous works can be seen as a computationally efficient solution under our general framework. That is,  That is, the solution q from  [2]\n\n$v = argmin_v ||v||_1 \\quad s.t. \\hat{B}_W^\\text{source} v =\\hat{B}_W^\\text{target} w_0$\n\n$q = v/||v||_1$\n\ncan be seen as an efficient approximate solution of our proposed in line 12,\n\n$\nargmin_q  (\\hat{B}_W^\\text{target} w_0)^\\top (\\hat{B}_W^\\text{source} Q (\\hat{B}_W^\\text{source})^\\top)^{-1}  (\\hat{B}_W^\\text{target} w_0)\n$,\nwhere $Q = diag(q)$ and $q$ is the distribution over all source tasks.\n\nThat is to say, we expect that, under this specific setting, our algorithm should be reduced to previous work. In fact, this is exactly what we do in our experiment: drone, where tasks are discrete, as stated line 757 -762.\n\n\n\n### Q3: Nit: The methods section also appeared to be accomplishing a lot of things at once---demonstrating the necessity of the continuous task space, showing the improvement in sample complexity over the passive approach, and minimizing expected excess risk---and it's not clear to the readers how these contributions relate or should be prioritized in our minds.\n\nWe are sorry for the confusion and will try to explain the logic here. The goal for our paper is to minimize the expected excess risk using as few source tasks and source samples as possible. To give a theoretical guarantee with explanations, Section 4 firstly provides an upper bound for the expected excess risk in terms of the sample complexity in Theorem 4.1. And then we explain this upper bound by showing that it is better to both the passive one (start from 243) and the previous active one (start from line 163). The comparison with the previous active one, consequently, implies the necessity of considering the continuous task – Because if we reduce the problem into discrete setting by random sample basis from the whole task space and apply previous AL approach, then we will get worse results. We will try to rewrite section 4 can give a brief summary on its structure. Please refer to the “scope” section in our responses to all reviewers for details and we will try to add those explanations in the extra page.\n\n### Q4: Why do the Pendulum metrics only include test loss and control error? I would expect to see a measure of success or some proxy for it (e.g., average reward).\nThe control error in this setting is the deviation from upright position, which is equivalent to negative reward https://www.gymlibrary.dev/environments/classic_control/pendulum/ [17].\n\n[17] Brockman, Greg, et al. \"Openai gym.\" arXiv preprint arXiv:1606.01540 (2016)."}, {"review_id": "Rr1LWl9se7", "reviewer": "Reviewer_F2sU", "summary": "This paper proposes a framework for active representation learning that allows learners to optimally choose which source tasks to sample from. The framework covers both task-aware and task-agnostic settings and is compatible with deep representation learning practices. The paper provides several instantiations of the framework, including a bilinear model and a neural network model. The authors demonstrate the effectiveness of their approach on a variety of simulated robotic tasks, showing that their method outperforms existing methods in terms of sample efficiency and generalization to new tasks. Overall, the paper's contributions include a novel framework for active representation learning, several instantiations of the framework, and empirical results demonstrating the effectiveness of the approach.", "questions": "Please see the Weakness.", "limitations": "Please see the Weakness.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper proposes a novel framework for active representation learning that allows learners to optimally choose which source tasks to sample from. This framework covers both task-aware and task-agnostic settings and is compatible with deep representation learning practices. It considers a more general setting where tasks are parameterized in a vector space, allowing for more effective leveraging of similarities between tasks.\n\nThe paper provides a thorough theoretical analysis of the proposed framework, including sample complexity bounds and convergence guarantees. The paper is well-written and organized, with clear explanations of the proposed framework and its instantiations.\n\nThe proposed framework has broad applicability in robotics and other domains where there are multiple related tasks to be learned.", "weaknesses": "The manuscript would be considerably enhanced by the inclusion of more comprehensive empirical assessments, specifically pertaining to real-world robotic assignments, as opposed to merely simulated tasks. Such addition would serve to confirm the applicability and efficacy of the suggested method in a broader, more realistic context.\n\nFurther elucidation through an in-depth comparative study with established methodologies, notably those tailored towards active representation learning, would aid in comprehending the proposed method's advantages and shortcomings in relation to contemporary state-of-the-art techniques.\n\nMoreover, a thorough discourse on the constraints inherent in the proposed method is warranted, especially considering the presumptions regarding the task space and task distribution. An exposition of this nature would lend itself to an improved understanding of the circumstances wherein the approach proves most efficient, as well as areas of potential inapplicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1: The manuscript would be considerably enhanced by the inclusion of more comprehensive empirical assessments, specifically pertaining to real-world robotic assignments, as opposed to merely simulated tasks. Such addition would serve to confirm the applicability and efficacy of the suggested method in a broader, more realistic context.\n\nYes, a real-world robotics data collection driven by our algorithm and more complicated robotic tasks  is definitely the next step. But this paper mainly focuses on providing a general framework with theoretical guarantees, supported by some preliminary experiments. Notably, in order to narrow the gap between simulation and the real-word, here we provide the *real-world* drone flight dataset (line 294) in the discrete task setting. Note that such a dataset is pre-collected and contains a finite number of tasks. The experiment on a real-world + continuous environment would require an interactive data collection procedure in the real world, and is left as future research directions.\n\n### Q2: Further elucidation through an in-depth comparative study with established methodologies, notably those tailored towards active representation learning, would aid in comprehending the proposed method's advantages and shortcomings in relation to contemporary state-of-the-art techniques.\n\nWe apologize for this confusion.  As we discussed in \"scope\" and \"literature review\" section in our responses to all, the directly relevant previous work in this area is very limited. Before we submit this paper, only [1] considers this problem in a discrete setting. Later there is a follow-up paper [2] with improved results.\n\nFirstly, because those previous works are designed for discrete tasks, they are strictly and provably worse than our algorithm in most standard scenarios. We discuss this in Line 263-268. Due to the rigorous theoretical guarantees, we believe there is no need to repeat the same idea with experiments.\n\nSecondly, under the discrete space setting (where each $w$ is a one-hot vector) with a single target, previous works can be seen as a computationally efficient solution under our general framework. That is,  That is, the solution q from  [2]\n\n$v = argmin_v ||v||_1 \\quad s.t. \\hat{B}_W^\\text{source} v =\\hat{B}_W^\\text{target} w_0$\n\n$q = v/||v||_1$\n\ncan be seen as an efficient approximate solution of our proposed in line 12,\n\n$\nargmin_q  (\\hat{B}_W^\\text{target} w_0)^\\top (\\hat{B}_W^\\text{source} Q (\\hat{B}_W^\\text{source})^\\top)^{-1}  (\\hat{B}_W^\\text{target} w_0)\n$,\nwhere $Q = diag(q)$ and $q$ is the distribution over all source tasks.\n\nThat is to say, we expect that, under this specific setting, our algorithm should be reduced to previous work. In fact, this is exactly what we do in our experiment: drone, where tasks are discrete, as stated line 757 -762.\n\n\n### Q3: Moreover, a thorough discourse on the constraints inherent in the proposed method is warranted, especially considering the presumptions regarding the task space and task distribution. An exposition of this nature would lend itself to an improved understanding of the circumstances wherein the approach proves most efficient, as well as areas of potential inapplicability.\n\nA: Please refer to the “limitation” section in our response to all reviewers. Notably, this is the first result in the continuous space setting so the work we can compare with is relatively limited.\n"}, {"review_id": "IyHWqcggL2", "reviewer": "Reviewer_7vYB", "summary": "This work considers the multi-task learning problem and proposes an algorithm to select related source tasks to tackle a known target task in an active learning like setup. The analysis is primarily for the linear function classes and handles discrete and continuous task spaces. The proposed algorithm operates in 3 stages: (1) a coarse sampling of the tasks (2) More efficient target-agnostic sampling of diverse tasks (3) target-aware sampling from a sub-space of related tasks. The 2nd and 3rd stage in particular allow for tighter sample complexity bounds since it allows us to efficiently sample from the task space. As a result, the sample complexity bounds depend on $k*$ which is the relationship between the target space and the relevant source space. Finally, this algorithm is used in three scenarios and an active learning strategy is shown to outperform a passive one.\n", "questions": "It seems like an important part of the algorithm is to use $g(f, A)$ from Equation 2, which estimates which estimates the most relevant source tasks. How would this change if were to use non-linear function classes. In general, I was wondering which parts of the algorithm rely on the function being linear.\n\nIn practice, the sample complexity bounds may be too pessimistic (which isn't easy to get around). How do we decide the number of samples to select in each round of active sampling and what are some other key hyper-paramters? It feels like the additional complexity of the methods leads to a lot more hyper-parameters which seems like it will be harder to get it to work in practice. \n\nThe paper has dense notation dense but I appreciate the authors effort to unpack all the results. While I was able to follow the paper, the notation can be hard to follow at times. It may help the reader if some parts are simplified in the main paper. For example the algorithm could be written without all the precise details needed to achieve the sample complexity bound.\n", "limitations": "The paper discusses some of the limitations in Appendix C in sufficient detail.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper presents a theoretically motivated algorithm to sample source tasks that are relevant to the target task. While the theme of active sampling is not entirely new, few of them provide algorithms with sample complexity bounds. While there are many gaps to fill (linear function classes, isotropic Gaussian inputs), this paper takes a step in the right direction.\n\nThere are two interesting ideas in this work. The first is to reduce the number of tasks that we consider by selecting a diverse set of tasks (subspace $V$) that spans the task space. This is the primary role of stage 2 or the task-agnostic exploration. The second idea (similar to Yifang et al.) is to perform a task-aware search of the task-space and only use source tasks that are relevant to the target task. This improves sample complexity and the benefit is stronger if a smaller subspace of source tasks is strongly related to the target task.\n\nThe paper presents many results on 3 different experimental scenarios that primarily compare passive multi-task and active target-aware learning with clear sample complexity benefits for the latter. \n", "weaknesses": "**Experimental setup**: To the best of my understanding, all the experiments in the manuscript (including appendix) compare passive to target-aware active learning strategies. How does the proposed method compare to a target-aware strategy that doesn't use stage 2 and only uses stage 3. I understand the theoretical benefit of stage 2 is clear and it allows us to work with fewer source tasks but is this reflected experimentally? \n\n**Improving the discussion on related work:** It would be nice to see the authors discuss some related work on multi-task learning and task-relatedness. This include some older work on passive multi-task learning [1,2], learning theory [3,4,5], task-distances [6,7,8], task-grouping [10,11] and weighted-training [6,12]. There are a lot more works that I haven't listed out that I think are very relevant to this work.\n\n\n1. Caruana, Rich. \"Learning many related tasks at the same time with backpropagation.\" Advances in neural information processing systems 7 (1994).\n2. Baxter, Jonathan. \"Learning internal representations.\" Proceedings of the eighth annual conference on Computational learning theory. 1995.\n3. Ben-David, Shai, et al. \"A theory of learning from different domains.\" Machine learning 79 (2010): 151-175.\n4. Hanneke, Steve, and Samory Kpotufe. \"On the value of target data in transfer learning.\" Advances in Neural Information Processing Systems 32 (2019).\n5. Crammer, Koby, Michael Kearns, and Jennifer Wortman. \"Learning from Multiple Sources.\" Journal of Machine Learning Research 9.8 (2008).\n6. Thrun, Sebastian, and Joseph O'Sullivan. \"Discovering structure in multiple learning tasks: The TC algorithm.\" ICML. Vol. 96. 1996.\n7. Zamir, Amir R., et al. \"Taskonomy: Disentangling task transfer learning.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.\n8. Achille, Alessandro, et al. \"Task2vec: Task embedding for meta-learning.\" Proceedings of the IEEE/CVF international conference on computer vision. 2019.\n9. Standley, Trevor, et al. \"Which tasks should be learned together in multi-task learning?.\" International Conference on Machine Learning. PMLR, 2020.\n10. Ramesh, Rahul, and Pratik Chaudhari. \"Model Zoo: A Growing\" Brain\" That Learns Continually.\" arXiv preprint arXiv:2106.03027 (2021).\n11. Fifty, Chris, et al. \"Efficiently identifying task groupings for multi-task learning.\" Advances in Neural Information Processing Systems 34 (2021): 27503-27516.\n12. Chen, Shuxiao, Koby Crammer, Hangfeng He, Dan Roth, and Weijie J. Su. \"Weighted training for cross-task learning.\" arXiv preprint arXiv:2105.14095 (2021).\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1.1: Experimental setup: To the best of my understanding, all the experiments in the manuscript (including appendix) compare passive to target-aware active learning strategies. \n\nWe want to correct the in our experiment 1: synthetic data, we provide comparison between passive and target-agnostic active learning strategies in Figure 3 and 4 in Appendix D.1.2. As expected, the target-agnostic AL performs similar or worse than target-aware one in terms of the excess risk on the target task, but obtains better overall model estimation, which implies it should perform better on other unaware targets. \n\nWe don’t provide the target-agnostic experiment 2:pendulum, experiment3: drone because the target-agnostic is designed for the case where $d_{\\psi_W} >> k$.\n\n### Q1.2 How does the proposed method compare to a target-aware strategy that doesn't use stage 2 and only uses stage 3. I understand the theoretical benefit of stage 2 is clear and it allows us to work with fewer source tasks but is this reflected experimentally?\n\nTo clarify what you mean for “stage 1 + stage 3”, we understand that as follows: instead of doing $\\epsilon^{-\\frac{4}{3}}$ target-agnostic exploration on the $O(k)$-subspace, we do exploration on the whole $d_{\\psi_W}$ space, and then do target-aware exploration. \n\nIn that case, in the experiment 1: synthetic data, the “error vs number of source samples” will be similar to the case using all three stages. But we need to work on $O(d)$ number of source tasks after the warmup stage, which largely increases the source tasks number we need to work to, as expected. \n\nOn the other hand, experiment 2: pendulum simulation and experiment 3: drone can not save much from stage 2 because of  $d_{\\psi_W} ~=O(k)$, as expected. Therefore, as suggested in Line 7 in Algorithm 1, in these two experiments, we did skip stage 2.\n\nBut if you are suggesting we only do $\\epsilon$-independent fix length warm-up stage once. Then we think it is unlikely to get any good results unless it’s in very special cases - the target-relevant sources are very sparse.\n\n### Q2: Improving the discussion on related work\n\nThanks for pointing this out. While some are different to our setting, we should definitely add a related work section for discussion. + standard (passive) multitask learning. Please refer to “literature review” in our comment section for more details.\n\n### Q3: It seems like an important part of the algorithm is to use from Equation 2, which estimates the most relevant source tasks... n general, I was wondering which parts of the algorithm rely on the function being linear.\n\nIf *not* considering the computational efficiency, then it is possible for $f$ in $g(f,A)$ be non-linear $\\phi_W$ as long as the structure between $\\phi_W$ and $\\phi_X$ is linear. The motivation of this optimal design idea is detailed in Appendix A.2.1. Specifically, by rearranging the previous passive learning bound, we are able to focus on minimizing the equation shown in line 490, which is equivalent to g(f, A) stated in the main algorithm. However it is unclear how to efficiently solve g(f, A) for any arbitrary nonlinear function. \n\nIn addition, our algorithm is based on previous passive learning results whose assumption on $\\phi_W$ is linear.  Estimating nonlinear $\\phi_W$ may introduce extra errors both from practical and theoretical perspectives. Therefore, we conservatively make this assumption.\n\n### Q4: How do we decide the number of samples to select in each round of active sampling and what are some other key hyper-paramters? \n\nWe agreed that, practically, some key parameters need to be adjusted either because of some idealistic assumptions or theory is pessimistically considering the worst case. Here are some main components:\n\n(1) **Number of samples $n_0$  in the warm-up stage**: One weakness is the warm-up length depends on $\\bar{\\kappa}$ which is the upper bound of the condition number of $B_W$. The exact value might be unknown to us in advance. Our current solution is to use some external knowledge to give an estimatation.\n\n(2) **Number of samples $n^j$ each round**:  In the theorem, $n^j$ depends on the model capacity. But in practice when using a neural network, it is well-known that its capacity is hard to estimate. Here, because we are using a shallow net, we simply replace $d_{\\psi_W} d_{\\psi_X}$ with the number of parameters in the net. How to choose with a more complicated model remains to be explored.\n\n(3) **Parameters in training oracle (learning rate schedule, optimization methods, epoch number)**: \n- Although we assume ERM during theoretical analysis, given the non-convexity of multi-task optimization, the training parameters of the optimizer need to be carefully selected . Here we use the initial warm-up stage result with a small fraction of held out validation data (½ of the total available few-shot target data)  to adjust the training parameters, Therefore, both passive and active algorithms use the exact same training oracles for fair comparison. \n- While here we do end-to-end training, some papers suggest more advanced training methods like alternatively minimizing the header and the representation layers may lead to better solutions.  Since the training method is not the main focus of this paper, we leave that for the future.\n\nNevertheless, we want to emphasize the importance of theory that offers a qualitative tuning method that we can refer to. For example, (1) we should have three stages and the first stage shouldn’t depend on the target accuracy. (2) The second stage is only useful when $k << d_{\\psi_W}$ (3) If using the second stage, the length should be $O( \\text{lengthOfThirdStage}^{2/3})$, etc.\n\n### Q5: The paper has dense notation dense but I appreciate the authors effort to unpack all the results ...\n\nA: Thanks for your suggestion, we will try to provide more explanation in the extra page.\n"}, {"review_id": "KJozB3fZIQ", "reviewer": "Reviewer_7KMj", "summary": "The paper introduces an algorithm for active representation learning in multi-task settings. The algorithm generalizes to settings in which the “task space” is either discrete or continuous.\n\nThe overall setting involves an (optionally) nonlinear map between an input space X and an intermediate feature space via $\\psi_X$ and $\\phi_X$, that then undergoes a final multiplication with “task features” obtained by applying $\\psi_W$ and $B_W$ to task parameters $w$. The goal is to learn $\\hat{\\phi}_X$ and $B_W$ given the possibility of actively selecting tasks $w$ within a “source” domain, and evaluating on tasks in a “target” domain.\n\nThe proposed approach can work for both “coarse exploration”, “fine target-agnostic exploration” and “fine target-aware exploration”.\n\nThe authors provide a theoretical analysis for the “benign setting” in which the source task space satisfies specific assumptions. Moreover, they provide experimental results beyond this setting, and on some real-world datasets.\n", "questions": "I would like the authors to address the points raised in “Weaknesses”, in order to better clarify my uncertainty related to the paper’s significance.\n\nEdit: increased the score after taking into account the authors' reply.", "limitations": "The authors address in the appendix some limitations related to the theoretical analysis. It would however be helpful if they could list some limitations from the point of view of how related the setting is to real-world multi-task learning problems.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "The paper is heavily theoretical in nature, and seeks to formally introduce the problem with precise assumptions, also offering theoretical guarantees given a “benign” version of the problem setting.\n\nThe experimental results show that the target-aware method indeed beats the target-agnostic method by a large margin, giving credence to the significance of the target-aware method.\n", "weaknesses": "Overall, the paper is very hard to parse, and seems to take much background as granted. In the preliminaries, a very large number of assumptions are made on the problem setting. It is unclear given my limited expertise in the subfield whether all these assumptions undermine the significance of the approach or not. It would be useful to more thoroughly compare the assumptions and design choices with the literature on the topic, to better contextualize what the paper brings to the table and what the contribution is.\n\nThe setting itself seems interesting, but not enough emphasis is put on why it is significant and worthwhile exploring. It seems that the approach effectively requires us to be able to at-will sample tasks within their “task parameter space”, which seems to be extremely hard to do for anything beyond simulation. The drone experiment seems to do away with this by defining the parameter space as a one-hot vector, but not enough emphasis is placed on the significance of such a choice.\n\nFurthermore, the results on the drone dataset experiment seem to show that the proposed method converges faster, but ultimately reaches the same accuracy as the passive strategy.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1: The setting itself seems interesting, but not enough emphasis is put on why it is significant and worthwhile exploring. It seems that the approach effectively requires us to be able to at-will sample tasks within their “task parameter space”, which seems to be extremely hard to do for anything beyond simulation. The drone experiment seems to do away with this by defining the parameter space as a one-hot vector, but not enough emphasis is placed on the significance of such a choice.\n\n**About the feasibility of “at-will sample”**:  According to my understanding, the infeasibility may comes from two perspectives: 1\\ The computationally tractable task parameter space is too idealistic  (e.g. ball, pure linear) that might not even exist in nature. 2\\ The required sample parameter exists but is hard to access due to many practical reasons.\n\nRegarding 1). In most cases, using a ball is a good approximation to the parameter space. Moreover, there always exists other approximation methods. For example, in this paper, we show adaptive sampling in this paper experiment 2: pendulum simulation. (initially mentioned in line 194-196 “harder geometry). Specifically, we first assume the augmented task parameter $\\psi_W(w)$ lies everywhere in the ball, therefore we can quickly solve a targeted “fake” $\\psi_W(w)$ whose corresponding $w$ might not exist. Next we use the method described in line 735-741 to iteratively find the closest projection onto the feasible space. The effectiveness of our method has been verified by the result.\n\nRegarding 2), we admit that there is no definitive answer at this point since the continuous hardware needs real-world interaction. We want to emphasize that this work focuses on initial theoretical validity. In addition, we conduct the pendulum simulation, which paves the way for the real-world interactive setting in the next step.\n\n**About the drone experiment**: While our experiment is ultimately designed for real-world interactive data collection, it is a common approach to start with an existing dataset as a preliminary verification step. (e.g. In usual pool-based active learning, they are aimed to simulate the interactive data annotation system but their experiments are mostly conducted on existing datasets.) \n\nTherefore, one of the reasons for using one-hot vectors is limited by the existing datasets, which are collected in a discrete way. But we admitted that this cannot help show the effectiveness of our algorithm under continuous space setting, therefore we add experiment 1: synthetic and experiment 2: pendulum simulation. \n\n### Q2: Furthermore, the results on the drone dataset experiment seem to show that the proposed method converges faster, but ultimately reaches the same accuracy as the passive strategy.\n\nIt is as expected.In our setting, the asymptotic performance of passive and active learning is supposed to be the same. Specifically, since we first learn a joint representation from the source tasks and then do a transfer learning on a few-shot target task, the final excess risk not only *depends* on the source sample complexity but also on *target sample complexity*. (see Line 242 in this paper, Eqn 18 in [1] where they consider the discrete case) As we shown in Table 5, the target number is 500, so the active and passive will finally converge to the same error level as long as they use the same number of target data.\n\nNotice that according to line 256-262 in our paper (continuous case) and Theorem 3.10 in [1] (discrete case), the $\\text{source sample number} / \\text{target sample number radio} \\approx d_{\\psi_W}$ when the target error stops decreasing. Because the other two tasks have larger $ d_{\\psi_W}$, 200 and 60, so their results didn’t reach their limits. But when it comes to the real-word data, our choice is relatively limited.\n\n### Q3: Overall, the paper is very hard to parse, and seems to take much background as granted. In the preliminaries, a very large number of assumptions are made on the problem setting. It is unclear given my limited expertise in the subfield whether all these assumptions undermine the significance of the approach or not. It would be useful to more thoroughly compare the assumptions and design choices with the literature on the topic, to better contextualize what the paper brings to the table and what the contribution is.\n\nA: We are sorry for confusion, please refer to our responses to all. We will add those discussions in the extra page in revision.\n"}]

Author rebuttal:
# Scope of this paper :

As pointed out by some reviewers, active learning, multi-task learning and representation learning is a broad area. However, our work is particularly focus on the following settings:

(1) We focus on assumptions that all the tasks share the same representation, which means learning from one task should not hurt another.

(2) We focus on data collection processing instead of the training process with a fixed candidate set of data. 

(3) We focus on the task-wise active selection (i.e., actively choose the task $w$) instead of the sample-wise selection (i.e, actively choose the input $x$) as in many classical active learning works. This is particularly useful for robotics settings where switching between tasks/environments is much more expensive than collecting data in a fixed task/environment.

(4) While we propose a general framework, one of our major contributions is on the continuous infinite task space where many robotics applications lie, which is in contrast to many previous works that particularly focus on discrete CV/NLP tasks.

Notably, this is a relatively new task so the previous works are quite limited. In particular, there is no previous active representation learning work [1][2] under the *continuous, infinite parameter space*, so we are the first one who give the general formulation with *provable* algorithm and some preliminary  experiments. More specifically, [1] propose an algorithm under discrete single target setting and [2] further showing changing regularization function in [1] will lead to better result.

Since most of the paper focused on this novel problem formulation and theoretical proofs, we regard empirical as demonstrations to theoretical results when the idealist assumptions are violated. We left a more comprehensive empirical study and real-world experiment in the future.

[1]  "Active multi-task representation learning." 

[2]  "Improved Active Multi-Task Representation Learning via Lasso." 

# Literature review on related works

Even though the setting we consider is novel and there is limited previous work, we agree with some reviewers that it is useful to discuss broad multi-task learning and representation learning settings. Here we give a brief summary of other representation learning or multi-task papers that are related but different in some aspects we mentioned above. We will add those in the extra page in the revision.

**Multi-task with negative correlation** Some multi-task works [6,7,8,9] assume different tasks *don’t* share the same representation, so learning on one task may hurt another. They usually group similar tasks and assign an independent model to each group [6,7,8] or assign high weights on target-relevant sources [9]. The essential difference between those work and ours is that they assume a pass over the whole dataset is possible and aim to achieve the ultimate best performance, whereas we assume it is not (setting a large amount of experiment environment or maintaining a long time real data collection is costly). Consequently, they should not be considered as active.

**Passive Multi-task training/Meta learning**  While our paper focuses on data collection, some papers focus on the training process with some given dataset. For example, [9] mentioned above reweighting and joint-training all tasks. Another large topic in this scope is called “Meta-learning” [10,11,12], which usually focuses on more detailed updating methods. In conclusion, this line of works is parallel to our work, and all those methods can be regarded as a plug-in oracle in Line 5, 9, 12 in Algo 1 in our paper. 

**Sample-wise data selection for representation learning** Classical pool-based active learning selects most informative data for a single task. Recently, some works [13,14,15] started to focus on selecting helpful data from a large corpus of web-scale for some known target task, where web-scale data could be seen as a mix of multi-task data without explicit “task” information.  Besides, those works usually focus on coarse labels and self-contrastive learning. Therefore, although they also aim to learn a presentation/pretrained model from non-target data, their detailed settings are quite different from ours.

**Discrete task space** Finally, most of the above papers consider discrete CV and nlp tasks while we consider robotic application that naturally has task parameters from the physical environment.

[3]  "Discovering structure in multiple learning tasks: The TC algorithm." 

[4]  "Taskonomy: Disentangling task transfer learning." 

[5]  "Task2vec: Task embedding for meta-learning." 

[6]  "Which tasks should be learned together in multi-task learning?."

[7]  "Model Zoo: A Growing" Brain" That Learns Continually."

[8]  "Efficiently identifying task groupings for multi-task learning." 

[9]  "Weighted training for cross-task learning." 

[10] [11]  [12]  (skip here due to charater limits) 

[13]  Data selection for language models via importance resampling.

[14]  DataComp: In search of the next generation of multimodal datasets.

[15] Cit: Curation in training for effective vision-language data. .

# Limitations of this setting

Some reviewers point out that the limitation of our works is not clear. Here we want to give a brief summary

**Limitations of our solutions** We have discussed the computational limitations with possible solutions in line 188 - 204, the suboptimality of generalization bound in benign case compared to the passive one in line 247-252, more theoretical limitations in Appendix C and more discussion on experiments in Appendix D

**Limitations from our setting** Comparing with related work, our solution

- is not robust to negatively correlated tasks.
- requires the existence of a good supervised (passive) multi-task representation learning oracle, which is a parallel to our study. 
- only focuses on supervised-learning, instead of modern representation learning techniques (e.g. self-contrast) .




Meta-review: The reviewers agreed to accept the paper on the grounds of introducing novel ideas for multi-task representation learning, sound theory with sample complexity bounds, and some limited experiments. The experimental side was the main concern, in particular the real world applicability. The discussion sufficiently addressed the weaknesses pointed out by the reviewers (especially comparisons with prior work), but the authors should include these updates to the camera ready.
Decision: Accept (poster)