Title: Active representation learning for general task space with applications in robotics
Venue: NeurIPS 2023
Accepted: yes
Keywords: active learning

representation learning

robotics

theory
Abstract: Representation learning based on multi-task pretraining has become a powerful approach in many domains. In particular, task-aware representation learning aims to learn an optimal representation for a specific target task by sampling data from a set of source tasks, while task-agnostic representation learning seeks to learn a universal representation for a class of tasks.  In this paper, we propose a general and versatile algorithmic and theoretic framework for \emph{active representation learning}, where the learner optimally chooses which source tasks to sample from. This framework, along with a tractable meta algorithm, allows most arbitrary target and source task spaces (from discrete to continuous), covers both task-aware and task-agnostic settings, and is compatible with deep representation learning practices. 
We provide several instantiations under this framework, from bilinear and feature-based nonlinear to general nonlinear cases. In the bilinear case, by leveraging the non-uniform spectrum of the task representation and the calibrated source-target relevance, we prove that the sample complexity to achieve $\varepsilon$-excess risk on target scales with $(k^*)^2 ||v^*||_2^2 \varepsilon^{-2}$
 where $k^*$ is the effective dimension of the target and $||v^*||_2^2 \in (0,1]$ represents the connection between source and target space. Compared to the passive one, this can save up to $\frac{1}{d_W}$ of sample complexity, where $d_W$ is the task space dimension. 
Finally, we demonstrate different instantiations of our meta algorithm in synthetic datasets and robotics problems, from pendulum simulations to real-world drone flight datasets. On average, our algorithms outperform baselines by 20%-70%.

Active Representation Learning for General Task Space with Applications in Robotics

Abstract

Representation learning based on multi-task pretraining has become a powerful approach in many domains. In particular, task-aware representation learning aims to learn an optimal representation for a specific target task by sampling data from a set of source tasks, while task-agnostic representation learning seeks to learn a universal representation for a class of tasks. In this paper, we propose a general and versatile algorithmic and theoretic framework for active representation learning , where the learner optimally chooses which source tasks to sample from. This framework, along with a tractable meta algorithm, allows most arbitrary target and source task spaces (from discrete to continuous), covers both task-aware and task-agnostic settings, and is compatible with deep representation learning practices. We provide several instantiations under this framework, from bilinear and feature-based nonlinear to general nonlinear cases. In the bilinear case, by leveraging the non-uniform spectrum of the task representation and the calibrated source-target relevance, we prove that the sample complexity to achieve ε -excess risk on target scales with ( k ∗ ) 2 ∥ v ∗ ∥ 2 2 ε -2 where k ∗ is the effective dimension of the target and ∥ v ∗ ∥ 2 2 ∈ (0 , 1] represents the connection between source and target space. Compared to the passive one, this can save up to 1 d W of sample complexity, where d W is the task space dimension. Finally, we demonstrate different instantiations of our meta algorithm in synthetic datasets and robotics problems, from pendulum simulations to real-world drone flight datasets. On average, our algorithms outperform baselines by 20% -70% . 1

1 Introduction

Recently, few-shot machine learning has enjoyed significant attention and has become increasingly critical due to its ability to derive meaningful insights for target tasks that have minimal data, a scenario commonly encountered in real-world applications. This issue is especially prevalent in robotics where data collection and training data is prohibitive to collect or even non-reproducible (e.g., drone flying with complex aerodynamics [1] or legged robots on challenging terrains [2]). One

promising approach to leveraging the copious amount of data from a variety of other sources is multi-task learning, which is based on a key observation that different tasks may share a common low-dimensional representation. This process starts by pretraining a representation on source tasks and then fine-tuning the learned representation using a limited amount of target data ([3-7]).

In conventional supervised learning tasks, accessing a large amount of source data for multi-task representation learning may be easy, but processing and training on all that data can be costly. In real-world physical systems like robotics, this challenge is further amplified by two factors: (1) switching between different tasks or environments is often significantly more expensive (e.g., reset giant wind tunnels for drones [7]); (2) there are infinitely many environments to select from (i.e., environmental conditions are continuous physical parameters like wind speed). Therefore, it is crucial to minimize not only the number of samples, but the number of sampled source tasks, while still achieving the desired performance on the target task. Intuitively, not all source tasks are equally informative for learning a universally good representation or a target-specific representation. This is because source tasks can have a large degree of redundancy or be scarce in other parts of the task space. In line with this observation, Chen et al. [8] provided the first provable active representation learning method that improves training efficiency and reduces the cost of processing source data by prioritizing certain tasks during training with theoretical guarantees. On the other hand, many existing works [9-13] prove that it is statistically possible to learn a universally good representation by randomly sampling source tasks (i.e., the passive learning setting).

The previous theoretical work of [8] on active multi-task representation learning has three main limitations. First, it only focuses on a finite number of discrete tasks, treating each source independently, and therefore fails to leverage the connection between each task. This could be sub-optimal in many real-world systems like robotics for two reasons: (1) there are often infinitely many sources to sample from (e.g., wind speed for drones); (2) task spaces are often highly correlated (e.g., perturbing the wind speed will not drastically change the aerodynamics). In our paper, by considering a more general setting where tasks are parameterized in a vector space W , we can more effectively leverage similarities between tasks compared to treating them as simply discrete and different. Secondly, the previous work only considers a single target, while we propose an algorithm that works for an arbitrary target space and distribution. This is particularly useful when the testing scenario is time-variant. Thirdly, we also consider the task-agnostic setting by selecting O ( k ) representative tasks among the d W high dimension task space, where k ≪ d W is the dimension of the shared representation. Although this result does not improve the total source sample complexity compared to the passive learning result in the bilinear setting [12], it reduces the number of tasks used in the training and therefore implicitly facilitates the training process.

In addition to those theoretical contributions, we extend our proposed algorithmic framework beyond a pure bilinear representation function, including the known nonlinear feature operator with unknown linear representation (e.g., random features with unknown coefficients), and the totally unknown nonlinear representation (e.g., deep neural network representation). While some prior works have considered nonlinear representations [9, 10, 14, 13] in passive learning, the studies in active learning are relatively limited [8]. All of these works only consider non-linearity regarding the input, rather than the task parameter. In this paper, we model task-parameter-wise non-linearity and show its effectiveness in experiments. Note that it particularly matters for task selections because the mapping from the representation space to task parameters to is no longer linear.

See more related works and how our problem scope is different from theirs in Appendix A.

1.1 Summery of contributions

We propose the first generic active representation learning framework that admits any arbitrary source and target task space. This result greatly generalizes previous works where tasks lie in the discrete space and only a single target is allowed. To show its flexibility, we also provide discussions on how our framework can accommodate various supervised training oracles and optimal design oracles. (Section 3)

We provide theoretical guarantees under a benign setting, where inputs are i.i.d. and a unit ball is contained in the overall task space, as a compliment to the previous work where tasks lie on the vertices of the whole space. In the target-aware setting, to identify an ε -good model our method requires a sample complexity of ˜ O ( kd X ( k ∗ ) 2 ∥ v ∗ ∥ 2 2 min { k ∗ , κ 2 } ε -2 ) where k ∗ is the effective dimension of the target, κ is the conditional number of representation matrix, and

∥ v ∗ ∥ 2 2 ∈ (0 , 1] represents the connection between source and target space that will be specified in the main paper. Compared to passive learning, our result saves up to a factor of k 2 d W in the sample complexity when targets are uniformly spread over the k -dim space and up to 1 d W when targets are highly concentrated. Our results further indicate the necessity of considering the continuous space by showing that directly applying the previous algorithm onto some discretized sources in the continuous space (e.g., orthonormal basis) can lead to worse result. Finally, ignoring the tasks used in the warm-up phases, in which only a few samples are required, both the target-aware and the target-agnostic cases can save up to ˜ O ( k ∗ + k ) number of tasks compared to the passive one which usually requires d W number of tasks. (Section 4)

We provide comprehensive experimental results under different instantiations beyond the benign theoretical setting, studying synthetic and real-world scenarios: 1) For the synthetic data setting in a continuous space, we provide results for pure linear, known nonlinear feature operator ψ X and unknown nonlinear representation ϕ X . Our target-aware active learning (AL) approach shows up to a significant budget saving (up to 68% ) compared to the passive approach and the target-agnostic AL approach also shows an advantage in the first two cases. 2) In a pendulum simulation with continuous task space, we provide the results for known nonlinear feature operator ψ X and ψ W and show that our target-aware AL approach has up to 20% loss reduction compared to the passive one, which also translates to better nonlinear control performance. 3) Finally, in the real-world drone dataset with a discrete task space, we provide results for unknown linear and nonlinear representation ϕ X and show that our target-aware AL approach converges much faster than the passive one. (Section 5)

2 Preliminary

Multi-task (or multi-environments). Each task or environment is parameterized by a known vector w ∈ R d W . We denote the source and target task parameter space as W source ⊂ R d W , W target ⊂ R d W . These spaces need not be the same (e.g., they could be different sub-spaces). In the discrete case, we set w as a one-hot encoded vector and therefore we have in total d W number of candidate tasks while in the continuous space, there exist infinitely many tasks. For convenience, we also use w as the subscript to index certain tasks. In addition, we use ν source ∈ ∆( W source ) , ν target ∈ ∆( W target ) to denote the task distribution for the sources and targets.

Data generation. Let X ∈ R d X be the input space. We first assume there exists some known feature/augmentation operator ψ X : X → R d ψ X ≥ d W , ψ W : W → R d ψ W ≥ d W , that can be some nonlinear operator that lifts w,x to some higher dimensional space (e.g., random Fourier features [15]). Notice that the existence of non-identical ψ indicates the features are not pairwise independent and the design space of W source is not benign (e.g., non-convex), which adds extra difficulty to this problem.

Then we assume there exists some unknown underlying representation function ϕ X : ψ ( X ) →R which maps the augmented input space ψ ( X ) to a shared representation space R ∈ R k where k ≪ d ψ X , k ≤ d ψ W , and its task counterparts ϕ W : ψ ( W ) → R which maps parameterized task space to the feature space. Here the representation functions are restricted to be in some function classes Φ , e.g., linear functions, deep neural networks, etc.

In this paper, we further assume that ϕ W is a linear function B W ∈ R k × d ψ W . To be more specific, for any fixed task w , we assume each sample ( x, y ) ∼ ν w satisfies

For convenience, we denote Z w as the collection of n w sampled data ( x 1 w , y 1 w ) , ..., ( x n w w , y n w w ) ∼ µ w . We note that when ψ X , ψ W is identity and ϕ X is linear, this is reduced to standard linear setting in many previous papers [9, 11, 12, 8].

The task diversity assumption. There exists some distribution p ∈ ∆( W source ) that E w ∼ p λ min ( B W ψ W ( w ) ψ W ( w ) ⊤ B ⊤ W ) > 0 , which suggests the source tasks are diverse enough to learn the representation.

Data collection protocol. We assume there exists some i.i.d. data sampling oracle given the environment and the budget. To learn a proper representation, we are allowed access to an unlimited n source number of data from source tasks during the learning process by using such an oracle. Then at

the end of the algorithm, we are given a few-shot of mix target data Z target = { Z w } w ∼ ν target which is used for fine-tuning based on learned representation ˆ ϕ X . Denote n target as the number of data points in Z target .

Data collection protocol for target-aware setting. When the target task is not a singleton, we additionally assume a few-shot of known environment target data ˙ Z target := { Z w , w } w ∈ ˙ W target , where | ˙ W target | = dim ( W target ) and ˙ W target = { arg max W ∈W target λ min ( WW ⊤ ) } . Again denote ˙ n target as the number of data points in ˙ Z target, we have ˙ n target ≈ n 2 / 3 target ≪ n source .

Remark 2.1 . Here | ˙ W target | represents vectors that can cover every directions of W target space. This extra ˙ Z target requirement comes from the non-linearity of l 2 loss and the need to learn the relationship between sources and targets. We want to emphasize that such an assumption implicitly exists in previous active representation learning [8] since ˙ Z target = Z target in their single target setting. Nevertheless, in a passive learning setting, only mixed Z target is required since no source selection process involves. Whether such a requirement is necessary for target-aware active learning remains an open problem.

Other notations. Let e i to be one-hot vector with 1 at i -th coordinates and let ϵ i = 2 -i .

2.1 Goals

Expected excess risk. For any target task space W target and its distribution ν target over the space, as well as a few-shot examples as stated in section 2, our goal is to minimize the expected excess risk with our estimated ˆ ϕ X

where ˆ w avg = arg min w ∑ ( x,y ) ∈ Z target ∥ ˆ ϕ X ( ψ X ( x )) w -y ∥ 2 , which average model estimation that captures the data behavior under the expected target distribution. Note that the W target , ν target are given in advance in the target-aware setting.

The number of tasks. Another side goal is to save the number of long-term tasks we are going to sample during the learning process. Since a uniform exploration over d source W -dimension is unavoidable during the warm-up stage, we define long-term task number as

where α is some arbitrary exponent and ε is the target accuracy and n w is number of samples sampled from task w as defined above.

3 A general framework

Our algorithm 1 iteratively estimates the shared representation ˆ ϕ X , ˆ B W and the next target relevant source tasks which the learner should sample from by solving several optimal design oracles

This exploration and exploitation (target-aware exploration here) trade-off is inspired by the classical ϵ -greedy strategy, but the key difficulty in our work is to combine that with multi-task representation learning and different optimal design problems. The algorithm can be generally divided into three parts, and some parts can be skipped depending on the structure and the goal of the problem.

Coarse exploration: The learner uniformly explores all the directions of the W source (denoted by distribution q 0 ) in order to find an initial k -dimension subspace V that well spans over the representation space (i.e., 1 c B W B ⊤ W ≤ B W V V ⊤ B ⊤ W ≤ cB W B ⊤ W for some arbitrary constant c ≤ d ψ W k ). To give an intuitive example, suppose B W ∈ R 2 × d source ψ W +1 has the first half column equals e 1 and the second hard equals e 2 . Then instead of uniformly choosing { e i } i ∈ [ d source ψ W ] task, we only

Algorithm 1 Active multi-task representation learning (general templates)

1: Inputs: Candidate source set W source. Classes of candidate representation function Φ X , Φ W and the known feature operator ψ X , ψ W .

2: [Target-aware only] Inputs: Target set W target and distribution ν target. Few-shot sample ˙ Z target as defined in the preliminary.

3: Stage 1: Coarse exploration. (Warm-up stage)

4: Set initial sampling distribution q 0 = g ( ψ W , I d ψ W ) where g is defined in Eqn. 2

̸

̸

5: Set n 0 ≈ poly ( d ψ X , k ) + poly ( d ψ W , k ) . Collect n 0 q 0 ( w ) data for each task denoted as { Z w } w | q 0 ( w ) =0 and update ˆ ϕ X ← O X offline 0 ( { Z w } w | q 0 ( w ) =0 , ψ X ) and ˆ B W ← O W offline ( { Z w } w | q 0 ( w ) =0 , ˆ ϕ X )

̸

6: for j = 1 , 2 , 3 , . . . do

7: Stage 2: Fine target-agnostic exploration (Directly choose q j 1 = q 0 when k = Θ( d W ) )

8: Compute the exploration sampling distribution q j 1 = g ( ˆ B W ◦ ψ W , I k )

̸

9: n j 1 ≈ poly ( d ψ X , k ) ϵ -4 3 j . Collect n j 1 q j 1 ( w ) data for each task denoted as { Z w } w | q ( w ) =0 and update ˆ ϕ X ←O X offline 1 ( { Z w } w | q j 1 ( w ) =0 , ψ X ) and ˆ B W ←O W offline ( { Z w } w | q j 1 ( w ) =0 , ˙ Z target , ˆ ϕ X )

̸

10: [Target-aware only] Stage 3: Fine target-aware exploration

11: Compute the exploitation sampling distribution q j 2 = g ( ˆ B W ◦ ψ W , Σ regu ) where Σ regu is the regularized version of ˆ B W ( E w 0 ∼ ν 0 w 0 w ⊤ 0 ) ˆ B ⊤ W after clipping out insignificant eigenvalues.

̸

12: Set n j 2 ≈ poly ( d ψ X , k ) ϵ -2 j . Collect n j 2 q j 2 ( w ) data for each task denoted as { Z w } w | q j 2 ( w ) =0 and update ˆ ϕ X ←O X offline 3 ( { Z w } w | q j 1 ( w ) =0 and q j 2 ( w ) =0 , ψ X ) .

13: end for

14: Return ˆ ϕ X

We want to highlight that the sample complexity of this warm-up stage only scales with d ψ X , k and the spectrum-related parameters of B W (i.e., κ ( B W ) , σ min ( B X ) ), not the desired accuracy ε .

Fine target-agnostic exploration: The learner iteratively updates the estimation of V and uniformly explore for ˜ O ( ϵ -4 3 j ) times on this k , instead of d ψ W subspace, denoted by distribution q 1 . (Note this ϵ -4 3 j comes from the exploration part in ϵ -greedy, which is ( n j 2 ) 2 3 ) Such reduction not only saves the cost of maintaining a large amount of physical environment in real-world experiments but also simplifies the non-convex multi-task optimization problem. Of course, when k = Θ( d ψ W ) , we can always uniformly explore the whole ( d ψ W space as denoted in the algorithm. Note that theoretically, q 1 only needs to be computed once as shown in 4. In practice, to further improve the accuracy while saving the task number, the q 1 can be updated only when a significant change from the previous one happens, which is adopted in our experiments as shown in appendix E.1.

Fine target-aware exploration. In the task-awareness setting, the learner estimates the mosttarget-related sources parameterized by { w } based on the current representation estimation and allocates more budget on those, denoted by distribution q 2 . By definition, q 2 should be more sparse than q 1 and thus allowing the final sample complexity only scales with k ∗ , which measures the effective dimension in the source space that is target-relevant.

Computational oracle for optimal design problem. Depending on the geometry of { ψ W ( w ) } w ∈W source , the learner should choose proper offline optimal design algorithms to solve g ( f, A ) . Here we propose several common choices. 1). When W source contains a ball, we can approximate the solution via an eigendecomposition-based closed-form solution with an efficient projection as detailed in Section 4. 2) When W source is some other convex geometry, we can approximate the result via the Frank-Wolfe type algorithms [16], which avoids explicitly looping over the infinite task space. 3) For other even harder geometry, we can use discretization or adaptive sampling-based approximation [17]. In our experiments, we adopt the latter one and found out that its running time cost is almost neglectable in our pendulum simulator experiment in Section 5, where the ψ W is a polynomial augmentation.

Offline optimization oracle O X offline . Although we are in the continuous setting, the sampling distribution q 0 , q 1 , q 2 is sparse. Therefore, our algorithm allows any proper passive multi-task

̸

̸

̸

learning algorithm, either theoretical or heuristic one, to plugin the O X offline . Some common choices include gradient-based joint training approaches[18-21], the general non-convex ERM [9] and other more carefully designed algorithms [12, 22]. We implement the first one in our experiments (Section 5) to tackle the nonlinear ψ X , ϕ X and give more detailed descriptions of the latter two in Section 4 and Appendix B.1 to tackle the bilinear model.

4 A theoretical analysis under the benign W source setting

4.1 Assumptions

Assumption 4.1 (Geometry of the task space) . We assume the source task space W source is a unit ball B d source W (1) that span over the first d source W ≥ 1 2 d W without loss of generality, while the target task space W target ⊂ R d W can be any arbitrary B d target W (1) .

Under this assumption, we let B source W denote the first d source W columns of B W , which stands for the source-related part of B W . And B target W

Then we assume the bilinear model where ϕ X = B X ∈ B d X × k and ψ X , ψ W = I . Therefore, d ψ X = d X , d ψ W = d W . Moreover the model satisfies the following assumptions

Assumption 4.2 (Benign B X , B W ) . B X is an orthonormal matrix. Each column of B W has magnitude Θ(1) and σ min ( B source W ) > 1 . Suppose we know ¯ κ ≥ κ ( B source W ) , σ max ( B target W ) and σ ≤ σ min ( B source W ) , σ min ( B target W ) . Trivially, ¯ κ = √ d W , σ = 1 .

Finally, the following assumption is required since we are using a training algorithm in [12] and might be able to relax to sub-gaussian by using other suboptimal oracles.

Assumption 4.3 (Isotropic Gaussian Input) . For each task w , its input i satisfies x i,w ∼ N (0 , I d ) .

4.2 Algorithm

Here we provide the target-aware theory and postpone the target-agnostic in the Appendix. C since its analysis is covered by the target-aware setting.

This target-aware algorithm 2 follows the 3-stage which corresponds to sampling distribution q 0 , q 1 , q 2 with explicit solutions. Notice that calculating q 1 once is enough for theoretical guarantees.

We use existing passive multi-task training algorithms as oracles for O X offline 1 , O X offline 2 and use the simple ERM methods for O W offline based on the learned ˆ B . For the coarse exploration and fine targetagnostic exploration stage, the main purpose is to have a universal good estimation in all directions of B X . ( i.e., upper bound the sin( ˆ B X , B X ) ) Therefore we choose the alternating minimization (MLLAM) proposed in [12]. On the other hand, for the fine target-aware exploration, we mainly care about final transfer learning performance on learned representation. Therefore, we use a non-convex ERM from [9]. We defer the details and its theoretical guarantees for O offline into Appendix B.1.

Note the major disadvantage from [9] comes from its sample complexity scaling with a number of training source tasks, which will not be a problem here since in O X offline, 3 since only k + k ∗ ≪ d W number of tasks are used. The major benefit of using non-convex ERM comes from its generality that it works even for the non-linear setting and is not tied with a specific algorithm. That is to say, as long as there exists other theoretical or heuristic oracles O X offline, 1 , O X offline, 2 giving a similar guarantee, stage 3 always works.

4.3 Results

Theorem 4.1 (Informal) . By running Algo. 2, in order to let ER ( ˆ ϕ X , ν target ) ≤ ε 2 with probability 1 -δ , the number of source samples n source is at most

Here k ∗ = rank ( E w 0 ∼ ν target B W w 0 w ⊤ 0 B ⊤ W ) represents the effective dimension of target and

Algorithm 2 Target-aware algorithm for benign source space

1: Inputs: Target probability δ , ¯ κ, σ . Some constant β 1 , β 2 , β 3 . Others same as Algo. 1.

2: Set q 0 as q 0 ( e t ) = 1 d W , ∀ t ∈ d W , and q 0 ( w ) = 0 otherwise

̸

3: Set n 0 = β 1 ¯ κ 2 ( k 3 d X ¯ κ 2 + d 3 2 W σ -2 √ k +log(1 /δ ) ) . Collect n 0 q 0 ( w ) data for each task denoted as { Z w } w | q ( w ) =0

̸

4: Update ˆ B X ←O X offline 1 ( { Z w } w | q 0 ( w ) =0 ) and ˆ B source W ←O W offline ( { Z w } w | q 0 ( w ) =0 , ˆ B X )

̸

5: Compute q 1 as q 1 ( v i ) = 1 k , ∀ i ∈ k , and q 0 ( w ) = 0 otherwise. Here v i is the i -th vector of V , where U, D, V ← SVD ( ˆ B source W )

̸

6: for j = 1 , 2 , 3 , . . . do

7: Set n j 1 = β 2 ϵ -4 3 j k 5 3 d 2 3 W d 1 3 X ( k 2 3 d 1 3 W σ -4 3 + ¯ κ 2 σ -2 3 ) . Collect n j 1 q 1 ( w ) data for each task denoted as { Z w } w | q 1 ( w ) =0 .

̸

̸

9: Find a set of target-aware tasks parameterized by ˜ W j with each column i as

8: Update ˆ B X ←O X offline 2 ( { Z w } w | q 1 ( w ) =0 ) , ˆ B source W ←O W offline ( { Z w } w | q 1 ( w ) =0 , ˆ B X ) and ˆ B target W ←O W offline ( ˙ Z target , ˆ B X )

10: Compute q j 2 as q j 2 ( w ) = 1 # col ( ˜ W j ) , ∀ w ∈ col ( ˜ W j ) and q j 2 ( w ) = 0 otherwise

11: Assign n j 2 total sampling budget as # col ( ˜ W j ) β 3 max i ∥ W ′ j ( i ) ∥ 2 2 ϵ -2 j

̸

12: Collect n j 2 ( w ) = n j 2 q j 2 ( w ) data for each task denoted as { Z w } w | q 2 ( w ) =0 .

13: Update the model, note that both data collected from stage 2 and stage 3 are used.

̸

̸

14: end for

15: Return ˜ B X

As long as the number of target samples satisfies

Comparison with passive learning. By choosing { e i } i ∈ [ d source W ] as a fixed source set, we reduce the problem to a discrete setting and compare it with the passive learning. In [9], the authors get N total as most kd X d W ∥ E w 0 ∼ ν target B W w 0 w ⊤ 0 B ⊤ W ∥ σ 2 min ( B source W ) ε -2 . We first consider the cases in their paper that the target task is uniformly spread ∥ E w 0 ∼ ν target B W w 0 w ⊤ 0 B ⊤ W ∥ = 1 k .

When the task representation is well-conditioned σ 2 min ( B source W ) = d W k . We have a passive one as ˜ O ( kd X ε -2 ) while the active one ˜ O ( kd X k 2 d W ε -2 ) (See Lemma B.8 for details), which suggests as long as d W ≫ k 2 , our active learning algorithm gain advantage even in a relatively uniform spread data and representation conditions.

Otherwise, we consider the extreme case that σ 2 min ( B source W ) = 1 . We have passive one ˜ O ( d X d W ε -2 ) while the active one ˜ O ( k 3 d X ε -2 ) . Notice here we require d W ≫ k 3 .

Both of them indicate the necessity of considering the continuous case with large d W even if everything is uniformly spread. On the other hand, whether we can achieve the same result as the passive one when d W ≤ k 3 remains to be explored in the future.

We then consider the single target w 0 case.

With well-conditioned B W , the passive one now has sample complexity O ( k 2 d X ε -2 ) while the active gives a strictly improvement O ( k 2 d X d W ε -2 ) .

With ill-conditioned B W where σ min ( B W ) = 1 and max i ∥ W ∗ i ∥ = 1 , that is, only a particular direction in source space contributes to the target. The Passive one now has sample complexity O ( kd X d W ε -2 ) while our active one only has kd X ε -2 , which demonstrates the benefits of our algorithm in unevenly distributed source space.

Comparison with previous active learning. By using the same discrete reduction and set single target w 0 , we compare our result with the current state-of-art active representation algorithm in [23]. They achieves ˜ O ( kd X ∥ ν ∥ 2 1 ε -2 ) , where ν = arg min ν ∥ ν ∥ 1 s.t B W ν = B W w 0 . On the other hand, our active one gives ˜ O ( kd X ∥ w ∗ ∥ 2 2 ε -2 ) , where w ∗ = arg min ν ∥ ν ∥ 2 s.t B W ν = B W w 0 , which is strictly better than the discrete one. This again indicates the separation between continuous and discrete cases where in fixed discrete sets, the L 1 norm regularization is strictly better than L 2 .

Furthermore, when a fixed discrete set is given, which is exactly the setting in [23]. Their algorithm can be seen as a computationally efficient reduction under ours.(Appendix B.5.)

Save task number. When ignoring the short-term initial warm-up stage, we only require maintaining ˜ O ( k + log( N total k ∗ )) number of source tasks, where the first term comes from q 1 in the target-agnostic stage and the second term comes from q 2 in the target-aware stage.

5 Experiment

In this section, we provide experimental results under different instantiations of the Algorithm 1, and all of them show the effectiveness of our strategy both in target-aware and target-agnostic settings.

5.1 Settings

Datasets and problem definition. Our results cover the different combinations of ψ X , ϕ X , ψ W as shown in Table 1. Here we provide a brief introduction for the three datasets and postpone the details into Appendix E. 2

Synthetic data. We generate data that strictly adhere to our data-generating assumptions and use the same architecture for learning and predicting. When ϕ X is nonlinear, we use a neural network ϕ X to generate data and use a slightly larger neural net for learning. The goal for synthetic data is to better illustrate our algorithm as well as serve as the first step to extend our algorithm on various existing datasets.

Pendulum simulator. To demonstrate our algorithm in the continuous space. we adopt the multienvironment pendulum model in [24] and the goal is to learn a w -dependent residual dynamics model f ( x, w ) ∈ R where x is the pendulum state and w ∈ R 5 including external wind, gravity and damping coefficients. f ( x, w ) is highly nonlinear with respective to x and w . Therefore we use known non-linear feature operators ψ X , ψ W . In other words, this setting can be regarded as a misspecified linear model. It is also worth noting that due to the non-invertibility of ψ W , the explicit selection of a source via a closed form is challenging. Instead, we resort to an adaptive sampling-based method discussed in Section 3. Specifically, we uniformly sample w from the source space, select the best w ′ , and then uniformly sample around this w ′ at a finer grain. Our findings indicate that about 5 iterations are sufficient to approximate the most relevant source.

Real-world drone flight dataset [7]. The Neural-Fly dataset [7] includes real flight trajectories using two different drones in various wind conditions. The objective is to learn the residual

aerodynamics model f ( x, w ) ∈ R 3 where x ∈ R 11 is the drone state (including velocity, attitude, and motor speed) and w is the environment condition (including drone types and wind conditions). We collect 6 different w and treat each dimension of f ( x, w ) as a separate task. Therefore w is reformulated as a one-hot encoded vector in R 18 .

For each dataset/problem, we can choose different targets. For simplicity, in the following subsection, we present results for one target task for each problem with 10 random seeds regarding random data generation and training, and put more results in Appendix E. In all the experiments, we use a gradient-descent joint training oracle, which is a standard approach in representation learning.

5.2 Results

Those results encapsulate the effectiveness of active learning in terms of budget utilization and test loss reduction. In the drone dataset, we further demonstrate its ability in identifying relevant source tasks (see Figure 2). We note that in two robotics problems (pendulum simulation and real-world drone dataset), the active learning objective is to learn a better dynamics model . However, in the pendulum simulation, we deploy a model-based nonlinear controller which translates better dynamics modeling to enhanced control performance (see Figure 1 and Appendix E.2).

0

20000

40000

60000

number of source sampels

0.04

0.06

0.08

0.10

0.12

test loss

strategy

AL_target_aware

passive

0.08

0.06

error

control

0.04

0.02

strategy

AL_target_aware

0.00

passive

0

5

10

15

20

simulation_time(s)

1.4

Strategy

lowerbound for similarityscore

0.35

AL_target_aware

0.30

1.2

passive

0.25

testloss

0.20

1.0

0.15

0.8

0.10

0.05

0.6

0.00

2000

4000

6000

8000

10000

numberofsourcesamples

drone_type_A_20_x

References

Guanya Shi, Xichen Shi, Michael O'Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural lander: Stable drone landing control using learned dynamics. In 2019 International Conference on Robotics and Automation (ICRA) , pages 9784-9790. IEEE, 2019.

Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. Learning quadrupedal locomotion over challenging terrain. Science robotics , 5(47):eabc5986, 2020.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 , 2021.

Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 , 2020.

Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917 , 2022.

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems , 35: 23716-23736, 2022.

Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural-fly enables rapid learning for agile flight in strong winds. Science Robotics , 7(66):eabm6597, 2022.

Yifang Chen, Kevin Jamieson, and Simon Du. Active multi-task representation learning. In International Conference on Machine Learning , pages 3271-3298. PMLR, 2022.

Simon S. Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei. Few-shot learning via learning the representation, provably, 2021.

Nilesh Tripuraneni, Michael I. Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity, 2020.

Nilesh Tripuraneni, Chi Jin, and Michael Jordan. Provable meta-learning of linear representations. In International Conference on Machine Learning , pages 10434-10443. PMLR, 2021.

Kiran Koshy Thekumparampil, Prateek Jain, Praneeth Netrapalli, and Sewoong Oh. Sample efficient linear meta-learning by alternating minimization. arXiv preprint arXiv:2105.08306 , 2021.

Ziping Xu and Ambuj Tewari. Representation learning beyond linear prediction functions. Advances in Neural Information Processing Systems , 34:4792-4804, 2021.

Liam Collins, Aryan Mokhtari, Sewoong Oh, and Sanjay Shakkottai. Maml and anil provably learn representations. In International Conference on Machine Learning , pages 4238-4310. PMLR, 2022.

Ali Rahimi and Benjamin Recht. Uniform approximation of functions with random bases. In 2008 46th annual allerton conference on communication, control, and computing , pages 555-561. IEEE, 2008.

Michael J Todd. Minimum-volume ellipsoids: Theory and algorithms . SIAM, 2016.

Youhei Akimoto, Yuichi Nagata, Isao Ono, and Shigenobu Kobayashi. Theoretical foundation for cma-es from information geometry perspective. Algorithmica , 64:698-716, 2012.

Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv:1909.09157 , 2019.

Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999 , 2(3):4, 2018.

Antreas Antoniou, Harrison Edwards, and Amos Storkey. How to train your maml. arXiv preprint arXiv:1810.09502 , 2018.

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence , 44 (9):5149-5169, 2021.

Shuxiao Chen, Koby Crammer, Hangfeng He, Dan Roth, and Weijie J. Su. Weighted training for cross-task learning, 2021.

Yiping Wang, Yifang Chen, Kevin Jamieson, and Simon Shaolei Du. Improved active multi-task representation learning via lasso. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 35548-35578. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/ v202/wang23b.html .

Guanya Shi, Kamyar Azizzadenesheli, Michael O'Connell, Soon-Jo Chung, and Yisong Yue. Meta-adaptive nonlinear control: Theory and algorithms. Advances in Neural Information Processing Systems , 34:10013-10025, 2021.

Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In International Conference on Machine Learning , pages 9120-9132. PMLR, 2020.

Rahul Ramesh and Pratik Chaudhari. Model zoo: A growing" brain" that learns continually. arXiv preprint arXiv:2106.03027 , 2021.

Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Processing Systems , 34:27503-27516, 2021.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126-1135. PMLR, 2017.

Hae Beom Lee, Hayeon Lee, Donghyun Na, Saehoon Kim, Minseop Park, Eunho Yang, and Sung Ju Hwang. Learning to balance: Bayesian meta-learning for imbalanced and out-ofdistribution tasks. arXiv preprint arXiv:1905.12917 , 2019.

Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic metalearning. Advances in Neural Information Processing Systems , 33:18860-18871, 2020.

Xingcheng Yao, Yanan Zheng, Xiaocong Yang, and Zhilin Yang. Nlp from scratch without large-scale pretraining: A simple and efficient framework. In International Conference on Machine Learning , pages 25438-25451. PMLR, 2022.

Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling. arXiv preprint arXiv:2302.03169 , 2023.

Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304.14108 , 2023.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an algorithm for active representation learning in multi-task settings. The algorithm generalizes to settings in which the “task space” is either discrete or continuous.\n\nThe overall setting involves an (optionally) nonlinear map between an input space X and an intermediate feature space via $\\psi_X$ and $\\phi_X$, that then undergoes a final multiplication with “task features” obtained by applying $\\psi_W$ and $B_W$ to task parameters $w$. The goal is to learn $\\hat{\\phi}_X$ and $B_W$ given the possibility of actively selecting tasks $w$ within a “source” domain, and evaluating on tasks in a “target” domain.\n\nThe proposed approach can work for both “coarse exploration”, “fine target-agnostic exploration” and “fine target-aware exploration”.\n\nThe authors provide a theoretical analysis for the “benign setting” in which the source task space satisfies specific assumptions. Moreover, they provide experimental results beyond this setting, and on some real-world datasets.\n\nSTRENGTHS:\nThe paper is heavily theoretical in nature, and seeks to formally introduce the problem with precise assumptions, also offering theoretical guarantees given a “benign” version of the problem setting.\n\nThe experimental results show that the target-aware method indeed beats the target-agnostic method by a large margin, giving credence to the significance of the target-aware method.\n\nWEAKNESSES:\nOverall, the paper is very hard to parse, and seems to take much background as granted. In the preliminaries, a very large number of assumptions are made on the problem setting. It is unclear given my limited expertise in the subfield whether all these assumptions undermine the significance of the approach or not. It would be useful to more thoroughly compare the assumptions and design choices with the literature on the topic, to better contextualize what the paper brings to the table and what the contribution is.\n\nThe setting itself seems interesting, but not enough emphasis is put on why it is significant and worthwhile exploring. It seems that the approach effectively requires us to be able to at-will sample tasks within their “task parameter space”, which seems to be extremely hard to do for anything beyond simulation. The drone experiment seems to do away with this by defining the parameter space as a one-hot vector, but not enough emphasis is placed on the significance of such a choice.\n\nFurthermore, the results on the drone dataset experiment seem to show that the proposed method converges faster, but ultimately reaches the same accuracy as the passive strategy.\n\nQUESTIONS:\nI would like the authors to address the points raised in “Weaknesses”, in order to better clarify my uncertainty related to the paper’s significance.\n\nEdit: increased the score after taking into account the authors' reply.\n\nLIMITATIONS:\nThe authors address in the appendix some limitations related to the theoretical analysis. It would however be helpful if they could list some limitations from the point of view of how related the setting is to real-world multi-task learning problems.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work considers the multi-task learning problem and proposes an algorithm to select related source tasks to tackle a known target task in an active learning like setup. The analysis is primarily for the linear function classes and handles discrete and continuous task spaces. The proposed algorithm operates in 3 stages: (1) a coarse sampling of the tasks (2) More efficient target-agnostic sampling of diverse tasks (3) target-aware sampling from a sub-space of related tasks. The 2nd and 3rd stage in particular allow for tighter sample complexity bounds since it allows us to efficiently sample from the task space. As a result, the sample complexity bounds depend on $k*$ which is the relationship between the target space and the relevant source space. Finally, this algorithm is used in three scenarios and an active learning strategy is shown to outperform a passive one.\n\nSTRENGTHS:\nThe paper presents a theoretically motivated algorithm to sample source tasks that are relevant to the target task. While the theme of active sampling is not entirely new, few of them provide algorithms with sample complexity bounds. While there are many gaps to fill (linear function classes, isotropic Gaussian inputs), this paper takes a step in the right direction.\n\nThere are two interesting ideas in this work. The first is to reduce the number of tasks that we consider by selecting a diverse set of tasks (subspace $V$) that spans the task space. This is the primary role of stage 2 or the task-agnostic exploration. The second idea (similar to Yifang et al.) is to perform a task-aware search of the task-space and only use source tasks that are relevant to the target task. This improves sample complexity and the benefit is stronger if a smaller subspace of source tasks is strongly related to the target task.\n\nThe paper presents many results on 3 different experimental scenarios that primarily compare passive multi-task and active target-aware learning with clear sample complexity benefits for the latter.\n\nWEAKNESSES:\n**Experimental setup**: To the best of my understanding, all the experiments in the manuscript (including appendix) compare passive to target-aware active learning strategies. How does the proposed method compare to a target-aware strategy that doesn't use stage 2 and only uses stage 3. I understand the theoretical benefit of stage 2 is clear and it allows us to work with fewer source tasks but is this reflected experimentally? \n\n**Improving the discussion on related work:** It would be nice to see the authors discuss some related work on multi-task learning and task-relatedness. This include some older work on passive multi-task learning [1,2], learning theory [3,4,5], task-distances [6,7,8], task-grouping [10,11] and weighted-training [6,12]. There are a lot more works that I haven't listed out that I think are very relevant to this work.\n\n\n1. Caruana, Rich. \"Learning many related tasks at the same time with backpropagation.\" Advances in neural information processing systems 7 (1994).\n2. Baxter, Jonathan. \"Learning internal representations.\" Proceedings of the eighth annual conference on Computational learning theory. 1995.\n3. Ben-David, Shai, et al. \"A theory of learning from different domains.\" Machine learning 79 (2010): 151-175.\n4. Hanneke, Steve, and Samory Kpotufe. \"On the value of target data in transfer learning.\" Advances in Neural Information Processing Systems 32 (2019).\n5. Crammer, Koby, Michael Kearns, and Jennifer Wortman. \"Learning from Multiple Sources.\" Journal of Machine Learning Research 9.8 (2008).\n6. Thrun, Sebastian, and Joseph O'Sullivan. \"Discovering structure in multiple learning tasks: The TC algorithm.\" ICML. Vol. 96. 1996.\n7. Zamir, Amir R., et al. \"Taskonomy: Disentangling task transfer learning.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.\n8. Achille, Alessandro, et al. \"Task2vec: Task embedding for meta-learning.\" Proceedings of the IEEE/CVF international conference on computer vision. 2019.\n9. Standley, Trevor, et al. \"Which tasks should be learned together in multi-task learning?.\" International Conference on Machine Learning. PMLR, 2020.\n10. Ramesh, Rahul, and Pratik Chaudhari. \"Model Zoo: A Growing\" Brain\" That Learns Continually.\" arXiv preprint arXiv:2106.03027 (2021).\n11. Fifty, Chris, et al. \"Efficiently identifying task groupings for multi-task learning.\" Advances in Neural Information Processing Systems 34 (2021): 27503-27516.\n12. Chen, Shuxiao, Koby Crammer, Hangfeng He, Dan Roth, and Weijie J. Su. \"Weighted training for cross-task learning.\" arXiv preprint arXiv:2105.14095 (2021).\n\nQUESTIONS:\nIt seems like an important part of the algorithm is to use $g(f, A)$ from Equation 2, which estimates which estimates the most relevant source tasks. How would this change if were to use non-linear function classes. In general, I was wondering which parts of the algorithm rely on the function being linear.\n\nIn practice, the sample complexity bounds may be too pessimistic (which isn't easy to get around). How do we decide the number of samples to select in each round of active sampling and what are some other key hyper-paramters? It feels like the additional complexity of the methods leads to a lot more hyper-parameters which seems like it will be harder to get it to work in practice. \n\nThe paper has dense notation dense but I appreciate the authors effort to unpack all the results. While I was able to follow the paper, the notation can be hard to follow at times. It may help the reader if some parts are simplified in the main paper. For example the algorithm could be written without all the precise details needed to achieve the sample complexity bound.\n\nLIMITATIONS:\nThe paper discusses some of the limitations in Appendix C in sufficient detail.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a framework for active representation learning that allows learners to optimally choose which source tasks to sample from. The framework covers both task-aware and task-agnostic settings and is compatible with deep representation learning practices. The paper provides several instantiations of the framework, including a bilinear model and a neural network model. The authors demonstrate the effectiveness of their approach on a variety of simulated robotic tasks, showing that their method outperforms existing methods in terms of sample efficiency and generalization to new tasks. Overall, the paper's contributions include a novel framework for active representation learning, several instantiations of the framework, and empirical results demonstrating the effectiveness of the approach.\n\nSTRENGTHS:\nThe paper proposes a novel framework for active representation learning that allows learners to optimally choose which source tasks to sample from. This framework covers both task-aware and task-agnostic settings and is compatible with deep representation learning practices. It considers a more general setting where tasks are parameterized in a vector space, allowing for more effective leveraging of similarities between tasks.\n\nThe paper provides a thorough theoretical analysis of the proposed framework, including sample complexity bounds and convergence guarantees. The paper is well-written and organized, with clear explanations of the proposed framework and its instantiations.\n\nThe proposed framework has broad applicability in robotics and other domains where there are multiple related tasks to be learned.\n\nWEAKNESSES:\nThe manuscript would be considerably enhanced by the inclusion of more comprehensive empirical assessments, specifically pertaining to real-world robotic assignments, as opposed to merely simulated tasks. Such addition would serve to confirm the applicability and efficacy of the suggested method in a broader, more realistic context.\n\nFurther elucidation through an in-depth comparative study with established methodologies, notably those tailored towards active representation learning, would aid in comprehending the proposed method's advantages and shortcomings in relation to contemporary state-of-the-art techniques.\n\nMoreover, a thorough discourse on the constraints inherent in the proposed method is warranted, especially considering the presumptions regarding the task space and task distribution. An exposition of this nature would lend itself to an improved understanding of the circumstances wherein the approach proves most efficient, as well as areas of potential inapplicability.\n\nQUESTIONS:\nPlease see the Weakness.\n\nLIMITATIONS:\nPlease see the Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies active representation learning under the assumption that the source and target tasks are parameterized by a vector space (instead of a discrete set of tasks). The proposed algorithm works by first finding a subspace of the task space that spans the representation space and then iterates between refining the subspace and estimating for the target. The theoretical analysis shows that under certain assumptions, their active approach can achieve a better sample complexity than a passive one.\n\nSTRENGTHS:\n- The paper explores an understudied problem statement in active learning and theoretically demonstrates its importance in comparison to less expressive active learning problem statements, which could have a large impact on future work on the applicability of active learning. \n- The paper is organized and the theory sections are well described.\n\nWEAKNESSES:\n- The problem statement and motivation were not described precisely or concisely. E.g., it's not clear to me why robotics is the only target for this work. The methods section was robotics-agnostic, but the introduction uses robotics as the motivating example and the paper is titled as a robotics paper.\n- The experiment section lacks a comparison to baselines. E.g., a natural baseline would be to compare against an active learning approach that does assume a discrete source and task space. This would help show that the problem statement introduced is important empirically as well as theoretically. For this reason, I am recommending weak reject.\n\n- Nit: The methods section also appeared to be accomplishing a lot of things at once---demonstrating the necessity of the continuous task space, showing the improvement in sample complexity over the passive approach, and minimizing expected excess risk---and it's not clear to the readers how these contributions relate or should be prioritized in our minds.\n- Nit: the text on the LHS of figure 1 is cut off.\n\nQUESTIONS:\nWhy do the Pendulum metrics only include test loss and control error? I would expect to see a measure of success or some proxy for it (e.g., average reward).\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces and studies a theoretic and algorithmic framework for actively and optimally selecting the source tasks in multi-task learning, accommodating task-aware and task-agnostic settings. \nThe authors present cohesive theoretical guarantees, particularly demonstrated through the pure bilinear setting with the task or sample complexity being improved. They further empirically validate their meta-algorithm with task-parameter-wise non-linearity settings, particularly with emphasis on applications in robotics, significantly outperforming baselines.\n\nSTRENGTHS:\nThe problem of active representation learning is interesting and important.\nThe proposed framework is novel and effective, demonstrated with both theoretical and empirical evidence.\nThe theoretical and experimental results are cohesive.\n\nWEAKNESSES:\n- While the method and algorithms seem sound, their explanation could be better with more interpretation and transitions.\n- The experiment section (in the main manuscript) is not well-presented, e.g., lack of baselines/setting description as well as the interpretation and justifications of results.\n- To my best assessment, the experiments are not systematically and thoroughly conducted.\n**Minor:**\n- The paper misses the conclusion, thorough literature review, and limitation parts\n\nQUESTIONS:\nSee some questions in the *weakness* sections.\n\n* What is the trade-off and limitations of the proposed algorithm compared to previous methods?\n* When it comes to a single target task -- does the proposed method outperform the baseline?\n* Given that the model applies to multiple target tasks, how does it compare to previous works when several target tasks are negatively correlated or have detrimental effects on each other?\n\nLIMITATIONS:\nThe paper does not explicitly describe the limitations of the approach.\nPlease provide the limitations and the trade-off of the method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-25"}

comments: SUMMARY:
Which source task to sample from is an important issue in multi-task learning. This paper proposes a general and versatile algorithm and framework for active representation learning. The proposed algorithm can be adapted to arbitrary target and source task space, and can cover task-aware and task-agnostic settings. The experiments demonstrate the efficacy of the proposed algorithm on various datasets.

STRENGTHS:
- Interesting and important topic.

WEAKNESSES:
- Incomplete contents.

- Poor writing and structure.

QUESTIONS:
None.

LIMITATIONS:
No limitations are addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-26