Auxiliary Learning by Implicit Differentiation
Authors: Aviv Navon
Idan Achituve
Haggai Maron
Gal Chechik
Ethan Fetaya
Venue: ICLR 2021 Poster (2021)
Keywords: Auxiliary Learning
Multi-task Learning

Abstract: Training neural networks with auxiliary tasks is a common practice for improving the performance on a main task of interest.
Two main challenges arise in this multi-task learning setting: (i) designing useful auxiliary tasks; and (ii) combining auxiliary tasks into a single coherent loss. Here, we propose a novel framework, AuxiLearn, that targets both challenges based on implicit differentiation. First, when useful auxiliaries are known, we propose learning a network that combines all losses into a single coherent objective function. This network can learn non-linear interactions between tasks. Second, when no useful auxiliary task is known, we describe how to learn a network that generates a meaningful, novel auxiliary task. We evaluate AuxiLearn in a series of tasks and domains, including image segmentation and learning with attributes in the low data regime, and find that it consistently outperforms competing methods.

Published as a conference paper at ICLR 2021
AUXILIARY LEARNING BY IMPLICIT DIFFERENTIATION
Aviv Navon"
Idan Achituve"
Haggai Maron NVIDIA, Israel
Bar-Ilan University, Israel www.vnavvn@biu.e..
Bar-Ilan University, Israel
i dan. a ach achituuee tuve@biu ac. i1
mmaron@@vidii.comm
Gal Chechik
Ethan Fetayal
Bar-Ilan University, Israel
Bar-Ilan University, Israel ethan. www.yaebiaauuaaa..
NVIDIA, Israel gal. gal.chechikbiuu.cc ac.
ABSTRACT
Training neural networks with auxiliary tasks is common practice for improving the performance main task of interest. Two main challenges arise in this multi-task learning setting: (i) designing useful auxiliary tasks; and (ii) combining auxiliary tasks into single coherent loss. Here, we propose novel framework, AuxiLearn, that targets both challenges based on implicit differentiation. First, when useful auxiliaries all losses into a single coherent objective function. This network can learn non- linear interactions between tasks. Second, when no useful auxiliary task is known, we describe how to learn a network that generates a meaningful, novel auxiliary task. We evaluate AuxiLearn in series of tasks and domains, including image segmentation and learning with attributes in the low data regime, and find that it consistently outperforms competing methods.
known, we propose learning network that combines
INTRODUCTION
The performance of deep neural networks can significantly improve by training the main task of interest with additional auxiliary tasks (Goyal et al. 2019; Jaderberg et al. 2016 Mirowski 2019). For example, learning to segment an image into objects can be more accurate when the model is simultaneously trained to predict other properties of the image like pixel depth 3D structure (Standley et al.) 2019). In the low data regime, models trained with the main task only are prone overfit and generalize poorly to unseen data (Vinyals et al. 2016). In this case, the benefits of learning with multiple tasks are amplified (Zhang and Yang) 2017). Training with auxiliary tasks adds an inductive bias that pushes learned models to capture meaningful representations and avoid overfitting to spurious correlations.
In some domains, it may be easy design beneficial auxiliary tasks and collect supervised data, For example, numerous tasks were proposed for self-supervised learning in image classification, including masking (Doersch et al., 2015), rotation (Gidaris et al. (018)) and patch shuffling (Doersch) and Zisserman 2017 Noroozi and Favaro 2016). In these cases, is not clear what would be the best way to combine auxiliary tasks into a single loss (Doersch and Zisserman 2017). The common practice is to compute weighted combination of pretext losses by tuning the weights of individual losses using hyperparameter grid search. This approach, however, limits the potential of learning with auxiliary tasks because the time of grid search grows exponentially with the number of tasks.
In other domains, obtaining good auxiliaries in the first place may be challenging or may require expert knowledge. For example, for point cloud classification, few self-supervised tasks have been proposed; however, their benefits so far are limited (Achituve et al. 2020 Hassani and Haley 2019
"Equal contributor
I Equal contributor

---

Published as a conference paper at ICLR 2021
Sauder and Sievers 2019 Tang et al. 2020). For these cases, it would be beneficial to automate the process of generating auxiliary tasks without domain expertise.
Our work takes step forward in automating the and design of auxiliary learning tasks. We name our approach AuxiLearn. AuxiLearn leverages recent progress made in implicit diffcrentiation for optimizing hyperparameters (Liao al., 2018 Lorraine al.] 2020). We demonstrate the effectiveness of AuxiLearn in two types of problems. First, in combining auxiliaries, for cases where auxiliary tasks are predefined. We describe how to train deep neural network (NN) on top of auxiliary losses and combine them non-linearly into a unified loss. For instance, we combine per-pixel losses in image segmentation tasks using a convolutional NN (CNN). Second, designing auxiliaries, for cases where predefined auxiliary tasks are not available. We present an approach for learning such auxiliary tasks without domain knowledge and from input data alone. This achieved by training an auxiliary network to generate auxiliary labels while training another, primary network to learn both the original task and the auxiliary task. One important distinction from previous works, such as (Kendall et al., 2018 Liu et al. 2019a), is that we do not optimize the auxiliary parameters using the training loss but rather on separate (small) auxiliary set, allocated from the training data. This is key difference since the goal of auxiliary learning is to improve generalization rather than help optimization on the training data.
To validate our proposed solution, we extensively evaluate AuxiLearn in several tasks in the low-data regime. In this regime, the models suffer from severe overfitting and auxiliary learning can provide the largest benefits. Our results demonstrate that using AuxiLearn leads to improved loss functions and auxiliary tasks, in terms of the performance of the resulting model on the main task. We complement our experimental section with two interesting theoretical insights regarding our model. The first shows that relatively simple auxiliary hypothesis class may overfit. The second aims to understand which auxiliaries benefit the main task.
To summarize, we propose a novel general approach for learning with auxiliaries using implicit differentiation. We make the following novel contributions: (a) We describe a unified approach for combining multiple loss terms and for learning novel auxiliary tasks from the data alone; (b) We provide a theoretical observation on the capacity of auxiliary learning: (c) We show that the key quantity for determining beneficial auxiliaries is the Newton update; (d) We provide new results variety of auxiliary learning tasks with a focus on the low data regime. We conclude that implicit differentiation can play a significant role in automating the design of auxiliary learning setups.
2 RELATED WORK
Learning with multiple tasks. Multitask Learning (MTL) aims at simultaneously solving mul- tiple learning problems while sharing information across tasks. In some cases, MTL benefits the optimization process and improves task-specific generalization performance compared to single-task learning (Standley et al. 2019). In contrast to MTL, auxiliary learning aims at solving single, main task, and the purpose of all other tasks is to facilitate the learning of the primary task. At test time, only the main task is considered. This approach has been successfully applied in multiple domains, including computer vision (Zhang et al. 2014), natural language processing (Fan et al. 2017 Trinh al. 2018), and reinforcement learning (Jaderberg et al. 2016 Lin et al. 2019).
Dynamic task weighting. When learning a set of tasks, the task-specific losses are combined into an overall loss. The way individual losses are combined is crucial because MTL-based models are sensitive to the relative weightings of the tasks (Kendall et al.) 2018). A common approach for combining task losses in linear fashion. When the number of tasks is small, task weights are commonly tuned with simple grid search. However, this approach does not extend to large number of tasks, or a more complex weighting scheme. Several recent studies proposed scaling task weights using gradient magnitude (Chen et al., (0188). task uncertainty (Kendall et al. 2018), or the rate of loss change (Liu et al. 2019b). Sener and Koltun (2018) proposed casting the multitask learning problem as multi-objective optimization. These methods assume that all tasks are equally important, and are less suited for auxiliary learning. [u et al. (2018) and Lin et al. 2019) proposed to weight auxiliary losses using gradient similarity. However, these methods do not scale well with the number of auxiliaries and do not take into account interactions between auxiliaries. In contrast, we propose to learn from data how combine auxiliaries, possibly in a non-linear manner.
2

---

Published as a conference paper at ICLR 2021
Jmotn
Brossin
Smat
Emecn
Emain
ft: :(:
Lr
ft: :W)
Lg( :1))
Cr
Foux
Mexer
Soux
Moser
Veur
(a) Combining losses
(b) Learning new auxiliary task
Figure 1: The AuxiLearn framework. (a) Learning to combine losses into single coherent loss term. Here, the auxiliary network operates over vector of losses. (b) Generating a novel auxiliary task. Here the auxiliary network operates over the input space. In both cases, g(: :) is optimized using IFT based on Ca.
Devising auxiliaries. Designing auxiliary task for given main task is challenging because it may require domain expertise and additional labeling effort. For self-supervised learning (SSL), many approaches have been proposed (see Jeeg and Tian (020)) for recent survey), but the joint representation learned through SSL may suffer from negative transfer and hurt the main task (Standley) al. 2019). Liu al. (2019a) proposed learning helpful auxiliary in meta-learning fashion, removing the need for handerafted auxiliaries. However, their system is optimized for the training data, which may lead degenerate auxiliaries. To address this issue, an entropy term is introduced to force the auxiliary network to spread the probability mass across classes.
Implicit differentiation based optimization. Our formulation gives rise to bi-level optimization problem. Such problems naturally arise in the context of meaa-nee (Finn et al.) 2017 Rajeswaran
al. 2019) and hyperparameter optimization (Bengio) 2000, |Foo al., [2008] Larsen al. 1996]
Liao et al.) 2018 Lorraine et al., 2020; Pedregosa, 2016). The Implicit Function Theorem (IFT) is
often used for computing gradients of the upper-level function, this operation requires calculating a vector-inverse Hessian product. However, for modern neural networks, it is infeasible to calculate it
explicitly, and an approximation must be devised. Luketina et al. (2016) proposed approximating the Hessian with the identity matrix, whereas Foo al. (2008); Pedregosa (2016); Rajeswaran et al. (2019) used conjugate gradient (CG) to approximate the product. Following Liao et al. (2018); Lorraine al. (2020), it was empirically shown to be more stable than CG.
truncated Neumann series and efficient vector-Jacobian products, as
OUR METHOD
We now describe the general AuxiLearn framework for learning with auxiliary tasks. For that purpose. we use two networks, a primary network that is optimized on all tasks and an auxiliary network that is optimized on the main task only. First, we introduce our notations and formulate the general objective. Then, we describe two instances of this framework: combining auxiliaries and learning new auxiliaries. Finally, we present our optimization approach for both instances.
3.1 PROBLEM DEFINITION
Let ((x}, } ; be the training set and {(x;, v,)); ; be distinet independent which term auxiliary ser. Let f( W)) denote the primary network, and let g(- :)) denote the auxiliary network. Here, W are the parameters of the model optimized on the training set, and 6 are the auxiliary parameters trained on the auxiliary set. The training loss is defined as:
= LT () Σ Emain (x), y:: W) + h(x vi W;
(1)
where Emain denotes the loss of the main task and is the overall auxiliary loss, controlled by ø. In Sections 3.2 & 3.3 we will describe two instances of h. We note that h has access to both W and ø. The loss on the auxiliary set is defined as LA = Σ; Emain (x) y; W). since we are interested the generalization performance of the main task.
3

---

Published as a conference paper at ICLR 2021
We wish to find auxiliary parameters (Ø) such that the primary parameters (W), trained with the combined objective, generalize well. More formally, we seek
arg CA (W* (o)), s.t. (0) arg min Cr.
(2)
W
3.2 LEARNING TO COMBINE AUXILIARY TASKS
Suppose we are given auxiliary tasks, usually designed using expert domain knowledge. We wish to learn how to optimally leverage these auxiliaries by learning to combine their corresponding losses. Let E(x. y; W) (Emain (x, y nm ; W), (x, y²; (W...... lx (x, y W)) denote a loss vector. We wish to learn an auxiliary network, g : R -> IR over the losses that will be added to main in order to output the training loss LT = Emain + g(e; ø). Here, h from Eq. ① is given by h(- :0) = g(:; o).
Typically, g(; σ) is chosen to be a linear combination of the losses: g(l: Ø) E, Φ3 with positive weights Φú 2 0 that are tuned using grid search. However, this method can only scale to a few auxiliaries, as the run time of grid search is exponential the number of tasks. Our method can handle a large number of auxiliaries and easily extends to a more flexible formulation in which g parametrized by a deep NN. This general form allows us to capture complex interactions between tasks, and learn non-lineat combinations of losses. See Figure Ta for illustration.
One way to view a non- combination of losses is as an adaptive linear weighting, where losses have different set of weights for each datum. If the loss at point is Emain (x, y ymain) g(((x, y)), then the gradients are Wain (x, y main + Σj dg V Vw y²). This is equivalent to an adaptive De,
loss where the loss of datum x is Emain + E, axxee and, aj 8f, approach to other studies that assign adaptive loss weighs (e.g.. Du et al. (2018); Liu et al. (2019b)).
0g. This observation connects our
Convolutional loss network. In certain problems there exists spatial relation among losses. For example, semantic segmentation and depth estimation for images. A common approach is to average the losses over all locations. In contrast, AuxiLearn can leverage this spatial relation for creating a loss-image in which each task forms a channel of pixel-losses induced by the task. We then parametrize g as CNN that acts on this loss-image. This yields a spatial-aware loss function that captures interactions between task losses. See an example of a loss image in Figure 3
Monotonicity. It common parametrize the function g(ê; Φ) as a linear combination with non- negative weights. Under this parameterization, g is a monotonic non-decreasing function of the losses. natural question that arises whether we should generalize this behavior and constrain g(4: Φ) to be non-decreasing w.r.t. the input losses as well? Empirically, we found that training with monotonic non-decreasing networks tends to be more stable and has better or equivalent performance. We impose monotonicity during training with negative weights clipping. See Appendix C.2 for a detailed discussion and empirical comparison to non-monotonic networks.
3.3 LEARNING NEW AUXILIARY TASKS
The previous subsection focused on situations where auxiliary tasks are given. In many cases, however, no useful auxiliary tasks are known in advance, and we are only presented with the main task. We now describe how to use AuxiLearn in such cases. The intuition is simple: We wish to learn an auxiliary task that pushes the representation of the primary network generalize better on the main task, as measured using the auxiliary set. We do so in a student-teache teacher manner: an auxiliary "teacher" network produces labels for the primary network (the "student") which tries to predict these labels as an auxiliary task. Both networks are trained jointly.
More specifically, for auxiliary classification, we learn a soft labeling function g(x; Φ) which produces pseudo labels Youz for input samples x. These labels are then provided to the main network f(x; W) for training (see Figure lb). During training, the primary network f(x; W) outputs two predictions, Imain for the main task and Dann for the auxiliary task. We then compute the full training loss LT Emain (Imain: Umain) + Eaux (Douz Youz) to update W. Here, the h component of Ly in update é using the auxiliary set with the loss LA Lmain- Intuitively, the teacher auxiliary network g is rewarded when provides labels to the student that help it succeed in the main task, measured using LA-
Eq. ! is given by h(- :)) Laux (f(x): W), 9x)3(x(; Φ)). As before,
4

---

Published as a conference paper at ICLR 2021
3.4 OPTIMIZING AUXILIARY PARAMETERS
We now return to the bi-level optimization problem in Eq. (2) and present the optimizing method for é. Solving Eq. (2) for é poses a problem due the indirect dependence of La on the auxiliary parameters. To compute the gradients of Φ, we need to differentiate through the optimization process over W. since VoLA= = VWLA VoW*. As in Liao al. (2018): Lorraine et al. (2020) we use the implicit function theorem (IFT) to evaluate V WW:
V&W'
(V (WWCT) VOWWCT.
(3)
W|x |
We can leverage the IFT to approximate the gradients of the auxiliary parameters Φi:
VaLa(W"()) --WCA (WW (eo)) VoWWLT
(4)
[W]X|W]
|W|X 61
See Appendix A for detailed derivation. To compute the vector and Hessian inverse product, we use the algorithm proposed by Lorraine et al. (2020), which uses Neumann approximation and efficient vector-Jacobian product. We note that accurately computing VoLA by IFT requires finding a point such that Vw Lr = O. In practice, we only approximate W* and simultaneously train both W and by altering between optimizing W on LT, and optimizing e using CA. We summarize our method Theoretical considerations regarding our method are given in Appendix
in Alg. and 2
Algorithm 1: AuxiLearn
Algorithm 2: Hypergradient
Initialize auxiliary parameters & and weights W;
Input: training loss Lr, auxiliary loss L., fixed point (ø', W"), number of
while not converged do for l = 1, 1,.,, NN do Lr Emain (x, y;; W) + 1(x,, v. 6)
iterations J, learning rate a
Vw C Caloo kkr
W t W Lr w.W
for j = 1, .... do
end
V av Vw VwCT
Φ 5- Hypergradient(CAn Lr, W)
p +- u
end
end
return V Vw Lr LTl(o'
return W
4 ANALYSIS
4.1
COMPLEXITY OF AUXILIARY HYPOTHESIS SPACE
In our learning setup, an additional auxiliary set is used for tuning a large set of auxiliary parameters. A natural question arises: could the auxiliary parameters overfit this auxiliary set? and what is the complexity of the auxiliary hypothesis space H.. Analyzing the complexity of this space is difficult because it is coupled with the hypothesis space Hw of the main model. One can think of this hypothesis space as a subset of the original model hypothesis space H.s (hw: 36 s.t. W = arg minw (W, x)} C Hw. Due to the coupling with Hw the behavior can be unintuitive. We show that even simple auxiliaries can have infinite VC dimensions.
Consider the following ID hypothesis space for binary classification Hw (ro)] W E R), which has infinite VC-dimension. Let the main loss be the zero-one loss and the auxiliary loss be h(o, W) = (Ø W)², namely, an Le regularization with learned center. Since the model hypothesis space Hw has an infinite VC-dimension, there exist training and auxiliary sets of any size that are shattered by Hw. Therefore, for any labeling of the auxiliary and training sets, we can let c = &. the parameter that perfectly classifies both sets. We then have that Φ is the optimum of the training with this auxiliary loss and we get that Hg also has infinite VC-dimension.
Example:
This important example shows that even scemingly simple- auxiliary losses can overfit due to the interaction with the model hypothesis space. Thus, it motivates our use of a separate auxiliary set.
4.2 ANALYZING AN AUXILIARY TASK EFFECT
When designing or learning auxiliary tasks, one important question is, what makes an auxiliary task useful? Consider the following loss with single auxiliary task L1 (W, = Σe Emain (x/, y W)
5

---

Published as a conference paper at ICLR 2021
Φ Lauz (x),, yi W). Here h = Φ. laur. Assume & = 0 so we optimize W only the standard main task loss. We now check if dCa de x0=0 O, namely would it help to add this auxiliary task?
Proposition 1. Let (r Φ) Emain (x), yi: W) + Φ faus (aus(x), V W). Suppose that oe = 0 and that the main task was trained until convergence. We have
dL dLx(W"(o)) dé
-VV V V
VWLT),
(5)
Lee the gradient with respect to the auxiliary weight is the inner product between the Newton methods update and the gradient of the loss on the auxiliary set.
Proof. In the general case, the following holds combination, we have V a the main task we obtain V V LT V WT.
dCs = ( VoWCr. For a linear
de
V w LT Eg Vwlauz (x,, v'). Since W is optimized till convergence of
This simple result shows that the key quantity to observe is the Newton update, rather than the gradient which often used (Lin et al., 2019 Du al. 2018). Intuitively, the Newton update the important quantity because if Aø is small then we are almost at the optimum. Then, due to quadratic convergence, a single Newton step is sufficient for approximately converging the new optimum.
EXPERIMENTS
We evaluate the AuxiLearn framework in series of tasks of two types: combining given auxiliary tasks into a unified loss (Sections(5.T] 5.3). and generating a new auxiliary task (S.ction(5.4). Further experiments and analysis of both modules are given in Appendix C Throughout all experiments, we use an extra data split for the auxiliary set. Hence, we use four data sets: training set, validation set, test set, and auxiliary set. The samples for the auxiliary set are pre-allocated from the training set. For fair comparison, these samples are used as part of the training set by all competing methods. Effectively, this means we have slightly smaller training set for optimizing the parameters W of the primary network. In all experiments, we report the mean performance (e.g.. accuracy) along with the Standard Error of the Mean (SEM). Full implementation details of all experiments are given in Appendix B Our code is available at wthhtps::/ /i gwt.ub......... com/ Av vNavon/Auxiiiinon
Model variants. For learning combine losses, we evaluated the following variants of auxiliary networks: (1) Linear: A convex linear combination between the loss terms; (2) Linear neural
network (Deep linear): A deep fully-connected NN with linear activations; (3) Nonlinear: A standard feed-forward NN over the loss terms. For Section 5.3 only (4) ConyNet: A CNN over the loss-images. The expressive power of the deep linear network is equivalent to that of a 1-layer linear network; However, from an optimization perspective, it was shown that the over-parameterization introduced by the network's depth could stabilize and accelerate convergence (Arora et al., 2018 Saxe al. 2014). All variants are constrained to represent only monotone non-decreasing functions.
5.1 AN ILLUSTRATIVE EXAMPLE
(a) main task
(b)
(c) =
We first present an illustrative exam- ple of how AuxiLearn changes the loss landscape and helps generaliza- tion the presence of label noise and harmful tasks. Consider regression problem with Ymain = w*7 x + eo and two auxiliary tasks. The first aux- iliary is helpful, y/1 = w x + E1. whereas the second auxiliary is harm- ful y/2 = wTx x + (2, W 7 w* We let 6o NO00, m and E1, 52 (O d with o oaur- We optimize a linear model with weights E IR3 that are shared across tasks, i.c., no task-specific parameters. We set w* (1, 1)T and w (2, -4)T. We train an auxiliary network to output linear task weights and observe the changes to the loss landscape in Figure 2 The left plot shows the loss landscape for the main task,
W
Ween
-1
Figure 2:
Loss landscape generated by the auxiliary network. Darker is higher. See text for details.

---

Published as a conference paper at ICLR 2021
(a) image
(b) GT labels
(c) aux. loss
(d) main loss
(e) pix. weight
Figure 3: Loss images on test examples from NYUv2: (a) original image: (b) semantic segmentation ground
truth: (c) auxiliaries loss: (d) segmentation (main task) loss; (e) adaptive pixel-wise weight Σ, OCT /
with training set optimal solution Wirain Note that Wtrain w" due the noise in the training
data. The loss landscape of the weighted train loss at the beginning (t = 0) and the end (t = T) of training is shown in the middle and right plots, respectively. Note how AuxiLearn learns ignore the harmful auxiliary and use the helpful one to find a better solution by changing the loss landscape. In Appendix C.3 we show that the auxiliary task weight inversely proportional to the label noise.
5.2 FINE-GRAINED CLASSIFICATION WITH MANY AUXILIARY TASKS
In ine-grained visual classification tasks, annotators should have domain expertise, making data labeling challenging and potentially expensive (e.g., in the medical domain). In some cases, however, non-experts can annotate visual attributes that are informative about the main task. As an example, consider the case of recognizing bird species, which would require an ornithologist, yet layman can describe the head color or bill shape of a bird. These features naturally form auxiliary tasks, which can be leveraged for training jointly with the main task of bird classification.
We evaluate AuxiLearn in this setup of fine- grained classification using the Caltech- UCSD Birds 200-2011 dataset (CUB) (Wah et al., 2011). CUB contains 200 bird species in 11,788 images, each asso- ciated with a set of 312 binary visual at- auxiliaries. Since we are interested setups where optimiz- ing the main task alone does not general- ize well, we demonstrate our method in a -uperiuper setting: we assume that auxiliary labels are available for all images but only K labels per class are available for the main task (noted as K K-shot).
Table I: Test classification accuracy on CUB 200-2011 dataset, averaged over three runs (+: SEM).
5-shot
10-shot
Top
Top 3
Top 1
Top
STL
35,50 + 0.7
54.79 + 0.7
54.79 + 0.3
74.00
0.1
tributes, which
Equal
41.47 + 0.4
62.62 + 0.4
55.36
1 0.3
75.51
= 0.4
Uncertainty
35.22 +: 0.3
54.99 i: 0.7
53.75
: 0.6
73.25
0.3
DWA
41.82
62.91
1 0.4
54.90
1 0.3
75.74
0.3
GradNor
41.49
63.12
0.4
55.23
0.1
75.62
= 0.3
GCS
4.7
62.60
55.65
0.2
75.71
0.1
Lincar
4.4
63.73 0.6
$4.77 + 0.2
75.51 = 0.7
Deep Linear
4..84
66.11
0.5
$7.08 + 02
75.3
4 0.6
Nonlincar
17.07
68.25 0.3
59.04 + 0.2
78.08 0.2
We compare AuxiLearn with the following MTL and auxiliary learning baselines: (1) Single-task learning (STL): Training only on the main task. (2) Equal: Standard multitask learning with equal weights for all auxiliary tasks. (3) GradNorm (Chen al. 2018): An MTL method that scales losses based on gradient magnitude. (4) Uncertainty (Kendall et al. 2018): An MTL approach that uses task uncertainty to adjust task weights. (5) Gradient Cosine Similarity (GCS) (Du et al. 2018): An auxiliary-learning approach that uses gradient similarity between the main and auxiliary tasks. (6) Dynamic weight averaging (DWA) (Liu et al. 2019b): An MTL approach that sets task weights based on the rate of loss change time. The primary network in all experiments is ResNet- 18 (He) jet al. 2016) are-trained on ImageNet. We use a 5-layer fully connected NN for the auxiliary network. Sensitivity analysis of the network size and auxiliary set size presented in Appendix C.4
7

---

Published as a conference paper at ICLR 2021
Table shows the test set classification accuracy. Most methods significantly improve over the STL baseline, highlighting the benefits of using additional (weak) labels. Our Nonlinear and Deep linear auxiliary network variants outperform all previous approaches by large margin. As expected, non-linear auxiliary network is better than its linear counterparts. This suggests that there are some non-linear interactions between the loss terms that the non-linear network is able capture. Also, notice the effect of using deep-linear compared to a (shallow) linear model. This result indicates that least part of the improvement achieved by our method is attributed to the per-parraettrizaaionn of the auxiliary network. In the Appendix we further analyze properties of auxiliary networks. Appendix C.3 visualizes the full optimization path of linear auxiliary network over a polynomial kernel on the losses, and Appendix C.6 shows that the last state of the auxiliary network is not informative enough.
5.3 PIXEL-WISE LOSSES
We consider the indoor-scene segmentation task from Couprie et al. (2013), that uses the NYUv2 dataset (Silberman et al. 2012). We consider the 13-class se- mantic segmentation as the main task, with depth and surface-normal prediction (Eigen and Fergus, (015) as auxiliaries. We use SegNet (Badrinarayanaa et al. 2017) based model for the primary network, and a layer CNN for the auxiliary network.
Table 2: Test results for semantic segmentation on NYUv2, averaged over four runs (#: SEM).
mloU
Pixel ncc.
STL.
18.90 +: 0.21
54.74 1: 0.94
Equal
19.20 + 0.19
55.37
1.00
Uncertainty
0.18
35.70
0.79
DWA
19.38 + 0.14
55.37
+ 0.35
GradNorm
19.52 0.21
56.70
Since losses in this task are given per-pixel, we can apply the ConvNet variant of the auxiliary network to the loss image. Namely, each task forms a channel with the per-pixel losses as values. Table 2 reports the mean Intersection over Union (mloU) and pixel accuracy for the main segmentation task. Here, we
MGDA
19.53 + 0.35
56.28
+ 0.46
GCS
19.94 + 0.13
56.58
0.81
Auxil (ours)
Lincar
20.04 0.38
56.80
0.14
Deep I. Linear
19.94
0.12
56.45
1 0.79
Nonlincar
20.09
+ 0.34
56.80
0.53
ConvNet
20.54 - 0.30
56.69 0.11
also compare with MGDA (Sener and Koltun (018)) which had extremely long training time in CUB experiments due to the large number of auxiliary tasks, and therefore was not evaluated in Section All weighting methods achieve a performance gain over the STL model. The ConvNet variant of Auxil learn outperforms all competitors in terms of test mloU.
Figure 3 shows examples of the loss loss-images for the auxiliary (c) and main (d) tasks, together with the pixel-wise -wiee weights (e). First, note how the loss-images resemble the actual input images. This suggests that a spatial relationship can be leveraged using CNN auxiliary network. Second, pixel weights are a non-trivial combination of the main and auxiliary task losses. In the top (bottom) row, the plant (couch) has a low segmentation loss and intermediate auxiliary loss. As result, higher weight is allocated to these pixels which increases the error signal.
5.4 LEARNING AUXILJARY LABELS
Table 3: Learning auxiliary task. Test accuracy averaged three runs (EMM without pre-training.
CIFARIO (5%)
CIPAR100 (5%)
SVHN (5%)
CUB (30-sbot)
Pet (30-shot)
Cars s30-sbot)
0.88
18.8
26.1 0.5
MAXL-F
50.1 0.1
20.4 = 0.6
75.4
39.6 a 1.3
26.2 a: 0.3
59.6 1.1
MAXI
58.2 0.3
2..0:0..
76.5 1: 0.4
40.7 0.6
26.3 1 0.6
60.4 1: 0.8
AuxiL
60.7 +
21.5 0.3
76.4 4:
44.5
37.0 0.6
64.4
In many cases, designing helpful auxiliaries is challenging. We now evaluate Auxil earn in learning multi-class classification auxiliary tasks. We use three multi-class elassification datasets: CIFAR 10, CIFAR100 100 (Krizhevsky et al. 2009), SVHN (Netzer et al. 20111, and three fine-graing une-ggained classification datasets: 2011, Oxford-IIIT Pet (Parkhi et al. 2012). and Cars (Krause et al. 2013). Pet contains 7349 images of 37 species of dogs and cats, and Cars contains 16, 185 images of 196 cars.
Following Liu et al. (2019)). we learn a different auxiliary task for each class of the main task. In all experiments and all learned tasks, we set the number of classes to 5 To examine the effect of the learned auxiliary losses in the low-data regime, we evaluate the performance while training with only 5% of the training set in CIFAR10, CIFAR100, and SVHN datasets, and ~ 30 samples per
8

---

Published as a conference paper at ICLR 2021
Auviitary Labels Frog
Audllary Labels Deer
Figure 4: t-SNE applied auxiliary labels learned for Frog and Deer classes, in CIFARIO. Best viewed in color.
class in CUB, Oxford-IIIT Pet, and Cars. We use VGG-16 (Simonyan and Zisserman 2014) as the backbone for both CIFAR datasets, 4-layers ConvNet for the SVHN experiment, and ResNet18 for the fine-grained datasets. In all experiments, the architectures of the auxiliary and primary networks set the same and trained from scratch without pre-training.
We compared our approach with the following baselines: (1) Single-task learning (STL): Training
the main task only. (2) MAXL: Meta AuXiliary Learning (MAXL) proposed by Liu al. (2019a)
for learning auxiliary tasks. MAXL optimizes the label generator in a meta-learning fashion. (3) MAXL-F: frozen MAXL label generator, that is initialized randomly. It decouples the effect of having teacher network from the additional effect brought by the training process.
Table 3howss that AuxiLearn outperforms all baselines in all setups, even-though it sacrifices some of the training set for the auxiliary set. It is also worth noting that our optimization approach is significantly faster than MAXI,, yielding x3 improvement in run-time. In Appendix [.9and C.10 we show additional experiments for this setup, including an extension of the method to point-cloud part segmentation and experiments with varying training data sizes.
Figure presents 2D t-SNE projection of the 5D vector of auxiliary (soft) labels that are learned using AuxiLearn. We use samples of the main classes Frog (left) and Deer (right) from the CIFARIO dataset. t-SNE applied to each auxiliary task separately. When considering how images projected to this space auxiliary soft labels, several structures emerge. The auxiliary network learns fine partition of the Frog class that separates real images from illustrations. More interesting, the soft labels learned for the class Deer have a middle region that only contains deers with antlers (in various poses and varying backgrounds). By capturing this semantic feature in the learned auxiliary labels, the auxiliary task can help the primary network to discriminate between main task classes.
6 DISCUSSION
In this paper, we presented a novel and unified approach for two tasks: combining predefined auxiliary tasks, and learning auxiliary tasks that are useful for the primary task. We theoretically showed which auxiliaries can be beneficial and the importance of using a separate auxiliary set. We empirically demonstrated that our method achieves significant improvement over existing methods on various datasets and tasks. This work opens interesting directions for future research. First, when training deep linear auxiliary networks, we observed similar learning dynamics to those of non-linear models. As result, they generated better performance compared to their linear counterparts. This effect was observed in standard training setup, but the optimization path in auxiliary networks is very different. Second, we find that reallocating labeled data from the training set to auxiliary set is consistently helpful. broader question remains what is the most efficient allocation.
ACKNOWLEDGEMENTS
This study was funded by a grant to GC from the Israel Science Foundation (ISF 737/2018), and by equipment grant to GC and Bar-Ilan -lan University from the Israel Science Foundation (ISF 2332/18). IA and AN were funded by grant from the Israeli innovation authority, through the AVATAR consortium.

---

Published as a conference paper at ICLR 2021
REFERENCES
Achituve, I,, Maron, H.,, and Chechik, G. (2020). Self-supervised learning for domain adaptation on point-clouds. arXiv preprint arXiv:2003.1264/
Alemi, A. A., Fischer. Dillon, V., and Murphy, K. (2017). Deep variational information bottleneck. In International Conference on Learning Representations.
Arora, S., Cohen, N., and Hazan, E. (2018). On the optimization of deep networks: Implicit acceleration by overparameterizatton.. International Conference on Machine Learning.
Badrinarayanan, V., Kendall, A., and Cipolla, (2017). Segnet: deep convolutional encoder- decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12):2481-23
Bengio, (2000). Gradient-bas based optimization byerrppaaametrrrss Neural computation, 12(8))1889- 1900.
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song. S., Su, H., et al. (2015). Shapenet: An information model repository. arXiv preprint arXiv:/512.000
Chen, Z., Badrinarayanan, V., Lee, C.-Y,, and Rabinovich, A. (2018). Gradnorm: Gradient normal- ization for adaptive loss balancing in deep multitask networks. In International Conference on Machine Learning, pages 794-803. PMLR.
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016). The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3213-3223.
Couprie, C., Farabet, C., Najman, L.,, and LeCun, Y. (2013). Indoor semantic segmentation using depth information. In International Conference on Learning Representations.
Deng, J., Dong. W., Socher, R., Li, L.-J,, Li, K., and Fei-Fei, L.. (2009). Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 248-255.
Doersch, C., Gupta, A., and Efros, A. A. (2015). Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision, pages 1422-1430.
Doersch, C. and Zisserman, A. (2017). Multi-task self-super visual learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 2051-2060.
Du, Y., Czarnecki, W. M., Jayakumar, S. M., Pascanu, R., and Lakshminarayanan, B. (2018). Adapting auxiliary losses using gradient similarity. arXiv preprint arXiv:1812.0222:
Eigen, D. and Fergus, (2015). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In Proceedings of the IEEE international conference on computer vision, pages 2650-2658.
Fan, X., Monti, E.,, Mathias, L.,, and Dreyer, M. (2017). Transfer learning for neural semantic parsing. In Proceedings of the 2nd Workshop on Representation Learning for NL.,, pages 48-56.
Finn, C., Abbeel, P., and Levine. S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. International Conference on Machine Learning, pages 1126-1135.
Foo, C.-s., Do, C. B., and Ng, A. Y. (2008). Efficient multiple hyperparameter learning for log-linear models. In Advances in neural information processing systems, pages 377-384.
Ganin, Y. and Lempitsky, V. (2015). Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning.
Gidaris, S., Singh, P., and Komodakis, N. (2018). Unsupervised representation learning by predicting image rotations. In International Conference on Learning Representations.
10

---

Published as a conference paper at ICLR 2021
Goyal, P., Mahajan, D., Gupta, A., and Misra, I. (2019). Scaling and benchmarking self self-supervised visual representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
Hassani, K. and Haley, M. (2019). Unsupervised multi-task feature learning point clouds. In Proceedings of the IEEE International Conference on Computer Vision, pages 1160-8171
Hc, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778.
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z.. Silver, D., and Kavukcuoglu, K. (2016). Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv: 16111653999
Jing, L. and Tian, Y. (2020). Supe-vupe visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Kendall, A., Gal, Y., and Cipolla, R. (2018). Multi-task learning using uncertainty to weigh losses for scene geometry and semanties. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482-7491.
Kingma, D. P. and Ba, J. (2014). ADAM: method for stochastic optimization. In International Conference Learning Representations.
Krause, J., Stark, M., Deng, J., and Fei-Fei, L. (2013). 3D object representations for fine- fine-graine grained categorization. In 4th International IEEE Workshop on 3D Representation and Recognition, Sydney, Australia.
Krizhevsky, A., Hinton, G., al. (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto.
Larsen, J., Hansen, L. K., Svarer, C., and Ohlsson, M. (1996). Design and regularization of neural networks: the optimal of validation set. In Neural Networks for Signal Processing VI. Proceedings of the IEEE Signal Processing Society Workshop, pages 62-71. IEEE.
Liao, R., Xiong, Y., Fetaya, E., Zhang, L., Yoon, K., Pitkow, X., Urtasun, R., and Zemel, R. (2018). Reviving and improving recurrent back-propagation. In International Conference Machine Learning.
Lin, X., Baweja, H., Kantor, G., and Held, D. (2019). Adaptive auxiliary task weighting for reinforcement learning. In Advances in Neural Information Processing Systems, pages 4773-4784.
Liu, S., Davison, A., and Johns, E. (2019a). Sele-supervised generalisation with meta auxiliary learning. Advances in Neural Information Processing Systems, pages 1677-1687.
Liu, S., Johns, E., and Davison, A. (2019b). End-to-end multi-task learning with attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1871- 1880.
Lorraine, J., Vicol, P., and Duvenaud, D. (2020). Optimizing millions of hyperparameters by implicit differentiation. In International Conference Artificial Intelligence and Statistics, pages 1540-1552. PMLR.
Luketina, J., Berglund, M., Greff, K., and Raiko, T. (2016). Scalable gradient- based tuning of continuous regularization hyperparameters. In International conference on machine learning, pages 2952-2960.
Mirowski, P. (2019). Learning to navigate. In 1st International Workshop on Multimodal Under- standing and Learning for Embodied Applications, pages 25-25.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng. A. Y. (2011). Reading digits in natural images with unsupervised feature learning.
Noroozi, M. and Favaro, P. (2016). Unsupervised learning of visual representations by solving jigsaw puzzles. In Proceedings of the European Conference Computer Vision, pages 69-84. Springer.
11

---

Published as a conference paper at ICLR 2021
Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. V. (2012). Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition.
Pedregosa, F. (2016). Hyperparameter optimization with approximate gradient. In International Conference on Machine Learning, pages 737-746.
Qi C. R., Su, H., Mo, K., and Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
Rajeswaran, A., Finn, C., Kakade, S. M., and Levine, S. (2019). Meta-learning with implicit gradients. In Advances in Neural Information Processing Systems, pages 113-124.
Salimans, T. and Kingma, D. P. (2016). Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in neural information processing systems, pages 901-909.
Sauder, J. and Sievers, B. (2019). Self-supervised deep learning on point clouds by reconstructing space. In Advances in Neural Information Processing Systems, pages 12942-12952.
Saxe, A. M., Mcclelland, J. L., and Ganguli, S. (2014). Exact solutions to the nonlinear dynamics of learning in deep linear neural network. In International Conference on Learning Representations. Citeseer.
Sener, O. and Koltun, V. (2018). Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, pages 527-538.
Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. (2012). Indoor segmentation and support inference from RGBD images. Proceedings of the European conference on computer vision, pages 746-760. Springer.
Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556.
Standley, T.. Zamir, A. R., Chen, D., Guibas, L., Malik, J., and Savarese, S. (2019). Which tasks should be learned together in multi-task learning? arXiv preprint arXiv:1905.075 175500533
Tang, L., Chen, K., Wu, C., Hong, Y., Jia, K., and Yang. Z. (2020). Improving semantic analysis on point clouds via auxiliary supervision of local geometric priors. arXiv preprint urXiv:2001.04880
Trinh, T., Dai, A., Luong, T., and Le, Q. (2018). Learning longer-term dependencies in RNNs with auxiliary losses. In International Conference on Machine Learning, pages 4965-4974.
Vinyals, O.,, Blundell, C., Lillicrap, T., Wierstra, D., et al. (2016). Matching networks for one shot learning. Advances in neural information processing systems, pages 3630-3638.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The Caltech- UCSD Birds- 200-2011 Dataset. Technical Report CNS-TR-2011 1-001, California Institute of Technology.
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., and Solomon, J. M. (2019). Dynamic graph enn for learning on point clouds. ACM Transactions on Graphics, 3815):1-1..
Yi, L., Kim, V. G., Ceylan, D., Shen, I.-C., Yan, M., Su, H., Lu, C., Huang, Q. Sheffer, A., and Guibas, L. (2016). A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics, 35(6):1-12.
Zhang, Y. and Yang, Q. (2017). A survey on multi-task learning. arXiv preprint arXiv: 14.
Zhang, Z., Luo, P., Loy, C. C., and Tang, X. (2014). Facial landmark detection by deep multi-task learning. European conference on computer vision, pages 94-108. Springer.
12

---

Published as a conference paper at ICLR 2021
Appendix: Auxiliary Learning by Implicit Differentiation
A GRADIENT DERIVATION
We provide here the derivation of Eq. 44 in Section One can look the function Vw (T, around certain local-minima point (W. .)) and assume the Hessian WT Φ) is positive- definite. At that point, we have Vw CrWWrrWW, .) = O. From the IFT, we have that locally around (W, Ø), there exists smooth function (Ø) such that Vw WwCr(W, Φ) = 0 Of = (o). Since the function V (W" Φ) is constant and equal to zero, we have that its derivative w.r.t is
also zero Taking the total derivative we obtain
Vi (W V + VVV
(6)
Multiplying by W (W, Φ) -1 and reordering we obtain
V V (Ø) = Vi CTW, Φ) VV, Φ)
We can use this result to compute the gradients of the auxiliary set loss w.r.t
V VoLa (6)) = V CA V (0) = - CA (V VW
(8)
As discussed in the main text, fully optimizing W to convergence is too computationally expensive. Instead, we update & once for every several update steps for W, as seen in Alg. To compute the vector inverse- Hessian product, we Alg. 2 that proposed in (Lorraine al. 2020).
B EXPERIMENTAL DETAILS
B.1
CUB 200-2011
Data. To examine the effect of varying training set sizes we use all 5994 predefined images for training according to the official split and, we split the predefined test set to 2897 samples for validation and 2897 for testing. All images were resized to 256 256 and Z-score normalized. During training, images were randomly cropped to 224 and flipped horizontally. Test images were centered cropped to 224. The same processing was applied in all fine-grain experiments.
Training details for baselines. We fine-tuned ResNet-18 88 (He et al. 2016) pre-trained on ImageNet (Deng et al., 2009) with a classification layer on top for all tasks. Because the scale of auxiliary losses differed from that of the main task, we multiplied each auxiliary loss, on all compared method, by the scaling factor v = 0. was chosen based on a grid search over 10.1, 0.3, 0.6, 1.0} using the Équal baseline. We applied grid search
the learning rates in (1e 3, 1e 4, le 5) and the weight decay in (5e- - 3, 5e 4, 5e - 5}. For DWA (Liu et al. 2019b), we searched over the temperature in {0.5, 2, 5) and for GradNorm (Chen al. 2018), over a in (0.3, 0.8, 1.5). The computational complexity of GSC (Du et al. 2018) 18 grows with the number of tasks. As result, we were able to run this baseline only in a setup where there are two loss terms: the main and the sum of all auxiliary tasks. We ran each configuration with 3 different seeds for 100 epochs with ADAM optimizer (Kingma and Ba 2014) and used early stopping based on the validation set.
The auxiliary set and auxiliary network. In our experiments, we found that allocating as little as 20 samples from the training set for the auxiliary set and using a NN with layers and 10 units in each layer yielded good performance for both deep linear and non-linear models. We found that our method was not sensitive to these design choices. We use skip connection between the main loss Caai and the overall loss term and Softplus activation.
Optimization of the auxiliary network. In all variants of our method, the auxiliary network was optimized using SGD with 0.9 momentum. We applied grid search over the auxiliary network learning rate in [le - 2, 1e - 3} and weight decay in {le 5, 5e - 5}. The total training time of all methods was 3 hours on 16GB Nvidia V100 GPU.
B.2 NYUv2
The data consists of 1449 RGB-D images, split into 795 train images and 654 test images. We further split the train set to allocate 79 images, 10% of training examples, construct validation
13

---

Published as a conference paper at ICLR 2021
set. Following (Liu al., 2019b), we resize images to 288 x 384 pixels for training and evaluation and use SegNet Badrinarayanan et al. 2017) based architecture as the backbone.
Similar to (Liu et al. 2019b), we train the model for 200 epochs using Adam optimizer (Kingma and Ba 2014 with learning rate 1e- - 4, and halve the learning rate after 100 epochs. We choose
the best model with early stopping a pre-allocated validation set. For DWA (Liu et al.) 2019b)
we set the temperature hyperparameter to 2, as in the NYUv2 experiment in (Liu et al.) (019b)) For GradNorm (Chen al. 2018)) we set 1.5. This value for a was used in (Chen al. (2018) for the NYUv2 experiments. In all variants of our method, the auxiliary networks are optimized using SGD with 0.9 momentum. We allocate 2.5% of training examples to form an auxiliary set. We use grid search to tune the learning rate {le 3, 5e 4, le - 4 and weight decay {le - 5, le - 4] of the auxiliary networks. Here as well, we use skip connection between the main loss Emain and the overall loss term and Softplus activation.
B.3 LEARNING AUXILIARIES
Multi-class classification datasets. On the CIFAR datasets, we train the model for 200 epochs using SGD with momentum 0.9, weight decay 5e - 4, and initial learning rates 1e - and le - 2 for CIFARIO and CIFARIOM respectively. For the SVHN experiment,
train for 50 epochs using SGD with momentum 0.9, weight decay 5e - 4. and initial learning rates le 1. The learning rate modified using cosine annealing scheduler. We use VGG-16 (Simonyan and Zisserman 2014) based architecture for the CIFAR experiments, and a 4-layer ConvNet for the SVHN experiment. For MAXL (Liu et al.) 2019a) label generating network, we tume the following hyperparameters: learning rate {le- 3, 5c 4f. weight decay (5e - 4, le - 4, 5e - 5}, and entropy term weight {.2, .4 .6} (see (Liu et al., 2019a) for details). We explore the same learning rate and weight decay for the auxiliary network in our method, and also tune the number of optimization steps between every auxiliary parameter update {5, 15, 25}, and the size of the auxiliary set {1.5%, 2.5%} (of training examples). We choose the best model on the validation set and allow for early stopping.
Fine-grain classification datasets. In CUB experiments we use the same data and splits as described in Sections 5.2 and B.I Oxford-IIIT Pet contains 7349 images of 37 species of dogs and cats. We use the official train-test split. We pre-allocate 30% from the training set to validation. As a results, the total number of train/validation/eest images are 2576/1 1 104/ respectively. Cars (Krause et al. 2013) contains 16, 185 images of 196 car classes. We use the official train-test split and pre-allocate 30% from the training set to validation. As results, the total number of train/validation/test images are 7700/2444/20 /8041 respectively. In all experiments we use ResNet-18 as the backbone network for both the primary and auxiliary networks. Importantly, the networks are not pre-trained. The task specific (classification) heads in both the primary and auxiliary networks is implemented using a 2-layer NN with sizes 512 and C. Where C' number of labels (e.g., 200 for CUB and 37 for Oxford-IIIT Pet). In all experiments we use the same learning rate of 1e 4 and weight decay of 5e - which were shown to work best, based on grid search applied on the STL. baseline. For MAXL and AuxiLeam we applied grid search over the auxiliary network learning rate and weight decay as described in the Multi-class classification datasets subsection. We tune the number of optimization steps between every auxiliary parameter update in (30, 60) for Oxford-IIT Pet and (40, 80} for CUB and Cars. Also, the auxiliary set size was tuned over (0.084%, 1.68% 3.33%) with stratified sampling. For our method, we leverage the module of AuxiLeam for combining auxiliaries. We use a Nonlinear network with either two or three hidden layers of sizes 10 (which was selected according to grid search). The batch size was set to 64 in CUB and Cars experiments and to 16 in Oxford-IIIT Pet experiments, We ran each configuration with 3 different seeds for 150 epochs with ADAM optimizer and used early stopping based on the validation set.
C ADDITIONAL EXPERIMENTS
C.I IMPORTANCE OF AUXILIARY SET
In this section we illustrate the importance of the auxiliary set to complement our theoretical observation in Section 4 We repeat the experiment in Section [.]] but this time we optimize the auxiliary parameters Φ using the training data. Figure (5) shows how the tasks' weights change during training. The optimization procedure is reduced to single-task learning, which badly hurts
14

---

Published as a conference paper at ICLR 2021
task Helpful
8
Harmful aux.
Weight
200
400
600
800
1000
Step
Figure Optimizing task weights the training reduce single-task task learning.
generalization (see Figure These results consistent with (Liu et al, 2019a) that added an entropy loss term to avoid the diminishing auxiliary task.
C.2 MONOTONOCITY
As discussed in the main text, it is a common practice combine auxiliary losses as convex combination. This is equivalent to parametrize the function g(:; Φ) as a linear combination over losses g(:; = Σド= Czly. with non-negative weights, Φ 2 0. Under this parameterization, g is monotonic non-decreasing function of the losses, since 0LT 2 0. The non-decreasing property means that the overall loss grows (or is left unchanged) with any increase to the auxiliary losses. As result, an optimization procedure that operates minimize the combined loss also operates in the direction of reducing individual losses (or not changing them).
natural question that arises is whether the function g should generalize this behavior, and be constrained to be non-decreasing w.r.f. the losses as well? Non-decreasing networks can "ignore" an auxiliary task by zeroing corresponding loss, but cannot reverse the gradient of a task by negating its weight. While monotonicity is very natural requirement, in some cases, negative task weights (i.e., non-monotonicity) seem desirable one wishes to "delete" input information not directly related to the task at hand (Alemi et al. 2017 Ganin and Lempitsky) (2015). For example, in domain adaptation, one might want to remove information that allows a discriminator to recognize the domain of given sample (Ganin and Lempitsky, 2015). Empirically, found that training with monotonic non-decreasing networks to more stable and has better or equivalent performance, see Table 4 for comparison.
Table compares monotonic and non-monotonic auxiliary networks in both the semi-supervised and the fully sulle-suee setting. Monotonic networks show small but consistent improvement non-monotonic ones. It is also worth mentioning that the non-monotonic networks were harder to stabilize.
Table 4: CUB 200-2011: Monotonic non-monotonic test classification accuracy (1: SEM) over three runs.
Top
Top 3
Non-Monotonic
46.3 + 0.32
67.46 + 0.55
5-shot
Monotonic
47.07 0.10
68.25 = 0.32
Non-Monotonic
58.84 + 0.04
77.67 + 0.08
10-shot
Monotonic
59.04 0.22
78.08 0.24
Mon-Monotonis
74.74 0.30
88.3 0.23
Full Dataset
Monotonic
74.92 + 0.21
88.55 + 0.17
15

---

Published as a conference paper at ICLR 2021
C.3 NOISY AUXILIARIES
We demonstrate the effectiveness of AuxiLearn in identifying helpful auxiliaries and ignoring harmful ones. Consider regression problem with main task y w x where e e~ NO, o²). We learn this task jointly with K = 100 auxiliaries of the form y, = w x + lcyl, where Ej ~ (O j o ) for j 1 1,.,, 100. We use the absolute value on the noise so that noisy estimations are no longer unbiased, making the noisy labels even less helpful the noise increases. We use linear auxiliary network to weigh the loss terms.
the learned weight for each task. We can see that the auxiliary network captures the noise patterns, and assign weights based on the noise level.
Weight
0.5
0.4
0.2
0.0
20
40
Auxiliary task
Figure 6: Learning with noisy labels: task ID is proportional to the label noise.
C.4 CUB SENSITIVITY ANALYSIS
In this section, we provide further analysis for the experiments conducted on the CUB 200-2011 I dataset the 5-shot setup. We examine the sensitivity of a non-linear auxiliary network to the size of the auxiliary set, and the depth of the auxiliary network. In Figure [7 we test the effect of allocating (labeled) samples from the training set to the auxiliary set. As seen, allocating between 10 50 samples results in similar performance picking 20. The figure shows that removing too many samples from the training set can be damaging, Nevertheless, we notice that even when allocating 200 labeled samples (out of 1000), our nonlinear method is still better than the best competitor GSC (Du et al. 2018) (which reached an accuracy of 42.57).
Figure 7b shows how accuracy changes with the number of hidden layers. As expected, there is a positive trend. As we increase the number of layers, the network expressivity increases, and the performance improves. Clearly, making the auxiliary network too large may cause the network to overfit the auxiliary set as was shown in Section 4 and empirically in (Lorraine et al. 2020).
C.5 LINEARLY WEIGHTED NON-LINEAR TERMS
To further motivate the use of non-linear interactions between tasks, we train linear auxiliary network over a polynomial kernel on the tasks segmentation, depth estimation and normal prediction from the NYUv2 dataset. Figure g shows the learned loss weights. From the figure, we learn that two of the three largest weights at the end of training belong to non-linear terms, specifically, Seg" and Seg Depth. Also, we observe scheduling effect, in which at the start of training, the auxiliary network focuses on the auxiliary tasks (first ~ 50 steps), and afterwards draws most of the attention of the primary network towards the main task.
C.6 FIXED AUXILIARY
As result of alternating between optimizing the primary network parameters and the auxiliary parameters, the weighting of the loss terms are updated during the training process. This means that the loss landscape is changed during training. This effect is observed in the illustrative examples
16

---

Published as a conference paper at ICLR 2021
48-
Accutoyy
60
43
40
20
50
200
@samples
Num of Hidden Leyers
(a) Effect of auxiliary set size
(b) Effect of Depth
Figure 7: Mean test accuracy (#: SEM) averaged runs function of the number of samples in the auxiliary set (left) and the number of hidden layers (right). Results are on 5-shot CUB 200-2011 datasct.
Polynomial kernel
linear weights
Seg
Depeh
Rormal
Sog²
Dape)?
We ht
Normal
100
150
200
250
300
350
Step
Figure 8: Learned linear weights for polynomial kernel the loss terms of the tasks segmentation, depth estimation and normal prediction from NYUv2 dataset.
described in Section 5.1 and Section 5
where the auxiliary network focuses on different tasks during different learning stages. Since the optimization is non-convex, the end result may depend not only the final parameters but also the loss landscape during the entire process.
We examined this effect with the following setup on the 5-shot setting on CUB 200-2011 dataset: trained non-linear auxiliary network and saved the best model. Then retrain with the same configuration, only this time, the auxiliary network initialized using the best model, and is kept fixed. We repeat this using ten different random seeds, affecting the primary network initialization and data shuffling. As a result, we observed drop of 6.7% on average in the model performance with std of 1.2% (46.7% compared 40%).
C.7 FULL CUB DATASET
we evaluated AuxiLeam and the baseline models performance under semi-supervised scenario which we have 5 or 10 labeled samples per class. For completeness sake, we show in Table 5 the test accuracy results in the standard fully-supervised scenario. As can be seen, in this case the STL baseline achieves the highest top-1 1 test accuracy while our nonlinear method is second on the top-1 and first the top-3. Most basclines suffer from severe negative transfer due to the large number auxiliary tasks (which are not needed this case) while our method cause minimal performance degradation.
In Section 5.2
17

---

Published as a conference paper at ICLR 2021
Table 5: CUB 200-2011: Fully supervised test classification accuracy (= SEM) averaged over three
Top
Top
STI.
75.2 0.52
88.4 + 0.36
Equal
70.16 + 0.10
86.87 + 0.22
Uncertainty
74.70 = 0.56
88.21 + 0.14
DWA
69.88 0.10
86.62 1 0.20
GradNorm
70.04 0.21
86.63 0.13
GSC
71.30 0.01
86.91 0.28
AuxiLearn (ours)
Linear
70.9744: 0.31
86.92
0.08
Deep Linear
73.6 = 0.72
88.37 0.21
Nonlinear
74.92 + 0.21
88.55 0.17
C.8 CITYSCAPES
Cityscapes (Cordts et al. 2016 is high-quality urban-scene dataset. We use the data provided in (Liu al. 2019b) with 2975 training and 500 test images. The data comprises of four learning tasks: 19-classes, 7-classes and 2-classes semantic segmentation, and depth estimation. We use the 19-classes semantic segmentation as the main task, and all other tasks as auxiliaries. We allocate 10% of the training data for validation set, to allow for hyperparameter tuning and early stopping. We further allocate 2.5% of the remaining training examples to construct the auxiliary set. All images resized 128 x 256 to speed up computation.
We train SegNet (Badrinarayanan et al. 2017) based model for 150 epochs using Adam opti- mizer (Kingma and Ba 2014) with learning rate Ic 4, and halve the learning rate after 100 epochs. We search over weight decay in {le - 4, le 5]. We compare Auxil Laarn to the same baselines used in Section 5.2 and search over the same hyperparameters in the NYUv2 experiment. We set the DWA temperature to similar to (Liu et al. 20196), and the GradNorm hyperparameter a to 1.5, used in (Chen al., 2018) for the NYUv2 experiments. We present the results in Table 6 The ConvNet variant of the auxiliary network achieves best performance in terms of mloU and pixel accuracy.
Table 6: 19-classes semantic segmentation test set results on Cityscapes, averaged over three runs (= SEM).
mloU
Pixel acc.
STL.
30.18 + 0.04
87.08 + 0.18
Equal
0.14
87.14
Uncertainty
30.49 0.21
86.89
0.07
DWA
30.79 + 0.32
86.97 + 0.26
GradNorn
30.62 1 0.03
87.15 1 0.04
GCS
30.32 + 0.23
87.02 + 0.12
AuxiLearn (ours)
Lincar
30.63 - 0.19
86.88 + 0.03
Nonlincar
30.85 + 0.19
87.19 + 0.20
ConvNet
30.99 0.05
87.21 + 0.11
C.9 LEARNING SEGMENTATION AUXILIARY FOR 3D POINT CLOUDS
Recently, several methods were offered for learning auxiliary tasks in point clouds (Achituve et al.. 2020; Hassani and Haley 2019 Sauder and Sievers 2019); however, this domain is still largely unexplored and is not yet clear which auxiliary tasks could beneficial beforchand. Therefore, it is desirable to automate this process, compared to human designed methods.
at the cost of performance degradation to some extent
We further evaluate method in the task of generating helpful auxiliary tasks for 3D point-cloud data. We propose to extend the use of AuxiLeam for segmentation tasks. In Section (5.4 we trained auxiliary network to output soft auxiliary labels for classification task. Here, we similar
18

---

Published as a conference paper at ICLR 2021
Table 7: Learning auxiliary segmentation task. Test mean IOU ShapeNet part dataset averaged over three runs (EM) 30 shot
Moin
Airplane
Bag
Cap
Chair
Farphone
Guitar
Knife
1 Lamp
Taptop
Motorhike
Mag
Pisnoi
Rocket
Skitchoard
Table
samples
2874
11
704
159
83
44
818
SIL
75.6
68.7
82,9
85.2
65.6
82.3
70.2
86.1
75.1
68.4
94.3
55.1
91.0
72.6
12.3
74.2
74.0
19.1
81.2
73.8
77.0
63.4
93.6
74.0
68.4
72.7
DefRee
74.6
68.6
81.2
83.8
63.6
82.1
72.9
86.9
72.7
09.4
93.4
51.8
89.7
72.0
57.2
70.55
71.7
RS
76.5
69.7
79.1
85.9
64.9
83.8
68.4
82.8
79.4
70.7
94.5
58.9
91.8
72.0
53.4
70.3
75,0
Aucil zarn
76.2
68.9
78.3
83.5
64.9
83.4
69.7
87.4
80.7
68.3
94.6
53.2
92,1
73.7
61.6
72.4
74.6
approach, assigning soft label vector to each point. We then train the primary network on the main task and the auxiliary task of segmenting each point based on the learned labels.
We evaluated the above approach in sert-negmentttion task using the ShapeNet part dataset Yi et al. 2016). This dataset contains 16,881 3D shapes from 16 object categories (including Airplane, Bag, Lamp), annotated with a total of 50 parts (at most parts per object). The main task is to predict a part label for each point. We follow the official train/val/test split scheme in (Chang et al.,
2015). We also follow the standard experimental setup in the literature, which assumes known object category labels during segmentation of a shape (see e.g., (Qi et al. 2017 Wang et al. 2019)). During training we uniformly sample 1024 points from each shape and we ignore points normal. During evaluation use all points of shape. For all methods (ours and baselines)
used the DGCNN architecture (Wang et al. 2019) as the backbone feature extractor and for part segmentation. We evaluated performance using Interssnttrnocttonn over Union (IoU) following (Q et al. 2017).
We compared AuxiLearn with the following baselines: (1) Single Task Learning (STL): Training with the main task only.
(2) DefRec: An auxiliary task of reconstructing a shape with a deformed
region (Achituve et al. 2020). (3) Reconstructing Spaces (RS): An auxiliary task of reconstructing shape from a shufiled version of (Sauder and Sievers 2019). and (4) Denoising Auto-encoder (DAE): An auxiliary task of reconstructing a point-cloud perturbed with an iid noise from (0, 0.01).
We performed hyper-parameter search over the primary network learning rate in {le 3, 1e 4}. weight decay in (5e - 5, Le 5} and weight ratio between the main and auxiliary task of {1 1. 1 0.25). We trained each method for 150 epochs, used the Adam optimizer with cosine scheduler. We applied early stopping based on the mean IoU of the validation set. We ran each configuration with different seeds and report the average mean IOU along with the SEM. We used the segmentation network proposed in (Wang et al. 2019) with an exception that the network wasn't 't supplied with the object label as input.
For AuxiLeam, we used smaller version of PointNet (Qi et al. 2017) the auxiliary network without input and feature transform layers. We selected PointNet because its model complexity is light and therefore is good fit in our case. We learned different auxiliary task per each object category (with 6 classes per category) since it showed better results. We performed hyper-parameter search over the auxiliary network learning rate in {le 2, 1c 3). weight decay in {5e 3, 5e 4). Two training samples from each class were allocated for the auxiliary set.
Table 7 shows the mean IOU per category when training with only 30 segmented point-clouds per object category (total of 480). As can be seen, Auxileearn performance is close to RS (Sauder and Sievers 2019) and improve upon other baselines. This shows that in this case, our method generates useful auxiliary tasks that has shown similar or better gain than those designed by humans.
1. LEARNING AN AUXILIARY CLASSIFIER
In Section 5.4 we show how AuxiLeamm learns a novel auxiliary to improve upon baseline methods. For the fine-grained classification experiments, we use only 30 samples per class. Here we also compare AuxiLearn with the bascline methods when there are only 15 images per class. Table g shows that AuxiLearn is superior baseline methods in this setup as well, even though it requires to allocate some samples from the training data to the auxiliary set.
To further examine the effect of learning novel auxiliary task with varying train set size, we provide here additional experiments on the CIFARIO 10 dataset. We evaluate the methods with of 10%, 15% and 100% of training examples. The results presented in Table 9 As expected, learning with
19

---

Published as a conference paper at ICLR 2021
Table 9: CIFARIO test results accuracy averaged over three runs (+SEM).
Table 8: Learning auxiliary task. Test accuracy averaged over three rums (#.SEM)- 15 shot
CIFARIO
CUB
Pet
STI.
22.6 0.2
13.6 + 0.7
100%
MAXL-F
2.7
14.1 0.1
MAXL
0.8
11.2 0.2
STL MAXL
72.63 + 2.14
80.30 + 0.09
93.36 0.05
75.85 1 0.32
81.37 1 0.26
93.49 0.02
AuxiLecam
0.7
18.0= 0.9
AuxiLearn
76.75 0.08 81.42 0.30
93.54 0.05
auxiliaries is mostly helpful in the low data regime. Nonetheless, AuxiLearn improves over single task learning and MAXL for all training set sizes.
D THEORETICAL CONSIDERATIONS
In this section, we discuss the theoretical limitations of AuxiLearn. First, we discuss the smoothness of our loss criterion while learning to combine losses using DNNs. Next, we present limitations that may arise from utilizing the IFT and their resolution. Finally, we discuss the approximations made for achieving an efficient optimization procedure.
Smoothness of the loss criterion. When learning to combine losses as described in Section 3.2 one must take into consideration the smoothness of the learn loss criterion as a function of W. This use
limits, at least in theory, the design choice of the auxiliary network. In our experiments smooth activation functions, namely Softplus, to ensure the existence of @ Nonetheless, using non-smooth activation (e.g. ReLU) results with piecewise smooth loss function hence might work well in practice.
Assumptions for IFT. One assumption for applying the IFT as described Section 3.4 that CT is continuously differentiable w.r.t to the auxiliary and primary parameters. This assump- tion limits the design choice of both the auxiliary, and the primary networks. For instance, must utilize only smooth activation functions. However, many non-smooth components can be replaced with smooth counterparts. For example, ReLU can be replaced with Softplus, (eLU(x) lim In (1 + cxp(ax))/ox and the beneficial effects of Noaiaalaimaiiiioionnnn can be captured with Weight-Norrmaizzttio as argued in (Slimaans and Kingma 2016).
For the setup of learning combine losses, Normalization, however for the learning a novel auxiliary setup, we share architecture between primary and auxiliary network (e.g. ResNet18). While using non-smooth components may, in theory, cause issues, we show empirically through extensive experiment that AuxiLean performs well in practice, and its optimization is stable. Furthermore, we note that while ReLUs are non-smooth, they are piecewise smooth, hence the set of non-smoothness points is pero-measure set.
the above substitutes, namely Softplus and Weight
Approximations. Our optimization procedure relies on several approximations to efficiently solve complex bi-level optimization. This trade-off between computation efficiency and accurate approxi- mation can be controlled by (i) The number of Neumann series components, and; (ii) The number of optimization steps between auxiliary parameters update. While cannot guarantee that the bi-level optimization process converges, empirically we observe a stable optimization process.
Our work builds on previous studies in the field of hyperparameter optimization (Lorraine al. 2020 Pedregosa (016). Lorraine et al. (2020) provide an error analysis for both approximations, in setup for which the exact Hessian can be evaluated in closed form. We refer the readers Pedregosal (2016) for theoretical analysis and results regarding the second approximation (i.c. sub-optimally of the inner optimization problem in Eq. 2)
20


Reviews:
[{"review_id": "rGNYyRYBaYD", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes AuxiLearn, a framework that can be used to combine losses from multiple auxiliary tasks (if present) into a single combined, loss function that does not require expensive grid search over possible linear combination. It uses an implicit differentiation-based approach to train a (deep) non-linear network that weighs the various auxiliary losses to optimize the generalization capabilities of the network. In the absence of such pre-defined tasks, a variation of the approach, using teacher-student networks, helps create relevant tasks to improve the performance of the network. Experiments across tasks such as classification (both few-shot and with limited labels) and segmentation show that the approach helps improve the performance of the model on the main task.\n\nComments:\n- There are some issues with notations and the overall presentation that make it a little hard to follow. For example, in section 5.1, why are there different weights/parameters (w* and \\tilde{w}) for the helpful and harmful tasks? Should the variation not just be w.r.t epsilon? Similarly, in Section 4.1, while the analysis is nice, how is this used in the proposed framework? Is the Newton update monitored in the proposed framework to identify the more important auxiliary task? If so, then I think that should be the core contribution since there can exist a non-trivial number of auxiliary tasks (e.g. 312 in the CUB dataset), and identifying them would greatly reduce the complexity of training. If the Newton update is not used to \"prune\" tasks, I am not convinced that it adds value to the overall theme of the paper and the space could be used to describe the evaluation of the approach in more detail (as done in the supplementary).\n- Ablation studies are missing that could add more value to the evaluation and help highlight the key contributions. For example, what is the effect of depth on the deep auxiliary network? It is mentioned that a 5 layer network with 10 neurons each was used. Ablations on this configuration would help visualize the effect of network complexity on the aux loss weights. \n- One question that arises is whether the aux losses are scaled based on the amount of data available for each of these tasks. For example, in CUB-200 2011, although there are 312 tasks, only around 130 \"tasks\" or attributes that have a significant amount of annotations for at least 5 of the classes. Does this have any effect on the combined loss? ", "extra_scores": {}, "extra_text": {}}, {"review_id": "p77UfreS62", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a way to combine the auxiliary tasks' losses. Moreover, when the auxiliary task is unknown, a DNN is utilized to learn the auxiliary task automatically.\n\n1. For the auxiliary tasks combing cases, how to ensure the g is smooth w.r.t. the main task parameters W, i.e., does pdiv{L_T}/pdiv{W} exists? When g is DNN, some non-smooth activation function will break down the framework and make the main task parameters hard to learn.\n2. As for the IFT, one assumption or say the requirement for Eq. (3) to hold is the function div{L_T}/div{W} should be continuously differentiable w.r.t. {W, \\phi}. This will limit the design of the DNN f and g, such as we cannot use BN and non-smooth activation function (e.g., ReLU, etc.). Although the code and the automatic differentiation tools may still work in practice, it at least wrong from the theoretical perspective.\n3. Since several approximations steps are utilized (approximation W^* and inexact vector and Hessian inverse product), it is necessary and better to provide the error analysis since the convergence of the bi-level optimization is not obvious.\n4. I do not see much insight from Sec. 4.2. If d{L_A}/d{\\phi} > 0 means the auxiliary task is helpless, does it mean that I can make it helpful by assigning it a negative weight say \\phi_i? But this is in contradiction with the experimental results in which the monotonic networks tend to have a better performance.\n\nminor comments:\n1. Some statements are inconsistent with (a) in figure 1. The subfigure (a) in Fig.1 seems to show that L_T is g(\\cdot;\\phi), while the end at page 3 states 'L_T = l_mian + g'. It is confused here, since subfigure (b) shows the way of 'L_T = l_mian + g'.", "extra_scores": {}, "extra_text": {}}, {"review_id": "b_aq1LJhSKh", "reviewer": "AnonReviewer6", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper pinpoints the key issues of Auxiliary Learning: (1). how to design useful auxiliary tasks, (2) how to combine auxiliary tasks into a single coherent loss. Motived by the issues, this paper proposes a novel Auxiliary Learning frame work, named AuxiLearn. The paper is globally well organized and clearly written. \n\nPros:\n1.\tThe motivation is straightforward.\n2.\tThe paper proposes sound the technique contributions. Adopting bi-level optimization in Auxiliary Learning makes sense.\n3.\tThe theoretical analysis in this paper supports the efficiency of the method proposed in this paper.\n4.\tThe experimental results and experimental analysis in this paper is plausible.\n\nCons:\n1.\tThe efficiency of the hypergradient, which is the main shortage, should be discussed. \n2.\tHow to determine the number of iterations J in Alg. 2 is not given.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "HJ8hXa1LJf", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to make use of implicit differerntiation to improve auxiliary learning, including learning to combine the manually designed auxiliary tasks and learning new auxiliary tasks. \n\nReasons for scores: Overall the paper looks interesting and technically sound. However, strong experiments could have been done for being more convincing. \n\nOverall the paper is clearly written and technically sound.  The problem studied is significant and should be of interest to a broad range of people in the machine learning community. Some concerns are listed below. \n\nFirst, I have concerns on the learning new auxiliary tasks, which is formulated as student-teacher maner. This step sounds somewhat counter-intuitive. The initial teacher network might simply produce meaningless labels which are used to guide the student network; it begs the question that what is the intuitition behind that pushs both the teacher and student network to learn non-trivial solutions.\n\nIn general the experiments could have been stronger. Although the authors compare with other multi-task works such as using uncertainty as  a way to combine loss, comparisons with the current best results in each benchmark task is desired as well. This will help reveal where the proposed method stands in terms of accuracy. \n\nIn sec 5.4, it demonstrates the utility of learnable auxiliary in the low-data regime and only 5% of the training data are used. This seems to indicate that the method is only useful in the case of scarce training data. The authors could have reported results under different amount of training data; this is important for readers to understand the range within which the method is expected to help. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "4hTy4SxESlT", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies a variant of multi-task learning, auxiliary learning, where one main task dominates, and other tasks are used to learn a good representation. To achieve this goal, the authors propose a learning-to-learn algorithm. In particular, the auxiliary losses are represented by a vector and then transformed to a new loss term via linear or nonlinear function $h$. They also made two more contributions. First, an approach of new auxiliary task generation is proposed. Second, an implicit differentiation based optimization method is proposed to find the solution. Both theoretical analysis and empirical studies demonstrate the superiority of their proposed model.\n\nPros:\n1. The intuition makes sense. When the main task dominates, its generalization can help guide the weighting of the weights of auxiliary tasks.\n2. The theoretical analysis further supports the claim of this work.\n3. Comprehensive experiments are made to verify the effectiveness of the proposal.\n\nCons:\n1. The idea of learning new auxiliary tasks is wired and less intuitive. The learning of the whole system is still the main task loss, without any useful supervision (or self-supervision). Therefore, it is doubtful whether the learned task is meaningful, or may involve some chaos in the system. Overall, I think this part does harm to this work and the authors may consider removing it.\n2. Notations are sort of unclear. In section 3.2, $\\mathbf{l}$ is a $K+1$-dim vector, but it seems $g(\\mathbf{l};\\phi)$ only maps the $K$ auxiliary tasks' losses to a scalar. \n3. There are some typos and the authors need to polish this paper again.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel framework to develop useful auxiliary tasks and combine auxiliary tasks into a single coherent loss. The idea is good and the experiments are sufficient to verify the arguments. All the reviewers agree to accept the paper.
Decision: Accept (Poster)