Paper: GatedMTL: Learning to Share, Specialize, and Prune Representations for Multi-task Learning (ICLR-2024)

Reviewer comment (Reproducibility):
The code is not provided.

Author response:
**Open-source Code Availability:** We acknowledge the critical role of open-sourcing in advancing research and reproducibility. We have initiated the process for obtaining legal approval to open-source our code. Additionally, to facilitate understanding and reproducibility in the interim, we reported all detailed hyperparameters and have included detailed pseudo-code for GatedMTL in Appendix E.

Reviewer comment (Novelty):
The description of the proposed method in Section 3 and the overall framework in Figure 1 are confusing. If my understanding is correct, the proposed method is very similar to the existing MoE-based MTL methods. However, this paper does not discuss and compare with MoE-based MTL methods.

Author response:
**Clarification of our method:** We apologize for any confusion caused and appreciate the opportunity to clarify. The encoder in GatedMTL consists of both shared parameters $\Psi$ and task-specific parameters $\Phi_t$. The shared $\psi$ and task-specific features $\varphi_t$ are computed using their respective parameters. In Figure 1, we illustrate the architecture of a *single GatedMTL layer* within the *unified encoder*, highlighting how task-specific gates blend shared and task-specific features to form $\varphi_t^{\prime}$, serving as the input to the following task-specific layers. The purple blocks represent the features extracted from the shared branch, while the green and yellow blocks represent features extracted by task $1$ and task $t$ (we show only two task-specific branches for simplicity). For each task, a corresponding task-specific gate (e.g. yellow round block) makes a selection between using shared features and task specific features resulting in $\varphi_t^{\prime}$ shown as a blended representation with two colors. Subsequently, $\varphi_t^{\prime}$ represents the input to the next task-specific non-linear transformations with parameters $\Phi_t$. The input to the next shared layer is constructed via a linear combination of the task-specific features of all tasks, followed by a non-linear transformation parameterized by $\Psi$. There are fundamental differences between GatedMTL and MoE-based approaches which we highlight in the next point.

**Comparison to MoE-based approaches:**
A major difference between GatedMTL and MoE-based approaches is in how they operate at inference. GatedMTL is designed to solve *all tasks simultaneously* in one forward pass, a strict requirement in many real-world practical scenarios (like XR and autonomous driving applications requiring edge deployment for solving multiple tasks simultaneously). In the contrary, existing MoE-based approaches, can only solve *one task at a time*, requiring one forward pass per task. In our setting, this makes MoE approaches more expensive than conventional MTL approaches, considering the cost of the accumulated forward passes. Unlike GatedMTL which targets efficient inference for solving all tasks simultaneously, MoE-based approaches focus on improving accuracy via dynamic routing to expert modules while solving one task at a time. Therefore, MoE models are not suitable for the setup we are targeting in this work. We have made these clearer in the manuscript and highlighted the key differences in the related work section.

Reviewer comment (Experiments):
The proposed method uses the single-task weights for initialization, which means it needs to train $T$ single-task models before training the proposed method, and it is unfair to compare with the baselines which do not use the information from single-task models.

Author response:
**Using single-task weights and implications on training cost:** Our work primarily aims to enhance inference efficiency, vital for deploying multi-task learning (MTL) models in edge computing environments. In numerous real-world applications, the gains in efficient inference over time, especially when scaled across millions of edge devices, can significantly offset the initial higher training costs. While we acknowledge the importance of training efficiency and have discussed GatedMTL’s training cost limitations in our manuscript, it’s important to highlight that competing methods like MTO approaches (PCGrad [1], CAGrad [2], MGDA [3], etc.) also bear considerable training costs, requiring multiple backward passes during training. Our GatedMTL significantly surpasses these in performance, demonstrating its advantage for practical applications that demand efficient inference on the edge.

In addition, in both MTL research and practice, establishing single-task baselines is a standard practice to gauge the relative performance of MTL models. Therefore, having access to single task models is not unreasonable from a practical perspective. In addition, pre-trained single task models for many applications are abundantly available which may obviate the need to separately train STL models.

Reviewer comment (Reproducibility):
How to choose $\omega_t$ in Eq. (1)?

Author response:
**The choice of $\omega_t$:** The hyper-parameter $\omega_t$ denotes the scalarization weights. We use the weights suggested in prior work but also report numbers of uniform scalarization. For NYUD-v2, we use uniform scalarization as suggested in [5,6], and for PASCAL-Context, we similarly use the weights suggested in the paper [5,6].

Reviewer comment (Evaluation):
"for a given computational budget" in the abstract and "matching the desired target computational cost" in the second contribution. What is the "given computational budget" or "desired target computational cost"? Is it $\tau_t$ in Eq. (4)? However, $\tau_t$ represents neither parameter size nor flops. Besides, although both $\lambda_s$ and $\tau_t$ can control the trade-off between performance and computational cost, the sparsity regularization cannot be guaranteed to be optimized to $0$.

Author response:
**Target computational cost:** We agree with the reviewer and reformulated the wording of the claim in the abstract and additionally mentioned this as a limitation of our method. As correctly described by the reviewer, although both $\lambda_s$ and $\tau_t$ can control the trade-off between performance and computational cost and closely result in the desired FLOPs count, we still cannot guarantee landing on a particular target computational cost.

Reviewer comment (Evaluation):
Why not report and compare the parameter size? It is very important in multi-task learning.

Author response:
**Comparison to recent baselines and parameter size:**
In response to the reviewers feedback regarding the selection of baselines, we have extensively expanded our experiments to include the suggested state-of-the-art methods, i.e. PCGrad [1], CAGrad [2], MGDA [3], and RDW [4]. The results are presented in the table below. As can be seen, GatedMTL substantially outperforms all competing baselines across both $\Delta_{MTL}$ and the newly added Mean Rank metric (suggested by R1). We additionally report the parameter counts of the models as suggested by the reviewer. Our method compares favorably against Cross-stitch and MTAN in terms of number of parameters and performance. Please note that as discussed above, our comparisons do not include MoE-based approaches as they are not suitable for our MTL setup for solving all tasks simultaneously.

[Tables omitted here but present in original R2 text]

Reviewer comment (Experiments):
Many recent or important baselines are missing. For example, MoE-based MTL methods like [1, 2, 3, 4, 5] and MTO approaches like [6, 7, 8].

Author response:
**Comparison to recent baselines and parameter size:**
In response to the reviewers feedback regarding the selection of baselines, we have extensively expanded our experiments to include the suggested state-of-the-art methods, i.e. PCGrad [1], CAGrad [2], MGDA [3], and RDW [4]. The results are presented in the table below. As can be seen, GatedMTL substantially outperforms all competing baselines across both $\Delta_{MTL}$ and the newly added Mean Rank metric (suggested by R1). We additionally report the parameter counts of the models as suggested by the reviewer. Our method compares favorably against Cross-stitch and MTAN in terms of number of parameters and performance. Please note that as discussed above, our comparisons do not include MoE-based approaches as they are not suitable for our MTL setup for solving all tasks simultaneously.

[Tables omitted here but present in original R2 text]

Reviewer comment (Writing):
$\odot$ in Eqs. (2), (3), (6), (7), and (8) is not defined.

Author response:
4) Thanks for pointing this out. We will fix it in the manuscript.

Reviewer comment (Theory):
The gating module to balance task-specific features and the shared features in the decoder seems a bit more reasonable. Since the encoder is responsible for encoding out the shared features across all tasks, it doesn't seem to make sense to split out the task-specific features in the encoder.

Author response:
**Where to gate, encoder or decoder?** We thank the reviewer for bringing up this interesting discussion. We absolutely agree that in typical MTL setups, the encoder is responsible for encoding out the *shared features across all tasks*. As highlighted in the above discussion, task interference is inherently the problem that arises with sharing representations across several tasks and the encoder is a natural choice because it is fully shared among all tasks. We would like to provide three additional major reasons for why we chose to resolve task interference within the encoder rather than the decoder:

1. **Computational efficiency and specialization of decoders:** Our design is grounded in the conventional MTL architecture, which typically comprises a large shared backbone and multiple task-specific decoders, as depicted in Figure 1 (left). In such architectures, the majority of computational resources are consumed by the encoder (backbone). For instance, in the context of NYUD-v2 with three tasks, the cumulative computational load of the HRNet18 decoders constitutes merely 17\% of the total. Given that the decoders are designed to be highly specialized and independent for each task to optimize performance, it is both computationally efficient and logical to address task interference predominantly in the backbone.
 
2. **Diversity in task decoding requirements:** In MTL, particularly in complex applications like autonomous driving, tasks can vary greatly in their nature, such as combining dense prediction tasks, object detection tasks, and classification tasks such as weather prediction. In such scenarios, decoders might possess distinctly different architectures or layers. Forcing feature sharing at the decoder level could be detrimental, as it disregards the unique requirements and structures of each task’s decoder.
 
3. **Encoder feature quality and decoder success:** The quality of features extracted by the encoder is pivotal for the success of the specialized decoders. Significant task interference at the encoder stage can lead to degraded feature quality, resulting in irreversible information loss and sub-optimal input for the decoders. Ensuring high-quality feature extraction by the encoder is therefore crucial for the effective functioning of the task-specific decoders.

Finally, our empirical evaluations strongly demonstrate the effectiveness of the encoder-based approach to mitigating task-interference. We leave the investigation of decoder-based approaches to future work and discuss that in the manuscript.

Reviewer comment (Novelty):
The proposed gating mechanism seems similar to a simplified variant of smooth Gating in DSelect-k [R1]. It is not possible to observe from Eqs. 2 and 6 that there is a point of novelty in the gating of this paper.

Author response:
**Distinction from DSelect-k:** Firstly, it is important to clarify that we do not claim innovation in the design of our gating modules. Our gates are simple binary units that make a binary decision between task-specific features and shared features, and they are trained using the well-established straight-through estimation (STE) [2] by Bengio, et al. This is fundamentally different from the DSelect-k mechanism, which is tailored for MoE architectures to select a limited number of experts (at most $k$ out of $n$) in a continuous and differentiable manner. Our method does not involve a Top-K selection process, thereby making the comparison to DSelect-k less applicable.

Furthermore, there are significant differences between GatedMTL and MoE-based methods. MoE architectures handle one task at a time, whereas our GatedMTL is designed to solve all tasks simultaneously. This simultaneous task-solving capability is crucial for a wide range of real-world applications, setting GatedMTL apart from MoE-based approaches.

Reviewer comment (Experiments):
Why are the results for Auto-λ not shown in Tables 3 and 4?

Author response:
**Missing results for Auto-$\lambda$ in Tables 3 and 4:** Despite extensive hyperparameter search for Auto-$\lambda$, the experiments using Transformer backbones diverged with exploding losses. We also observed similar convergence issues when using Auto-$\lambda$ on a larger number of tasks such as Pascal-Context. Unfortunately, because of divergent training, the results were not reasonable to be reported.

Reviewer comment (Experiments):
The gating module to balance task-specific features and the shared features in the decoder seems a bit more reasonable. Since the encoder is responsible for encoding out the shared features across all tasks, it doesn't seem to make sense to split out the task-specific features in the encoder. Have the authors considered this?

Author response:
**Where to gate, encoder or decoder?** We thank the reviewer for bringing up this interesting discussion. We absolutely agree that in typical MTL setups, the encoder is responsible for encoding out the *shared features across all tasks*. As highlighted in the above discussion, task interference is inherently the problem that arises with sharing representations across several tasks and the encoder is a natural choice because it is fully shared among all tasks. We would like to provide three additional major reasons for why we chose to resolve task interference within the encoder rather than the decoder:

1. **Computational efficiency and specialization of decoders:** Our design is grounded in the conventional MTL architecture, which typically comprises a large shared backbone and multiple task-specific decoders, as depicted in Figure 1 (left). In such architectures, the majority of computational resources are consumed by the encoder (backbone). For instance, in the context of NYUD-v2 with three tasks, the cumulative computational load of the HRNet18 decoders constitutes merely 17\% of the total. Given that the decoders are designed to be highly specialized and independent for each task to optimize performance, it is both computationally efficient and logical to address task interference predominantly in the backbone.
 
2. **Diversity in task decoding requirements:** In MTL, particularly in complex applications like autonomous driving, tasks can vary greatly in their nature, such as combining dense prediction tasks, object detection tasks, and classification tasks such as weather prediction. In such scenarios, decoders might possess distinctly different architectures or layers. Forcing feature sharing at the decoder level could be detrimental, as it disregards the unique requirements and structures of each task’s decoder.
 
3. **Encoder feature quality and decoder success:** The quality of features extracted by the encoder is pivotal for the success of the specialized decoders. Significant task interference at the encoder stage can lead to degraded feature quality, resulting in irreversible information loss and sub-optimal input for the decoders. Ensuring high-quality feature extraction by the encoder is therefore crucial for the effective functioning of the task-specific decoders.

Finally, our empirical evaluations strongly demonstrate the effectiveness of the encoder-based approach to mitigating task-interference. We leave the investigation of decoder-based approaches to future work and discuss that in the manuscript.

Reviewer comment (Writing):
Minor error: $\Delta_{MTL}$ and $\Delta$ denote the same metric. The authors are encouraged to keep them consistent.

Author response:
**$\Delta_{MTL}$ and $\Delta$ notations:** We thank the reviewer for pointing this out. We fixed all the inconsistencies.

Reviewer comment (Reproducibility):
No source code is provided. Although the experimental setup is detailed and the results are extensive, it is still necessary to provide the code for reference and reproducibility checking.

Author response:
**Open-source Code Availability:** We acknowledge the critical role of open-sourcing in advancing research and reproducibility. We have initiated the process for obtaining legal approval to open-source our code. Additionally, to facilitate understanding and reproducibility in the interim, we reported all detailed hyperparameters and have included detailed pseudo-code for GatedMTL in Appendix E.

Reviewer comment (Theory):
Since a shared feature branch acts like a memory bank where task-specific features can communicate, a task-specific gate still learns features from other tasks, which can cause task interference.

Author response:
**Interference impact on gating:** This is indeed a very interesting point raised by the reviewer. While our gating mechanism does indeed select between the shared and task-specific features, potentially influenced by task interference, it also serves as a mechanism to mitigate the negative impact of task-interference by avoiding the selection of representations with severe task interference issue. This is also supported by the results in Table 6 (first row) showing that in the absence of a sparsity objective, the gates inherently and strongly favor selecting task-specific over shared features. Additionally, our empirical findings show that in practice even with strong sparsity regularization, the gates can cope with task-interference by selecting the least interfering shared features, suggesting that task-interference at this meta-level might be less of a concern for the gates.

Reviewer comment (Evaluation):
The reported performance in each table is based on a single run. The standard deviation based on multiple random runs is highly encouraged to be provided.

Author response:
**Multiple runs with random seeds:** All reported results for the baselines in our study were averaged across *three random seeds*. We are sorry for the oversight of not mentioning this in the original manuscript and thank the reviewer for notifying us. We have now included this information in Section 4.1. We additionally report the standard deviation for the runs in the appendix. For GatedMTL runs, we inherently had numerous results for the Pareto optimal performance vs. FLOPs curve. But based on the reviewer's suggestion we are providing the GatedMTL results for three random seeds as well. Please note that the experiments are running and in the Table below we provide the mean+std for the completed runs and will update the final manuscript with the full set of results. Note that in the Table below we additionally include results for 4 new baselines suggested by wdHJ and 7nfe.

|Model|Semseg|Depth|Normals|$\Delta_\text{MTL}$(\%)|Flops(G)|MR|
|-|-|-|-|-|-|-|
|STL|41.70|0.582|**18.89**|0$\pm$0.12|65.1|8.0|
|MTL(Uni.)|41.83|0.582|22.84|-6.86$\pm$0.76|24.5|11.0|
|DWA|41.86|0.580|22.61|-6.29$\pm$0.95|24.5|8.7|
|Uncertainty|41.49|0.575|22.27|-5.73$\pm$0.35|24.5|8.3|
|Auto-$\lambda$|42.71|0.577|22.87|-5.92$\pm$0.47|24.5|8.0|
|**RDW**|42.10|0.593|23.29|-8.09$\pm$1.11|24.5|11.7|
|**PCGrad**|41.75|0.581|22.73|-6.70$\pm$0.99|24.5|10.3|
|**CAGrad**|42.31|0.580|22.79|-6.28$\pm$0.90|24.5|8.7|
|**MGDA**|41.23|0.625|21.07|-6.68$\pm$0.67|24.5|11.3|
|GatedMTL|**43.58**|**0.559**|19.32|**+2.06$\pm$0.13**|43.2|**1.3**|
|GatedMTL|42.95|0.562|19.73|+0.68$\pm$0.09|38.3|2.3|
|GatedMTL|42.36|0.564|20.04|-0.55$\pm$0.17|36.0|4.0|
|GatedMTL|42.73|0.575|21.01|-2.55$\pm$0.11|33.1|4.0|
|GatedMTL|42.35|0.575|21.70|-4.07$\pm$0.38|29.2|5.7|

|Model|Semseg|Depth|Normals|$\Delta_\text{MTL}$(\%)|Flops(G)|MR|
|-|-|-|-|-|-|-|
|STL|43.20|0.599|**19.42**|0$\pm$0.11|1149|9.0|
|MTL(Uni.)|43.39|0.586|21.70|-3.04$\pm$0.79|683|9.7|
|DWA|43.60|0.593|21.64|-3.16$\pm$0.39|683|9.7|
|Uncertainty|43.47|0.594|21.42|-2.95$\pm$0.40|683|10.0|
|Auto-$\lambda$|43.57|0.588|21.75|-3.10$\pm$0.39|683|10.0|
|**RDW**|43.49|0.587|21.54|-2.74$\pm$0.09|683|8.3|
|**PCGrad**|43.74|0.588|21.55|-2.66$\pm$0.15|683|7.3|
|**CAGrad**|43.57|0.583|21.55|-2.49$\pm$0.11|683|7.0|
|**MGDA**|42.56|0.586|21.76|-3.83$\pm$0.17|683|11.3|
|MTAN|**44.92**|0.585|21.14|-0.84$\pm$0.32|683|4.0|
|Cross-stitch|44.19|0.577|19.62|+1.66$\pm$0.09|1151|2.7|
|GatedMTL|44.38|**0.576**|19.50|**+2.04$\pm$0.07**|916|**1.7**|
|GatedMTL|43.63|0.577|19.66|+1.16$\pm$0.10|892|3.7|
|GatedMTL|43.05|0.589|19.95|-0.50$\pm$0.05|794|9.7|

Reviewer comment (Experiments):
The biggest problem I have with the manuscript is that it does not discuss or show the limitations of the proposed approach _at all_, which can really easily mislead the readers (and thus, the reviewers). For example, to my understanding, the proposed approach at training time is $T$ individual models that are trained altogether. However, this is a _huge_ setback as it scales poorly with $T$ in memory and time (for example, the usual CelebA setting in MTL is to do a 40-task binary classification, but the authors reduce it to 3 tasks). The authors should discuss it in the manuscript and show training times for each of the experiments.

Author response:
**Training time:** The primary focus of GatedMTL is on inference-time efficiency, crucial in real-world applications. An example use case is XR applications requiring edge deployment for solving multiple tasks concurrently. We acknowledge that GatedMTL is not optimized for training time efficiency and we have added a limitation section in the paper, but we do not concur that this poses a significant drawback for practitioners. The training cost of GatedMTL is comparable, if not more efficient, than most existing MTO approaches. During training, many MTO approaches require $T$ backward passes and additional computations such as computing gradient conflicts/storing per task gradients simultaneously. GatedMTL, however, allows for sparsification into both forward and backward passes, meaning parameters that are not chosen do not get gradients and hence enables significant memory and compute saving (depending on the sparsity level). At inference, where GatedMTL is primarily designed for, we outperform all competing MTO baselines offering a significantly better performance-compute trade-off.

**Scaling to an extremely high number of tasks:** We agree that GatedMTL may not be the optimal choice for scenarios requiring simultaneous training/inference for an exceptionally high number of tasks, such as 40. However, we observe that this limitation is not unique to GatedMTL but is a common challenge across the MTL field. Our framework, therefore, remains relevant and effective within the practical range of multi-task learning scenarios that the vast majority of the related literature is focusing on.

Reviewer comment (Evaluation):
I find $\Delta_{\text{MTL}}$ a brittle metric, as it is sensitive to low-magnitude metrics and task metrics are not comparable. I would add a more robust metric like the rank mean (see, e.g., [1]).

Author response:
**Mean Rank Metric:** We thank the reviewer for this suggestion. We added the rank mean measure to all tables in the paper and updated the manuscript. This new metric further strengthens the merits of GatedMTL, consistently showing strong performance across both the $\Delta_{MTL}$ and rank mean measures.

Reviewer comment (Experiments):
The chosen baselines are inconsistent across experiments and mostly outdated. From the MTO side, DWA and Uncertainty are quite old and weak in comparison with other methods like PCGrad, CAGrad, or NashMTL. From the side of adaptive architectures, more modern approaches like Adashare should be included.

Author response:
**Baselines:** In response to the reviewers feedback regarding the selection of baselines, we have extensively expanded our experiments to include the suggested state-of-the-art methods, i.e. PCGrad [1], CAGrad [2], RDW [3], and MGDA [4]. The results are presented in the table below. As can be seen, GatedMTL substantially outperforms all MTO baselines across all the metrics. We also integrated the implementation of NashMTL into our MTL baselines, however, despite extensive hyperparameter search, the experiments diverged which are not reasonable to be reported. 

Updated performance comparison on NYUD-v2 using HRNet-18 backbone. The new baselines are shown in bold.
Model|Semseg|Depth|Normals|$\Delta_\text{MTL}$(\%)|Flop(G)|MR
-|-|-|-|-|-|-
STL|41.70|0.582|**18.89**|0|65.1|8.0
MTL(Uni.)|41.83|0.582|22.84|-6.86|24.5|11.0
DWA|41.86|0.580|22.61|-6.29|24.5|8.7
Uncertainty|41.49|0.575|22.27|-5.73|24.5|8.3
Auto-$\lambda$|42.71|0.577|22.87|-5.92|24.5|7.7
**RDW**|42.10|0.593|23.29|-8.09|24.5|11.7
**PCGrad**|41.75|0.581|22.73|-6.70|24.5|10.3
**MGDA**|41.23|0.625|21.07|-6.68|24.5|11.3
**CAGrad**|42.31|0.580|22.79|-6.28|24.5|8.7
GatedMTL|**43.58**|**0.557**|19.32|**+2.18**|43.2|**1.3**
GatedMTL|42.93|0.5613|19.72|+0.70|38.4|2.3
GatedMTL|42.43|0.5671|20.01|-0.54|36.0|4.0
GatedMTL|42.85|0.5778|21.01|-2.58|33.0|5.0
GatedMTL|42.35|0.5755|21.70|-4.06|29.1|6.0

----------
Updated performance comparison on NYUD-v2 using ResNet-50 backbone.

Model|Semseg|Depth|Normals|$\Delta_\text{MTL}$(\%)|Flops(G)|MR
-|-|-|-|-|-|-
STL|43.20|0.599|**19.42**|0|1149|9.0
MTL(Uni.)|43.39|0.586|21.70|-3.04|683|9.7
DWA|43.60|0.593|21.64|-3.16|683|9.7
Uncertainty|43.47|0.594|21.42|-2.95|683|10.0
Auto-$\lambda$|43.57|0.588|21.75|-3.10|683|10.0
**MGDA**|42.56|0.586|21.76|-3.83|683|11.3
**RDW**|43.49|0.587|21.54|-2.74|683|8.3
**PCGrad**|43.74|0.588|21.55|-2.66|683|7.3
**CAGrad**|43.57|0.583|21.55|-2.49|683|7.0
MTAN|**44.92**|0.585|21.14|-0.84|683|4.0
Cross-stitch|44.19|0.577|19.62|+1.66|1151|2.3
GatedMTL|44.15|**0.573**|19.49|**+2.08**|916|**2.0**
GatedMTL|43.70|0.578|19.65|+1.16|892|4.0
GatedMTL|42.99|0.589|19.93|-0.48|798|9.7

Please note that the MTL setting we are targeting, solves multiple tasks *simultaneously in one forward pass*. In contrast, Adashare and existing MoE-based approaches, can only solve one task at a time, requiring one forward pass per task.
Therefore, these models are not suitable for the setup we are targeting in this work.

Reviewer comment (Theory):
Do you use a different Lagrange multiplier for each task when using L1 regularization? Otherwise, I don't see how it is comparable to the hinge loss in Eq. 4.

Author response:
**Q) Different Lagrange multipliers for L1 regularization:**
Yes, we indeed experimented with varying Lagrange multipliers for the L1 sparsity term for each task. However, we observed that the Lagrangian multipliers did not yield similar training dynamics for the gates as compared to the hinge loss. We hypothesize that, without a hinge target rate, the sparsification endlessly continues throughout the training making it difficult for the model to converge and the task-specific and shared representations to co-adapt.