Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a method to transfer knowledge from multiple diverse models. First, it learns pathway routes that activate a different set of models depending on input data. Later, it aggregates the outputs to generate task-specific predictions. The authors propose an exploration loss to promote the utilization of all the models. Finally, the paper discusses the efficient fine-tuning of models on a subset of data that activates them. The paper claims state-of-the-art performance for model hub transfer learning on computer vision and reinforcement learning tasks.
strengths_and_weaknesses: Strengths:
- The paper is well-organized and clearly written.
- The paper's primary idea to facilitate knowledge transfer from multiple pre-trained models is important.
- The paper proposes a novel approach for transfer learning from multiple models. Hub-Pathway learns to identify top-k models best suited for input datum. The aggregator network generates a final prediction based on the output of k models and path weights. 
- The authors propose an interesting exploration strategy to improve the transferability and exploitation of activated models via selective fine-tuning. 
- The paper highlights the high efficiency compared to ensembles since not all models are active for both training/inference phases.

Weaknesses:
- The paper makes a broad claim on the effectiveness of the proposed method on heterogeneous model architectures. However, it's unclear how it could be used for autoregressive models where subsequent output tokes depend on the previous ones.
- The paper discusses replacing output head layers with task-specific layers (#119-#120). It is unclear how aggregator network could be set up for heterogeneous models with different output dimensions.
questions: - Please refer to the weaknesses section. A bit more clarification on those aspects would be appreciated.
- How Hub-Pathway fares against data drift compared to pre-trained models? Likely, some pre-trained models are more robust against data drift; however, Hub-Pathway fails to activate them?
limitations: There could be limitations on what all heterogeneous model architectures Hub-Pathway could support. Please refer to the weaknesses section.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors propose a transfer-learning method called Hub-Pathway. to leverage a library of pre-trained models instead of a single pre-trained model. The proposed Hub-Pathway utilized the idea of data-dependent transfer learning to find the best transfer learning path through multiple pre-trained models for each input. The output of each pre-trained model is then aggregated to produce the final prediction. The generator for generating the pathway is trained with softmax policy with additional noise to encourage exploration. The authors also propose exploitation loss to better leverage the activated model. Extensive experiments on multiple tasks demonstrate the benefits of the proposed approach. 
strengths_and_weaknesses: Strengths:

1. The paper is generally well-written and the organization is clear. 

2. The idea of using multiple pre-trained models for transfer learning is interesting and the authors also propose a well-designed method for leveraging multiple models.

3. The experiments are extensive which show the effectiveness of the proposed method. 


Weaknesses:
1. The additional storage is an overhead for using multiple pre-trained models for transfer learning. Also, the introduction of the pathway generator is another additional cost. 

2. The key experimental details should be included in the main text rather than in the supplementary.  The choices of the hyperparameters are also not clearly specified. 

3. Several related papers on dynamic computation and transfer learning are not cited.

4. Although the authors stated that the experiments are repeated for 3 times, no error bar is reported. 

###########################

Post-rebuttal:

Thanks for the rebuttal. Most of my concerns are addressed in the rebuttal. I encouraged the authors to include the additional results in the final draft to improve the significance of the paper. I increase the score from 4 to 5. 
questions: 1. In Table 4, the ablation studies are only considered on COCO and Cars, I am wondering if the observation is consistent across all the test datasets. 

2. What's the total inference time (pre-trained models + pathway generator) for a single image? 

3. What's the effect of $\lambda$ and why $\lambda$ is 0.3?

4. Would the convergence of the model become slower because of the introduction of pathway finding?
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper tackles the problem of transfer learning from a zoo / hub of pre-trained models unto a specific end-task. 
They treat the problem as data-point dependent and route each data-point through an adaptive subset of the available pre-trained models. 
They also fine-tune the pre-trained models on the data-sets that activate them. 
This paper compares to some relevant baselines and shows improvement over them.
strengths_and_weaknesses: **Strengths**
1. Method works for heterogeneous architectures which is not the case for previous approaches 
2. Improved performance over reasonable baselines
3. Method is relatively easy to understand


**Weaknesses**

1. Though the authors claim that the inference and training time cost is better for hub-transfer over ensembling, I am not 100% convinced. Note that the hub-transfer approach introduces significant overhead in terms of shuffling models in and out of memory as data-points are adaptively assigned.  Also - with the exploration bonus (and also early in training), it stands to reason that a batch of data could activate all models (though each data-point activates an individual subset but the union could be the whole hub) which presents a computational / memory hurdle to handle efficiently. Can the authors provide more details about how the approaches were implemented and benchmarked for Table 5 ?
2. Method performance  (in the case of out-of-domain transfer)  is on par with the ensembling approach which may be simpler to implement 
3.  Seems technically complex to implement - would involve swapping models in and out of memory unless one has access to a large memory budget.
4. No error bars on results even though experiments were run for 3 seeds



**Update after rebuttal**
Score updated from 4->6 after rebuttal discussion
questions: 1. Could you provide error bars on your experiments since you have 3 runs per experiment
2. Are all the inputs similarly structured ? Do all the models expect the same type of input structure - no discussion of adapting model input to task structure.
3. Why add to the computational cost by using a randomized sub-network G_n -> why not just do something like temperature annealing ?
4. How about a domain expert baseline ? Assuming oracle access to the best single model apriori - can you update the tables with what the performance would be ? This is the baseline of an expert who knows how to pick the best single model for each new task. 
5. For the RL experiments (Fig 2), the author say "We observe that the proposed Hub-Pathway outperforms these state-of-the-art methods" - however the errorbars with the ZooTuning experiments clearly overlap. Am I missing something ? 
limitations: 1. The method introduces extra hyper-parameters over methods like ensembling
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This study proposes Hub-Pathway, a method for maximizing the knowledge gained from multiple pre-trained deep neural network models in a transfer learning problem setting. The basic idea of the Hub-pathway is to select the best subset of models for each datum from the model hub to make predictions. There are two main challenges of this approach: (i) how to select the best path for each data set and (ii) how to aggregate knowledge from multiple models. For the first challenge, the paper introduces a gating function with randomness to achieve a variety of path options for each datum. For the second challenge, the paper develops a mechanism that combines the aforementioned gate function with a function for knowledge aggregation to output a prediction result for the hub as a whole. In the experiments, the paper compared the proposed method with prior methods in the tasks of classification, facial landmark detection, and reinforcement learning, and showed that the proposed method achieves superior performances by dynamically routing paths on model hubs.
strengths_and_weaknesses: ### Strength
* Research questions and motivations are clear.
* The proposed method is quite simple so it is easy to reimplement.
* The proposed method can handle heterogeneous sets of models, unlike prior methods.
* Comparison with prior methods in a variety of experimental settings.

### Weakness
* Insufficient explanation and evaluation of the need to change paths for each datum.
* A little lack of evaluation in a homogenous setting.
* Larger memory size and execution time required during inference time.

The paper is well-structured and clearly describes the research questions and motivations. The proposed method provides new insights into this research area by achieving knowledge aggregation from heterogeneous model hubs, which has been difficult before, and by showing better experimental results than models in homogeneous settings. On the other hand, comparisons with state-of-the-art methods and verification of the validity of switching paths on a datum-by-datum basis are insufficient (these will be discussed in Questions). In addition, the proposed method arises new challenges in terms of memory size and execution time during inference. However, these issues might not be considered serious enough flaws to damage the main argument of the paper. 
questions: ### Q1. Do we need to change the path for each datum?
One of the key ideas in this paper is to efficiently extract knowledge from pre-trained models by switching paths for each datum. We have a concern on this point. That is, wouldn't it be sufficient to just select a model for each dataset? We could not validate the proposed method of switching on each datum because the evaluation in the current paper analyzed only the performance for each target dataset. Rather than holding a large number of models and training them simultaneously, a more realistic setting in terms of inference cost would be to test a set of fine-tuned models on the target dataset and select Top-K models by validation scores to ensemble. In order to argue for the validity of switching on each datum, we believe that a comparison with such a method of switching models on each target dataset is necessary. This evaluation can easily be done using the single model that appeared in Tables 1 and 2.

### Q2. How is the performance of the proposed method compared to ModelSoups on homogeneous?
This is related to Q1. In the homogeneous setting, a recent paper presented a method to aggregate the knowledge in multiple models called ModelSoups[1], which averages weights of fine-tuned models and applies the averaged weights to a single model. ModelSoups also adopts a Top-K selection technique as well as hub-pathway and thus it can be considered a model aggregating knowledge for each target dataset level discussed above. Obviously, we recognize that ModelSoups is not a perfect competitor to hub-pathway since it cannot be applied to heterogeneous settings, but we consider a comparison with ModelSoups would be helpful for investigating the characteristics of hub-pathway.

```
[1] Wortsman, Mitchell, et al. "Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time." International Conference on Machine Learning (2022).
```

### Q3. About experimental settings of the image classification task
In the image classification experiments, the number of training epochs for hub-pathway and the other competitors did not seem to be described. Does hub-pathway require more training time than other methods?

limitations: Nothing to report.
review_title: null

Meta-review: metareview: The submission introduces an approach called Hub-Pathway to leverage a diverse collection of pre-trained models for transfer learning. Hub-Pathway trains a pathway generator network to route examples to various models in a data-dependent manner and aggregates the outputs to produce task-specific predictions. Noise is added to the pathway generator and its output is entropy-regularized to encourage exploration, and the activated models are also individually trained on the target loss to encourage exploitation. The approach is evaluated on several image classification, facial landmark detection, and reinforcement learning tasks.

Reviewers noted the paper's clarity and writing quality and found the empirical evaluation extensive and convincing. On the other hand, they expressed doubts regarding Hub-Pathway's computational and memory complexity at training and inference time, in particular in comparison to the alternative of using an ensemble of models.

The authors responded by citing existing results in the submission and providing new results in the revised appendix showing that Hub-Pathway does in fact have lower computational complexity than an ensemble. They also argued through additional results that the forward propagation through multiple models (as opposed to holding the parameters of multiple models in memory) is the main memory bottleneck (which model ensembles also face), and that Hub-Pathway does better in that regard than model ensembles due to the pathway activation mechanism.

Overall the authors' response was satisfying to the reviewers, and their consensus is that the submission should be accepted. I therefore recommend acceptance.
confidence: Certain
Decision: Accept