Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: In this paper, the authors develop DynaTune which achieves faster convergence speed to optimize a DNN model when compared to the state-of-the-art DL compiler, AutoTVM. The key idea is a time-slot-based scheduling method based on UCB-type multi-armed bandit policy. At each time, the scheduler chooses an action to maximize the latency reduction. In practice, A Bayesian belief model via MCMC is used to capture current knowledge of the optimization results to predict future performance, which helps make better decisions and expedites the convergence speed. The idea of using MAB in DL compiler is very interesting. The numerical experiments also demonstrate clear advantage of the proposed DynaTune. My concerns are as follows. 

1. In the experiments, it would be more convincing to numerically compare with more advanced DL compiler, e.g., Adams et al. (2019); Chameleon in Ahn et al. (2020). The Chameleon is also a RL based approach for DL compiler.

2. The authors used $C=0.2$ when initial latency of an operator is <1ms, and $C=2$ for all the other cases. Can authors provide more justification on the choice of $C$? Some sensitivity analysis could be helpful. 

3. In the end of the first paragraph in Section 5 (line 8 on page 6), the sentence is not finished. 




review_title: Review of DynaTune

final_score: rating: 6: Marginally above acceptance threshold
review_content: ##########################################################################

Summary:

The paper proposes an algorithm to optimize the auto-tuning time for compiling neural networks.
 It dynamically allocates time to different operators with a multi-armed bandit algorithm and a Bayesian belief model.
The evaluation results show the proposed method can achieve significant speedup.

##########################################################################

Reasons for score: 

I lean to rejection because of some flaws in the paper.
1. It does not mention a highly related paper that follows the same high-level idea (i.e., dynamically allocating time resource). The section 6 task scheduler in this paper (https://arxiv.org/abs/2006.06762) proposes a heuristic-based algorithm for dynamically allocating time resource. It also predicts room for improvement of each task. It supports user-defined objective functions and utilizes similarity between tasks. The authors should at least include a comparison with this method. The code of this method is merged into the TVM project (https://github.com/apache/incubator-tvm/blob/main/python/tvm/auto_scheduler/task_scheduler.py), which is the baseline of DynaTune.
2. The formulation of reward can be improved
In section 4.1, the authors define the reward as the latency reduction of one operator. However, an operator can appear multiple times in a network. So a weight should be multiplied to the reduction. The weight of an operator is the number of the appearance of it in the network.
3. Some assumptions can be improved
In challenge 2 of section 3, the authors claim "We note that the optimization of tensor operators is independent of each other". This is not true because different operators can share a cost model. They can contribute training data to the shared cost model. The shared cost model then influences the search of all operators. By using a shared cost model and transfer learning, the search time can be greatly reduced (as shown in AutoTVM and Ansor), so we cannot say the optimization of different operators is independent. How to model this effect is very challenging.


##########################################################################

Pros:

- Overall, the observations and ideas are correct. The paper is well-written with necessary background information.
- The proposed methods are all reasonable
- The evaluation results are satisfactory and well structured. It correctly evaluates all aspects that I want to know.

##########################################################################

Suggestion for improvement:

The authors can improve their formulation and add a comparison with the related paper I mentioned.
review_title: Nice idea. But lacking a comparision with a highly related paper.

final_score: rating: 7: Good paper, accept
review_content: The paper investigates the use of online optimization to improve the performance in Deep Neural Network (DNN) compilation. 
The three drawbacks of the existing approach/challenges in DNN compilation highlighted in the paper are (1) optimizing individual tensor operator performance instead of that of the entire model, (2) Static scheduling is oblivious to the optimization behavior, (3) extrapolating estimated performance while tuning.  

The authors model the model as scheduling in a time-slotted system, where they model the average performance (mean reward) of each operator using some performance curve obtained empirically. These curves capture the variability of operators across training duration, which helps with (1) and (2). For tackling (3),  the variability of the performance (noise) of each operator is estimated using some MCMC sampling techniques. Finally, using the mean reward and noise they cast the problem as finding the expected optimal schedule - a sequence of operators to be chosen during the training, as a Multi-armed bandit problem. The authors then design the DynaTune Algorithm base on a well-known UCB algorithm to solve this problem. 

The experiments show DynaTune outperforms AutoTVM (Chen et al., 2018b) the apparent state-of-the-art (see, disclaimer below) in many important DL architectures. 


Pros:
The problem of optimizing DL architectures is an important problem, as they are ubiquitous. The three challenges mentioned here sound credible (to an outsider), and in gains of dynamic scheduling is well known. Overall, the problem is well-motivated, and the solution proposed is justified. The experiments seem to give DynaTune an edge over an existing method. 



Cons:
* How Fig 2,3,4 are obtained is not clear to me. Some pointers to empirical studies or the methodology used are necessary. 

* The details behind the variability estimation using the MCMC method is also not clearly written. 

* The baseline AutoTVM (Chen et al., 2018b) is somewhat old. However, I am not sure if a better baseline is available. 







Disclaimer: Although I am familiar with works on Bandits, I am a complete outsider to the topic of DNN compilation.
review_title: The paper approaches DNN compilation by formulating it as a Multi-armed bandit problem. Interesting challenges to DNN compilation are identified, a sound methodology is proposed and experimentally validated.

final_score: rating: 7: Good paper, accept
review_content: The paper tackles the optimization speed of the deep neural networks. The paper models the operator optimization scheduling as a Multi-Armed Bandit problem. DynaTune first selects the operation to optimize in a discrete slot of time using a Bayesian Belief Model, then optimizes the model given the slot of time.

This interesting paper may help expedite the compilation time of the deep neural networks. As such, this paper tackles on a very important problem. I view this paper as a lineage of papers on "Expedited Compilation" and I believe this fresh view of the problem will open up more possibilities for exploration.

Modeling uncertainty to understand the gains of optimizing a particular operator is very interesting, and the way it leverages this to schedule the optimization process itself is novel.

It seems that the approach can be generically combined with different optimization frameworks such as genetic algorithms. It would be nice to have such discussion and would provide interesting insights.

As a side note, however, there was a paper the tackled the optimization of the inference speed of the end-to-end network [1] which is also applied to TVM. It would be interesting to see how the overall latency achieved by DynaTune compares to this approach.

Questions:
1. How does the end-to-end latency compare to [1]?
2. How are the results affected when put together with GA and Random Search for the actual optimization (line 6, 9 of Algorithm 1)

[1] "Optimizing CNN Model Inference on CPUs", ATC 2019
review_title: Official Blind Review #2

Meta-review: This paper applies multi-armed bandits to tuning deep learning code optimization. All reviewers agreed that this is an exploratory paper that opens up a new research area. My main criticism is algorithmic. In particular, the paper applies a 20-year old algorithm to a problem with a small number of arms. It is definitely not as impressive as

https://papers.nips.cc/paper/2018/file/f33ba15effa5c10e873bf3842afb46a6-Paper.pdf

who studied a different (but related) problem. The tuning problem in this paper also seems non-stochastic and contextual, while the authors apply a stochastic non-contextual bandit algorithm.

I shared these concerns with the reviewers, who insisted that the application is important enough to justify the acceptance of the paper. I respect their opinion and therefore suggest an acceptance. I encourage the authors to take my comments into account when revising the paper.
Decision: Accept (Poster)