Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Summary: This work proposes to use a combination of graph neural networks (GNNs) and proximal policy optimization (PPO) to train policies for generalized device placement in dataflow graphs. Essentially, (1) a GNN is used to learn representations of a dataflow graph (in an inductive manner), (2) a transformer is used to output a device placement action for each node in the graph, and (3) the entire system is trained end-to-end via PPO. Extensive experimental results show very impressive results compared to strong baselines.

Assessment: Overall, this is a solid application paper. The authors GNNs, PPO, and Transformers in an effective, well-motivated, and sound manner. Moreover, the task is interesting and relevant. There is not significant methodological novelty, as the authors are essentially combining standard components in a straightforward way. That said, the results are strong and the paper is well-written, so it certainly has merits as an application paper. 

Will code be released? This is essential for reproducibility, as the paper does not contain sufficient technical details to allow for reproduction.

Reasons to accept:
- Strong empirical results on an interesting application 
- Well-written paper
- Thorough experiments

Reasons to reject:
- Incremental methodological contribution
- Likely difficult to reproduce
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: In this paper the authors propose an end-to-end policy for graph placement and partitioning of computational graphs produced "under-the-hood" by platforms like Tensorflow. As the sizes of the neural networks increase, using distributed deep learning is becoming more and more necessary. Primitives like the one suggested by the authors are very important in many ways, including improving the ability of the NN to process more data, reduce energy consumption etc. The authors compared to prior work propose a method that can take as input more than one data flow graphs, and learns a policy for graph partitioning/placement of the operations in a set of machines that minimizes the makespan. This problem in principle is NP-hard as it entails both graph partitioning and graph scheduling as its components. The authors propose a heuristic that composes of two existing methods: graph neural networks are used to produce an embedding of the computation/data flow graph, and then a seq-2-seq placement network. The method is able to generalize to unseen instances.

I vote for weak reject since some issues that I would like to see addressed by the author(s) are not. These include the fact that since the goal is to minimize the makespan,  scheduling within each machine the operations should be addressed in a better way. Also, while the objective J(\theta) is reasonable, the distribution for the makespans could be very skewed (e.g., heavy tails over the dataflow graphs). Doesn't this affect the results? Finally, the novelty from a deep learning perspective is limited.

- How do the authors address the issue of scheduling the operations within each machine? 
- How is \mathcal{D} formally defined (i.e., the range of the mapping function)? Do you take into account the different number of machines, their memory footprints that can be significantly different, the different processing units they may have (GPUs, CPUs, TPUs)? Is the number of machines used for the partition automatically learned by the policy? That part was not very clear.
- Since the authors compare with METIS, it is worth also comparing with Scotch https://www.labri.fr/perso/pelegrin/scotch/ that is also publicly available.
- Can the authors comment on the scalability of their method as a function of n (number of nodes), and k (number of  devices)? 


Update: Thanks to the author(s) for the detailed feedback. I have upgraded my score accordingly. 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: GDP: Generalized Device Placement for Dataflow Graphs

This paper presents a method to assign the individual operations making up the dataflow graph of a deep neural network to a set of connected devices on which they should be executed, with the objective of maximizing runtime performance of inference. The paper’s proposed approach relies on graph neural networks to produce embeddings of the dataflow graph nodes that are claimed to be transferable to unseen graphs. These embeddings serve as input to a transformer-based sequence model that outputs the final assignment. The end-to-end model is trained with a reinforcement learning criterion minimizing the expected placement runtime. Experimental comparisons against alternative placement methods (including human expert) are presented for several large neural networks of different architectures.

Overall, the paper is well written, easy to follow, and addresses an important concern is scaling up neural networks to large model sizes. The proposed approach combining graph neural networks with transformer-based placement network appears, at first glance, novel. 

My main reservations with the paper is that it lacks many details in several key sections, preventing a full appreciation of the contributions, and making reproducibility of results impossible to contemplate. In particular:

1. How the placement network is designed and used (section 3.2) is completely lacking.
2. The details of how the training and testing datasets are obtained are also lacking. In particular, the specific model architectures for the likes of RNNLM, including hyperparameter choices, the runtime data fed to those models, etc. (This can be given in appendix). It is also not clear if the models listed in Table 1 are singular models with fixed values of hyperparameters, or if they correspond to distributions over models (e.g. with differences in hyperparameters).
3. How PPO is used is glossed over: for instance, is the proposed placement of a given model tried « live » in the inner loop of PPO to generate the reward corresponding to its runtime? What runtime data is fed to the model in order to do this? Since this would be a somewhat unusual setup, an illustration of the overall training loop would certainly solidify understanding. Moreover, a few sentences or equations explaining how PPO is used in this context would also help. Is training with PPO imperative for getting good model performance? Have other methods been tried?

In section 3.3, the parameter superposition mechanism appears to perform a kind of meta-learning. Explicit connections to such should be made, such as « Tadam: Task dependent adaptive metric for improved few-shot learning » (NeurIPS 2018).

Moreover, at the end of section 4.4, the paper claims to « report superhuman results on 8-layer GNMT ». Unless this reviewer misunderstands, the results in Table 1 (wherein GDP-one clocks in at 0.649 for 8-layer GNMT, versus 0.610 for Human Placement) would contradict this claim. This should be clarified.

Given these reservations, in spite of the potential of the proposed approach, the paper appears in its current form too immature to recommend acceptance at ICLR.

review_title: Official Blind Review #3

Meta-review: This paper presents a new reinforcement learning based approach to device placement for operations in computational graphs and demonstrates improvements for large scale standard models.

The paper is borderline with all reviewers appreciating the paper even the reviewer with the lowest score. The reviewer with the lowest score is basing the score on minor reservation regarding lack of detail in explaining the experiments.

Based upon the average score rejection is recommended. The reviewers' comments can help improve the paper and it is definitely recommended to submit it to the next conference.


Decision: Reject