Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper seems clear enough and original enough. The idea of jointly forming groups of operations to colocate and figure out placement on devices seems to hold merit. Where the paper falls short is motivating the problem setting. Traditionally, for determining optimal execution plans, one may resort to cost-based optimization (e.g., database management systems). This paper's introduction provides precisely 1 statement to suggest that may not work for deep learning. Here's the relevant phrase: "the cost function is typically non-stationary due to the interactions between multiple devices". Unfortunately, this statement raises more questions than it answers. Why are the cost functions non-stationary? What exactly makes them dynamic? Are we talking about a multi-tenancy setting where multiple processes execute on the same device? Unlikely, because GPUs are involved. Without a proper motivation, its difficult to appreciate the methods devised.

Pros:
- Jointly optimizing forming of groups and placing these seems to have merit
- Experiments show improvements over placement by human "experts"
- Targets an important problem

Cons:
- Related work seems inadequately referenced. There exist other linear/tensor algebra engines/systems that perform such optimization including placing operations on devices in a distributed setting. This paper should at least cite those papers and qualitatively compare against those approaches. Here's one reference (others should be easy to find): "SystemML's Optimizer: Plan Generation for Large-Scale Machine Learning Programs" by Boehm et al, IEEE Data Engineering Bulletin, 2014.
- The methods are not well motivated. There are many approaches to devising optimal execution plans, e.g., rule-based, cost-based, learning-based. In particular, what makes cost-based optimization inapplicable? Also, please provide some reasoning behind your hypothesis which seems to be that while costs may be dynamic, optimally forming groups and placing them is learn-able.
- The template seems off. I don't see the usual two lines under the title ("Anonymous authors", "Paper under double-blind review").
- The title seems misleading. ".... Device Placement" seems to suggest that one is placing devices when in fact, the operators are being placed.
review_title: The paper proposes to jointly learn groups of operators to colocate and to place learned groups on devices so as to distribute operations for deep learning via reinforcement learning.

final_score: rating: 5: Marginally below acceptance threshold
review_content: In a previous work [1], an auto-placement (better model partition on multi GPUs) method was proposed to accelerate a TensorFlow model’s runtime. However, this method requires the rule-based co-locating step, in order to resolve this problem, the authors of this paper purposed a fully connect network (FCN) to replace the co-location step. In particular, hand-crafted features are fed to the FCN and the output is the prediction of group id of this operation. Then all the embeddings in each group are averaged to serve as the input of a seq2seq encoder. 

Overall speaking, this work is quite interesting. However, it also has several limitations, as explained below.

First, the computational cost of the proposed method seems very high. It may take more than one day on 320-640 GPUs for training (I did not find enough details in this paper, but the training complexity will be no less than the in [1]). This makes it very hard to reproduce the experimental results (in order to verify it), and its practical value becomes quite restrictive (very few organizations can afford such a cost).

Second, as the author mentioned, it’s hard to compare the experimental results in this paper wit those in [1] because different hardware devices and software versions were used. However, this is not a very sound excuse. I would encourage the authors to implement colocRL [1] on their own hardware and software systems, and make direct comparison. Otherwise, it is very hard to tell whether there is improvement, and how significant the improvement is. In addition, it would be better to have some analysis on the end-to-end runtime efficiency and the effectiveness of the placements.

 [1] Mirhoseini A, Pham H, Le Q V, et al. Device Placement Optimization with Reinforcement Learning[J]. arXiv preprint arXiv:1706.04972, 2017. https://arxiv.org/pdf/1706.04972.pdf 

review_title: In a previous work [1], an auto-placement (better model partition on multi GPUs) method was proposed to accelerate a TensorFlow model’s runtime. However, this method requires the rule-based co-locating step, in order to resolve this problem, the authors of this paper purposed a fully connect network (FCN) to replace the co-location step.

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper proposes a device placement algorithm to place operations of tensorflow on devices. 

Pros:

1. It is a novel approach which trains the placement end to end.
2. The experiments are solid to demonstrate this method works very well.
3. The writing is easy to follow.
4. This would be a very useful tool for the community if open sourced.

Cons:

1. It is not very clear in the paper whether the training happens for each model yielding separate agents, or a shared agent is trained and used for all kinds of models. The latter would be more exciting. The adjacency matrix varies size for different graphs, so I guess a separate agent is trained for each graph? However, if the agent is not shared, why not just use integer to represent each operation in the graph, since overfitting would be more desirable in this case.
2. Averaging the embedding is hard to understand especially for the output sizes and number of outputs.
3. It is not clear how the adjacency information is used.

review_title: Elegant method with impressive results

Meta-review: The authors provide an alternative method to [1] for placement of ops in blocks. The results are shown to be an improvement over prior RL based placement in [1] and superior to *some* (maybe not the best) earlier methods for operations placements. The paper seems to have benefited strongly from reviewer feedback and seems like a reasonable contribution. We hope that the implementation may be made available to the community.

[1] Mirhoseini A, Pham H, Le Q V, et al. Device Placement Optimization with Reinforcement Learning[J]. arXiv preprint arXiv:1706.04972, 2017. 
Decision: Accept (Poster)