Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The authors study a combinatorial multi-robot scheduling problem (in fact the robot part is a bit inflated, since the experiments only involve agents in a simulated discrete state-space maze) using a method that builds upon recent advances from [Dai et al. (2017)]. The main contribution is to consider each of the steps taken by Dai et al. to solve combinatorial problems on graphs, and adapt them to the considered scheduling problem.

Not being an expert in RL, my assessment should be discounted. However, I am not sure I follow properly the main idea of the paper. The point of Dai et al. was to use RL to solve a wide family of combinatorial problems. Now, the authors claim to build upon these ideas to solve... what looks essentially like a far more standard RL problem, and not necessarily a combinatorial optimization problem. The main insight by Dai et al. was to highlight the fact that combinatorial problems are usually solved (or approximated) without "warm starts", i.e. they do not consider distributions on problem instances to learn from. The problem considered by the authors is, quite on the contrary, a typical RL problem where information is extracted from the problem's structure (here a maze). Therefore, I feel there is something of a fundamental contradiction going on at a fairly high-level, in the sense that the paper "uses RL to solve a subset of combinatorial problems that were studied by RL before". The absence of other baselines in experiments make this even more suspicious. Therefore I believe the paper's presentation could be greatly improved if it were better "located" within the RL literature (which is almost non-existent in the very brief bibliographic section) and that the authors were able to show that  their proposals are original, within an RL context.

minor points:
* the comment "While learning-based methods are generally believed to suffer exponentially increasing training requirements as problem size (number of robots and tasks) increases, our method’s training requirement is empirically shown not to scale while maintaining near-optimal performance" --> this is too loose a statement. Provide more evidence or references.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: Paper addresses the problem of centralized multi-machine task assignment in an RL setting ("multi-robot reward collection"). Claim is that this has not been successfully done in a RL setting before, so a new problem is proposed (multi-agent pac-man) and results are presented on this problem. Approach proposed extends prior work from Dai 2017 and 2016 (which I am a priori unfamiliar with), and it seems to me that the exposition of this method leans a bit too heavily on presumed familiarity with those works. An auction-consensus approach is proposed whereby each machine makes a bid for each unclaimed task, then the coordinator picks the highest bid and assigns that task-machine pairing, after which the remaining machines make bids for the remaining tasks, and so forth.

As it stands, part of me leans toward rejecting for a couple reasons.
1. The exposition of the method needs to be improved to assume less background knowledge of the heuristic PGM and  structure2vec methods, investing some text introducing them. Appendix C seems to do part of this, and probably should be integrated into the body of the paper.
2. Another view of "random graphical models" is the sampling trace of a universal PPL. This is studied in, e.g. https://cocolab.stanford.edu/papers/daipptr.pdf so it seems like this deserves at least a brief additional literature review as opposed to simply diving into MFI. Appendix D looks OK: since the action space is discrete, then a fixed point approach becomes feasible.

On the other hand, the experiments are good, the auction approach is a nice idea/novel. The ablation experiment is good, and the comparison against OR tools is also good to have. Insofar as the structure2vec is representation-oriented, it seems like a decent fit to the venue.

On balance, I think the paper needs too much polish and revision to accept at this time.

Minor nits:
The word "seminar" is used a couple times, where from context I think "seminal" is intended.
Some figure refs are broken.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: In this paper, the authors propose a reinforcement learning method for multi-robot scheduling problems. They state the method's scalable performance and transferability. My major concerns are as follows.

1. The paper is not easy to read. In my understanding, multi-robot scheduling is a very important problem and is very similar to many scheduling problems in complex platforms such as the dispatch system for ride sharing and package delivery. However, I did see any real application in this paper. It is very difficult to understand how this proposed method works and what is the benefit under non trivial environment.

2. The experiments (2~8 robots, 20~50 tasks) cannot support the scalable performance or large problems very well. How about thousands and millions of robots/tasks, e.g. routing planning or dispatching for vehicles in a ride sharing platform?  

3. It is not convincing without comparison with necessary baseline methods.

4. There is no in-depth analyses for the transferability.

5. There are many typos, such as the missing figure citation with Figure ??.   

review_title: Official Blind Review #1

Meta-review: Unfortunately, the reviewers of the paper are all not certain about their review, none of them being RL experts.  Assessing the paper myself—not being an RL expert but having experience—the authors have addressed all points of the reviewers thoroughly.  

Decision: Reject