There is no scientific consensus on whether quantum annealers such as the D-Wave 2000Q that use the transverse-field Ising models yield any gains over classical methods  (c.f. https://arxiv.org/abs/1703.00622).  However, it is an exciting research area  and this paper is an interesting demonstration of the feasibility of using quantum annealers for reinforcement learning.  This paper builds on Crawford et al. (2016), an unpublished preprint, who develop a quantum Boltzmann machine reinforcement learning algorithm (QBM-RL).  A QBM consists of adding a transverse field term to the RBM Hamiltonian (negative log likelihood),  but the benefits of this for unsupervised tasks are unclear  (c.f. https://arxiv.org/abs/1601.02036, another unpublished preprint).  QBM-RL consists of using a QBM to model the state-action variables:  it is an undirected graphical model whose visible nodes are clamped to observed state-action pairs.  The hidden nodes model dependencies between states and actions, and the weights of the model are updated to maximize the free energy or Q function (value of the state-action pair). The authors extend QBM-RL to work with quantum annealers such as the D-Wave 2000Q, which has a specific bipartite graph structure and requires special consideration  because it can only yield samples of hidden variables in a fixed basis.  To overcome this, the authors develop a Suzuki-Trotter expansion and call it 'replica stacking', where a classical Hamiltonian in one dimension higher is used to approximate the quantum Hamiltonian.  This enables the use of quantum annealers.  The authors compare their method to standard baselines in a grid world environment. Overall, I do not want to criticize the work.  It is an interesting proof of concept.  But given the high price of quantum annealers, limited applicability of the technique, and unclear benefits of the authors' method, I do not think it is relevant to this specific conference.  It may be better suited to a workshop specific to quantum machine learning methods.  ======================================= + please add an algorithm box for your method.  It deviates significantly from QBM-RL.  For example, something like: (1) init weights of boltzmann machine randomly (2) sample c_eff ~ C from the pool of configurations sampled from the transverse-field Ising model using a quantum annealer with chimera graph (3) using the samples, calculate effective classical hamiltonian used to approximate the quantum system (4) use the weight update rules derived from Bellman equations (spell out the rules).  + moving the details of sampling into the appendix would help;  they are not important for understanding the main ingredients of your method There are so many moving parts in your system,  and someone without a physics background will struggle to understand it.  Clarifying the algorithm in terms familiar to machine learning researchers will go a long way toward helping people understand your method.  + the benefits of your method is unclear -  it looks like the method works, but doesn't outperform the others.  this is fine,  but it is better to be straightforward about this and bill it as a 'proof of concept'  + perhaps consider rebranding the paper as something like 'RL using replica stacking for sampling from quantum boltzmann machines with quantum annealers'.  Elucidating why replica stacking is a crucial contribution of your work would be helpful, and could be of broad interest in the machine learning community.  Right now it is too dense to be useful for the average person without a physics background:  what difficulties are intrinsic to a quantum Hamiltonian?  What is the intuition behind the Suzuki-Trotter decomposition you develop?  What is the 'quantum' Boltzmann machine in machine learning terms (hidden-hidden connections in an undirected graphical model!)?  What is replica-stacking in graphical model terms  (this would be a great ML contribution in its own right!)?  Really spelling these things out in detail (or in the appendix) would help ========================================== 1) eq 14 is malformed 2) references are not well-formatted 3) need factor of 1/2 to avoid double counting in sums over nearest neighbors (please be precise)