reviews: 
Comments: This paper derives a connection between generative flow networks (GFlowNets) and policy gradient methods. An equivalence of gradients between GFlowNet objectives and gradients of variational objective is (re)derived, and from this emerges an equivalence with a reinforcement learning problem that can be solved using policy-based RL methods such as TRPO. Experiments are performed on three amortized sampling problems from past work and the algorithm is claimed to deliver improved samplers on two of them.
Strengths: - The connection between GFlowNets and policy-based RL is stated for the first time, which can be valuable if it can produce improved training algorithms for GFlowNets.
- New result extending the SubTB gradient analysis from [Malkin et al., 2022b] to state flows.
- Experiments on diverse problems, representing a nontrivial engineering effort by the authors.
Weaknesses: - There are many instances of unclear and incorrect mathematical language and notation in section 2 and 3.
  - Section 2.1:
    - "Assume there are $T+1$ topological orderings" -- that is not what "topological ordering" means; a topological ordering is a partial order homomorphism from a given partial order to a total order (i.e., a list of the states where no state is preceded by its descendant). Is it meant that the DAG is graded (as stated in the following paragraph) and that the layers are indexed by $0,\dots,T$? 
    - Similarly, an instance of two elements being in a binary relation is not a "partial order". A partial order is a set with a binary relation (satisfying some properties), equivalent to a DAG.
    - It is incorrect to **define** ${\cal S}_0:=\{s_0\}$ and similar for ${\cal S}_T$, if the stratification into ${\cal S}_t$ already comes with the data of a graded DAG. That $s^0$ is the only state in the first graded component ${\cal S}_0$ is an **assumption/requirement**.
  - Section 2.2:
    - The second sentence of 2.2 is self-contradictory. "Probability measure" means the measure of the entire $\cal T$ is 1. In fact $F$ is just a measure, and $P$, its normalization, is a probability measure.
    - Equation (2) is missing a condition that this should hold only for $s\neq s^0$.
    - In equation (3), backward policy was not introduced/defined.
    - I did not understand this: "$\hat Z$ is a constant whose value is clamped to $Z$".
  - Section 3 start: I don't believe [Madan et al., 2023] discusses the relationship between TB and KL divergence. However, that relationship is discussed in [Malkin et al., 2022b] and in [Zimmermann et al., "A variational perspective on generative flow networks", TMLR].
  - Section 3.1:
    - "Expectation over $P_B$ is intractable due to unknown $Z^*$" -- even with known $Z^*$, sampling $P_B$ is not necessarily tractable.
    - "This forward gradient equivalence [in [Malkin et al., 2022b]] does not take the total flow estimator into account" -- in fact, that paper shows that the equivalence on $P_F$ parameter gradients holds independently of $Z$ and *also* states the equivalence for the $\log Z$ gradient (after equation (12) in that paper). Thus there is **nothing original in Proposition 1**.
    - In Proposition 1, it should be made clear that we do not propagate gradients to the distribution $P_F$ over which the expectation is taken. Otherwise, the proposition is not true. (This remark is made in Appendix A but not pointed to.)
- The paragraphs following Definition 1 are extremely difficult to understand, switching between GFlowNets and regular RL settings. 
  - I object to the statement that in "regular RL" the Markov chain determined by the policy and transition environment is ergodic (many RL settings feature "irreversible" actions).
  - It should be noted that the math following Definition 1 is related to that in max-ent RL, which effectively places negative log-policy into the reward.
    - Most importantly, how can the policy gradient methods described here work off policy (which is the important advantage of GFlowNets, and important for preventing mode collapse)?
- The second sentence of section 3.3 seems key but does not make sense to me. The backward policy is a distribution over parents of each state and induces a distribution over subtrajectories. How can the $P_B$ be higher/lower for trajectories that precede a high-reward state $x$ if it is a distribution conditioned on $x$? 
  - Also, please check grammar in that sentence.
- The experiment results are not very convincing for a few reasons:
  - On the hypergrid, the policy-based methods effectively learn from partial trajectory information, so a fairer comparison may be to subtrajectory balance (which actually performs better on large hypergrids [Madan et al.]).
  - Why is RL-G not tested in sections 4.2 and 4.3?
  - How were training hyperparameters selected?
  - I am not convinced by the evaluation of total variation in 4.2 and 4.3 if it is performed by sampling, which can induce high variance in the estimate. It should be possible to compute the true terminating distribution, at least for the structure learning problem. JSD between the full distributions would allow comparison with past work; right now, there is no way to validate that TB faithfully reproduces results from [Deleu et al.] and [Malkin et al., 2022b].
  - The same concerns apply to the bit sequence problem (4.2).
- Miscellaneous: 
  - Please check your citation type (`\citet`/`\citep`) and consistent capitalization in "GFlowNet", which is wrong in multiple places.
  - Broken ref at the bottom of p.5.
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 2 fair