Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the relationship between variational inference and GFlowNet. This work finds out these two algorithms are equivalent in the sense of the expected gradients of their learning objectives. Moreover, they demonstrate the superiority of the GFlowNet on off-policy training.
strength_and_weaknesses: Strength:
- The connection between the variational inference and GFlowNet is a new finding. This paper finds the intrinsic property, variance reduction, that makes the GFlowNet stand out. They also verify that GFlowNet is capable of stable off-policy training without importance sampling.
- This paper supplies a missing baseline result for hierarchical VI algorithms on modeling discrete random variables.

Weaknesses:
-   As the author mentioned in the conclusion section, this paper only studies the performance of GFlowNet and VI algorithms on modeling discrete random variables (since GFlowNet was mostly studied on such regime), which could be limited considering that VI algorithms are mostly used for continuous variables. 



clarity,_quality,_novelty_and_reproducibility: This paper is well-written and explains clearly the relationship between variational inference and GFlowNet in certain cases. Moreover, this paper conducted plenty of experiments on hypergrid, molecule synthesis, and DAG generation to verify the superiority of the GFlowNet on off-policy training, which should be reproducible with the provided code. 

Typo:
A typo in page 3, line 7, q(z_1,...,z_n)=q(z_1)q(z_2|z_1)...q(z_n|z_{n-1}) rather than x_1


summary_of_the_review: Overall this paper is well-written and provides an interesting connection between GFlownet and VI algorithms, and demonstrate the advantage of GFlowNet on off-policy training.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a connection between GFlowNets and variational inference (VI) algorithms for hierarchical variational models (HVMs), demonstrating that special cases of training HVMs via VI are equivalent to training GFlowNets via the trajectory balance (TB) objective. This is interesting because GFlowNets can be shown to automatically perform variance reduction in gradient estimation for reinforcement learning (RL), showing that GFlowNets may be helpful for RL settings where VI would have been used instead.
strength_and_weaknesses: Strengths:
- I liked the experiments the authors conducted which compared the empirical behaviors of various training objectives in practice. Understanding the settings in which the objectives differ and figuring out when each method performs better or worse than others is important, and I thought the authors did a good job on this on the synthetic task. It was also nice to see it scaled to slightly harder problems.
- I thought the connection to HVMs and wake-sleep algorithms was interesting, and could open up new research areas along this direction.

Weaknesses:
- The paper was a bit hard to follow at times.

clarity,_quality,_novelty_and_reproducibility: - Clarity/quality: The paper was a bit hard to follow. I think it’d be helpful if the technical exposition could be smoothed out.
- Novelty: The connection between VI algorithms and special cases of GFlowNets is novel and interesting.
- Reproducibility: The submission included code for the DAG and molecule synthesis experiments.

summary_of_the_review: The paper provides a theoretical and empirical investigation into the relationship between hierarchical VI and GFlowNets, outlining when they are equivalent and settings when one should be preferred over the other.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: The authors discuss the relationship between specific families of variational inference approaches (hierarchical variational inference) with generative flow networks as introduced recently by Bengio et al. (2021). The relation refers to equality in the expected gradients of the respective objectives. 
The authors evaluate this theoretical relationship further in a set of three experiments of various domains, discussing both similarities and differences.

strength_and_weaknesses: ## Strengths
- The paper demonstrates a novel connection between VI and GFlowNet approaches
- All statements and empirical claims are clearly stated and demonstrated
- A diverse set of experiments in different domains is provided

## Weaknesses
- A, rather minor, weakness is the lack of a stronger evaluation of the connection into the continuous domain, demonstrating the usefulness of the proposed new point of view compared to common VI approaches. 

clarity,_quality,_novelty_and_reproducibility: The paper is well-written and can be followed clearly. The discussed connection between the two sets of algorithmic approaches is novel and should lead to interesting new results, especially for continuous domains where the relative performance between them would be an interesting task to explore. The authors are encouraged to further pursue this direction.


### Typos
- p1 last line: hierarhical 
summary_of_the_review: A well justified, well written, and well evaluated paper.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper introduces a connection between variational inference and GFlowNet and evaluates the theoretical relationship in diverse experiments. 

Main strength: 
+ A novel connection between hierarchical variational inference and generative flow networks
+ Empirical claims are clearly stated and demonstrated in multiple sets of experiments.

Main weaknesses:
- Only studies algorithms on modeling discrete random variables. 
- The technical exposition was a bit hard to follow (Reviewer CLT1)

The reviewers appreciate the authors' response. The AC noted the additional experiment on a domain with continuous action spaces in Appendix F. This alleviates the concerns from Reviewer 6mPn. 

Overall, all reviewers are positive about this work, the AC recommends to accept this paper.


justification_for_why_not_higher_score: This is a nice paper, but two of the reviewers (6mPn, CLT1) do not express enthusiastic support for this paper (only rated the work marginally above the acceptance threshold). The reviewers also did not raise their scores after the rebuttal. The AC believes that Poster is a suitable choice. 
justification_for_why_not_lower_score: Based on the overall positive reviews, the AC thinks it deserves a poster presentation.
Decision: Accept: poster