Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces a class of software called Automatic Propagation (AP), an extension of autodifferentiation software as well as a specific instance of it called Proppo. AP allows users to implement custom learning algorithms into the base unit of a "propagator" that can be reused. The authors note that many ML software packages adhere to the same paradigm of a surrogate loss to be used with backpropagation, while other forms of calculations are not optimized for or considered as they are not the standard. As such, they design AP to be a general message passing framework for ML.  The basic unit of Proppo is a propagator that implements both a forward mode (which performs computation, and stores any necessary data for the backward mode ) and backward mode of computation, which performs backward computation sending and recieving messages to other nodes. The whole process is managed by a propagation manager. 

The utility of Proppo and the AP general system is demonstrated by the implementation of total propagation and gaussian shaping gradient algorithms for MC gradient estimation, which are then shown to perform (in terms of gradient accuracy) orders of magnitude better than the standard surrogate loss + autodiff method for certain classes of problems.
strengths_and_weaknesses: Strengths: The paper is well written and motivated. The contribution is important and original. The experimental results are thoughtful and validate the claim that there are situations where total propagation algorithm can be more accurate gradient estimator than those that can be achieved by standard AD software. 

Weaknesses: The paper does not seem to link to any open sourced code, which would be a nice addition to the paper, to see some examples of how the more sophisticated propagators could be implemented. 
questions: In the minimalistic code example section starting on line 163, it could be useful for some readers to understand how Proppo configuration code could be modified to do the default pytorch behavior (normal backpropagation) (I acknowledge that you introduce the BackPropagator later but it might provide some clarity to new readers if mentioned in the first minimal code section that you could simply also use backprop in addition to changing the behavior of the code completely.

I suggest rewording the sentence on 167-169, as it is confusing what the intention is due to the grammatical errors. 

If you could open source some code for Proppo, it would make an even more impactful contribution.
limitations: The authors have addressed some limitations of the work (i.e. that it is ultimately up to the user to use the framework well to achieve the desired aims of composability and customisability. Giving further guidelines and documentation for users on the best ways to use the framework would be worthwhile, especially if open sourced.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The authors present a software framework (Proppo) for constructing propagation algorithms on generic computational graphs. These can be used to implement a variety of learning approaches. The authors make the case that existing ML frameworks, because of their limited capabilities of operating on generic computation graphs, discourage researchers from pursuing particular directions. The authors present example approaches to Monte Carlo gradient estimator and the Total Propagation algorithm implementations using Proppo, then show an experiment demonstrating Total Propagation can improve stability of an RNN-based system with nearly-chaotic dynamics.

strengths_and_weaknesses: ##### Strengths

The idea of developing a generic framework to perform propagation on computational graphs is important and indeed opens the doors for many new learning algorithms that may not be based on gradient-based learning or may involve gradient computation with higher order or other methods. There is a need for such a generic framework to implement such algorithms.

##### Weaknesses

The paper is very light on the details of the framework itself — the APIs, semantics for implementing other learning algorithms, or implementation details are not explained at all in neither the paper nor the appendix. This makes verifying and validating the quality and correctness of the authors’ implementations difficult.

The work is missing a set of performance benchmarks, which are especially relevant for Monte Carlo-based estimation methods. Total Propagation or similar methods are not new, and while an implementation of them using a generic framework has utility, it is not clear if such an implementation is competitive computationally with other implementations.

The authors present no new learning algorithms imagined with Proppo, instead benchmarking existing ones. The paper would be made stronger if the authors were able to present case studies demonstrating the utility of their approach in more detail. The Monte Carlo gradient estimation example in the paper, while an interesting setting in which to study stability, is not convincingly broadly useful. The authors should make such a case and should expand the set of learning settings in which they explore these algorithms, taking advantage of their interoperability with PyTorch.

questions: > implemented entirely in Python

Given that the entire framework is implemented in Python, how does performance hold up for large graphs? Large graphs emerge in many learning paradigms (e.g. transduction, graph learning). Performance benchmarks are missing from the paper.

> [Messages are implemented via] a pointer to a parent node

More implementation details regarding statements like this would be very helpful.

> propagation managers working in parallel

How is this accomplished? Threading? Efficient graph traversal?
limitations: Proppo as a framework allows users to implement RP and LR algorithms as discussed, but these algorithms are not new, and the authors do not detail why an implementation enabled by Proppo is any better than existing implementations of these algorithms, beyond positing that there may be correctness issues with existing implementations that Proppo helps avoid. These ideas are very underspecified.
review_title: null

final_score: rating: 2: Strong Reject: For instance, a paper with major technical flaws, and/or poor evaluation, limited impact, poor reproducibility and mostly unaddressed ethical considerations.
review_content: summary: Existing automatic differentiation (AD) frameworks do not provide flexibility for composing learning algorithms. This paper invents Automatic Propagation (AP) software, which generalizes AD, and allows the construction of complex learning algorithms.


strengths_and_weaknesses: This work serves as, in my opinion, a kind of software illustration that incorporates modern learning algorithms. As a result, I didn't find this paper to make a major academic contribution.
questions: See "Strengths And Weaknesses"
limitations: See "Strengths And Weaknesses"
review_title: null

Meta-review: metareview: As an AC of the paper, I have read this paper and made the decision not only based on the reviewers' reviews but also considering my own judgment as well. Unfortunately, one reviewer could not provide a review with high confidence, and the other two disagreed where Reviewer KJqN was willing to nominate the paper for publication. 

# Summary
This paper presents a software framework for a method that the authors called as "Automatic Propagation" (AP.)  AP generalizes AD and allows us to implement custom and composable learning algorithms. However, most of the ML frameworks previously focused on AD rather than implementing general message passing algorithms for learning with Monte-Carlo gradient estimation. This paper proposes a software framework called Proppo implemented in PyTorch to fill this gap. Proppo has "base units" called *propagators* that implements the forward and backwards mode of the computation. Propagators are quite flexible and users of the framework could easily download propagators to implement algorithms like backpropagation and the propagators are managed by "*Propagation Managers*". The paper uses this software framework (Proppo) to implement algorithms like total propagation and gaussian-shaped gradients. Total propagation is, in particular, interesting because it combines LR and RP. It did not have a python implementation before. The authors provide extensive analysis of those both in terms of variance and the speed of their framework. 

# Strengths and Weaknesses
This paper studies an important and interesting problem in machine learning on designing software frameworks that could easily enable the implementation of learning algorithms that make use of Monte Carlo gradient estimation. I am not an expert on message-passing frameworks, but the overall proposal looks novel to me. Overall, it feels like this paper's strengths outweigh the weaknesses. I think majority of the weaknesses could be easily addressed in the camera-ready version of the paper.

**Strengths** 

* Important problem and provides a well-thought solution for automatic propagation.
* Thorough and convincing experiments.
* Provides a simple and flexible solution for AP, which I hope will enable future research by enabling the implementations of MC gradient estimation algorithms like total propagation that didn't have a python implementation before.

**Weaknesses**

* *Clarity:* The writing could be improved; as Reviewer JEb5 noted, the paper is very light on the details of the framework. Figure 1 feels unnecessary, and the same story could be told without it. Figure 2 is not referred from anywhere in the main text. That figure lacks sufficient description and is difficult to understand. For example, what do the dashed lines in Figure 2 mean? 
In the paper, it is not clear what makes an algorithm like "Total Propagation" to be difficult to implement in AD-based frameworks, such as in Storchastic framework. The authors should clarify in this paper why it is easier/better to implement these algorithms with AP and Proppo. Especially it feels like the authors can do a better job at justifying and motivating message passing algorithms in the paper. In the current version, this is briefly done in the intro by citing Minka et al., 2019 slides.
* *Code* The current version of the paper lacks the source code, but the authors promised to release the code with the paper. I would at least expect to see a submission of code with the supplementary material for an early review. Especially it would be very helpful if the authors could present an example of an ML model such as a variational auto-encoder implemented with Proppo.

## Decision
The paper and the software framework Proppo are important contributions to the community that might interest the broader NeurIPS community. As far as I am aware, AP proposed in this paper is novel, and it helps implement Monte-Carlo gradients on stochastic computational graphs. There are also a bunch of other possible use-cases of Prop the authors mentioned. I can definitely see this type of framework becoming popular quickly in the community if it is implemented well and is easy to use. From the examples provided in the paper, Proppo seems to have a relatively simple interface and is easy to use. I am willing to nominate this paper for acceptance with the condition that the framework will be open-sourced with the publication of this paper. Without open-sourcing Proppo this paper would not be as impactful. Besides, I would recommend the authors clarify the following points in the camera-ready version of the paper:
* Refer to the computational time experiments in App. B.3 (previously B.2) from the main paper.
* The paper should incorporate the clarifications requested by reviewer KJqN into the main paper.
* The paper should compare and clarify its differences with respect to other stochastic gradient estimation PyTorch libraries such as  Storchastic.
* I would recommend that the authors address my concerns about the clarity and code in the weaknesses section for the camera-ready version of the paper.


confidence: Less certain
Decision: Accept