Venue: NeurIPS 2019 Workshop Program_Transformations

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The article is a good overview of all the AD features that have been made to work in Zygote, an implementation of AD in Julia. The whole language (including recursion and higher-level functions), excluding macros themselves, is supported.
Additional features have also been implemented, such as checkpointing, hooks, and mixed-mode AD.
It is interesting to see that simple design decisions on the representations of data (cells and boxes) and derivatives (pullback closures) enabled different extensions.
It would be great to learn about how those concepts needed to be specialized or optimized for different use cases, such as sparse arrays, asynchronous execution.
review_title: Solid implementation of AD in Julia

final_score: rating: 6: Marginally above acceptance threshold
review_content: Although the introduction does mention some other examples (like Tapenade and Swift for TensorFlow) I still think that the first section slightly misrepresents the state of AD: Performing AD by transforming the source code (or some representation thereof) has been around for a long time and has been implemented on a variety of languages such as Fortran, Java, Matlab, etc. (see https://arxiv.org/abs/1502.05767 for an overview). In that sense, Zygote and other recent frameworks such as Swift for TensorFlow are probably best presented as new implementations of old and tested techniques (that maybe had fallen out of favor in ML).

That said, while I see little novelty in the underpinnings of the framework, the range of supported features is impressive and suggests that the implementation details of the framework were thought through and well-executed. As such, I would love to see this work presented in more detail at the workshop. I would suggest that the authors focus the presentation of their work on advanced features such as concurrency/parallelism, checkpointing, and mixed-mode AD. I would also like to see a discussion of the frameworks shortcomings or unsupported features, e.g., how does Zygote handle mutability of arrays or dynamic redefinition of functions?


review_title: Impressive set of supported language constructs in AD

Meta-review: An overview of a strong AD system for Julia, but lacking significant novelty.
Decision: Accept (Poster)