Venue: ICLR 2018 Workshop

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper shows the outer-product structure in gradient, which has been reported by previous works, specifically the "sufficient-factor" work in the context of first-order gradient.  The authors show that similar structure exists in the second order gradient. 

However, it is unclear how this structure can be applied to algorithms that speedups the update. I would encourage the author to come up with such algorithm and along with appropriate experiments to show the applicability of the method. 

One potential drawback of such approach is that the factored representation relies on small batch size. While second order method usually requires bigger batch-sizes.
review_title: useful gradient calculation mechanism generalises to second order but lacks algorithm and experiments

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper is well-written and well-motivated, but the central claim, "[feed-forward and recurrent neural networks] exhibit an outer product derivative structure", is not spelled out in sufficient mathematical clarity -- nor is the contrasting claim, that convolutional networks do not possess this structure. No experiments or specific applications of the finding are described in detail. In my view this would nonetheless be an interesting workshop presentation.
review_title: Interesting, though opaque paper

Meta-review: Based on the reviews, this paper has not been accepted for presentation at the ICLR workshop. However, the conversation and updates can continue to appear here on OpenReview.
Decision: Reject