Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper discusses the distribution of function mappings governed by deep infinitely wide neural networks and, in theory, how to sample the weights using the canonical feature mappings (or the so-called kernel reparameterisation trick) and map the data points to features. 

The paper then proposes an initialization scheme for finite-width neural networks, inspired by the above analysis. I don't think I quite understand the procedure you use. Can you please clarify: 

i) what do you mean by "guide by the idea of ... in a supervised fashion", what is supervised here? Perhaps it's mean you use the inputs to initialise the weights, compared to other techniques that only use the size of the layers?

ii) the procedure you use to initialise weights, say for a single layer net.

While the contribution is theoretically interesting, I'm afraid it doesn't reach the level required (i.e. late-breaking work) for the workshop.
review_title: review: interesting but need stronger experimental evidence

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper is an attempt to gain a better understanding of deep learning through kernels. I think the paper does not add much to the previous works such as (Hazan and Jaakkola 2015) and (Daniely et al. 2016). The suggested trick to construct weights is interesting but I don't understand why such initialization should be useful. In particular, what is the value of supervised initialization? The experimental results show that the supervised initialization cannot improve the final results. Moreover, the reported test errors are far worse than what is being reported for single layer network of similar size.
review_title: null

Decision: Accept