Fast Training of Convolutional Networks through FFTs
Authors: Michael Mathieu, Mikael Henaff, Yann LeCun
ICLR 2014

Abstract: Convolutional networks are one of the most widely employed architectures in computer vision and machine learning. In order to leverage their ability to learn complex functions, large amounts of data are required for training. Training a large convolutional network to produce state-of-the-art results can take weeks, even when using modern GPUs. Producing labels using a trained network can also be costly when dealing with web-scale datasets. In this work, we present a simple algorithm which accelerates training and inference by a significant factor, and can yield improvements of over an order of magnitude compared to existing state-of-the-art implementations. This is done by computing convolutions as pointwise products in the Fourier domain while reusing the same transformed feature map many times. The algorithm is implemented on a GPU architecture and addresses a number of related challenges.


Peer review:
[Review]:
The paper describes the use of FFTs to speed-up the computation during training for convolutional neural networks working on images.
Essentially this is presented as a pure speed-up technique and doesn't change the learning algorithm, or (in an interesting way) the representation. 

The idea of applying FFTs to speed up image processing systems, particularly 'sliding windows' systems, is far from new and there is a large literature on this.In particular combining FFTs with Neural networks is not new,
e.g. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.36.1967
Some of this prior literature should be cited. 

I am not aware of any work that applys the back-propagation in the Fourier domain too. 

The resulting speed-ups are significant for the scenario the authors are considering, and it is useful to know that the practical implementation delivers these gains. As they conclude, these results may change the way such problems are formulated by removing the pressure to use small kernels.

Expand the caption for Figure 2. Total number of operations for what? 

Figure 3 needs units for the y axis (text says seconds?), and for the x axes - ie areal or linear pixels?
Also for each of the 3 sets of graphs, there needs to be an indication of what are the values of the parameters which are  held constant.

Please say in the text that all 3 systems (Torch, Authors' and Krizhevsky) were running on the same (which?) GPU. 

Citation for Cooley-Tukey FFT?