Paper:
Due to their wide field of view, omnidirectional cameras are frequently used by autonomous vehicles, drones and robots for navigation and other computer vision tasks. The images captured by such cameras, are often analyzed and classified with techniques designed for planar images that unfortunately fail to properly handle the native geometry of such images. That results in suboptimal performance, and lack of truly meaningful visual features. In this paper we aim at improving popular deep convolutional neural networks so that they can properly take into account the specific properties of omnidirectional data. In particular we propose an algorithm that adapts convolutional layers, which often serve as a core building block of a CNN, to the properties of omnidirectional images. Thus, our filters have a shape and size that adapts with the location on the omnidirectional image. We show that our method is not limited to spherical surfaces and is able to incorporate the knowledge about any kind of omnidirectional geometry inside the deep learning network. As depicted by our experiments, our method outperforms the existing deep neural network techniques for omnidirectional image classification and compression tasks.

Review:
This paper proposed to use graph-based deep learning methods to apply deep learning techniques to images coming from omnidirectional cameras. It solves the problem of distorsions introduced by the projection of such images by replacing convolutions by graph-based convolutions, with in particular a combinaison of directed graphs which makes the network able to distinguish between orientations.
The paper is fairly well written and easy to follow, and the need for treating omnidirectional images differently is well motivated. However, since the novelty is not so much in the graph convolution method, or in the use of graph methods for treating spherical signals, but in the combined application of the particular graph method proposed to the domain of omnidirectional images, I would expect a more thorough experimental study of the merits of the method and architectural choices.
1. The projected MNIST dataset looks very localized on the sphere and therefore does not seem to leverage that much of the global connectivity of the graph, although it can integrate deformations. Since the dataset is manually projected, why not cover more of the sphere and allow for a more realistic setting with respect to omnidirectional images?
More generally, why not use a realistic high resolution classification dataset and project it on the sphere? While it wouldn't allow for all the characteristics of omnidirectional images such as the wrapping around at the borders, it would lead to a more challenging classification problem. Papers such as [Khasanova & Frossard, 2017a] have at least used two toy-like datasets to discuss the merits of their classification method (MNIST-012, ETH-80), and a direct comparison with these baselines is not offered in this work.
2. The method can be applied for a broad variety of tasks but by evaluating it in a classification setting only, it is difficult to have an estimate of its performance in a detection setting, where I would see more uses for the proposed methods in such settings (in particular with respect to rotationally invariant methods, which do not allow for localization).
3. I fail to see the relevance of the experiments in Section 4.2 for a realistic application. Supposing a good model for spherical deformations of a lens is known, what prevents one from computing a reasonable inverse mapping and mapping the images back to a sphere? If the mapping is non-invertible (overlaps), then at least using an approximate inverse mapping would yield a competitive baseline.
I am surprised at the loss of accuracy in Table 2 with respect to the spherical baseline. Can you identify the source of this loss? Did you retrain the networks for the different deformations, or did you only change the projection of the network trained on a sphere? 
4. While the papers describes what happens at the level of the first filters, I did not find a clear explanation of what happens in upper layers, and find this point open to interpretation. Are graph convolutions used again based on the previous polynomial filter responses, sampling a bigger region on the sphere? Could you clarify this?
5. I would also like to see a study of the choice of the different scales used (in particular, size of the neighborhood).
Overall, I find that the paper introduces some interesting points but is too limited experimentally in its current form to allow for a fair evaluation of the merits of the method. Moreover, it leaves some important questions open as to how exactly it is applied (impact of sampling/neighborhood size, design of convolutions in upper layer...) which would need to be clarified and tested.
Additional small details:
- please do not use notation $\mathbb{N}_p$ for the neighborhood, it suggests integers
- p. 4 "While effective, these filters ... as according to Eq. (2) filter..." -> article missing for the word "filter"