Task: Language Identification
Input (Spanish, English): This paper essentially presents a new inductive bias in the architecture of (convolutional) neural networks (CNN). The mathematical motivations/derivations of the proposed architecture are detailed and rigorous. The proposed architecture promises to produce equivariant representations with steerable features using fewer parameters than traditional CNNs, which is particularly useful in small data regimes. Interesting and novel connections are presented between steerable filters and so called “steerable fibers”. The architecture is strongly inspired by the author’s previous work, as well as that of “capsules” (Hinton, 2011). The proposed architecture is compared on CIFAR10 against state-of-the-art inspired architectures (ResNets), and is shown to be superior particularly in the small data regime. The lack of empirical comparison on large scale dataset, such as ImageNet or COCO makes this largely a theoretical contribution. I would have also liked to see more empirical evaluation of the equivariance properties. It is not intuitively clear exactly why this architecture performs better on CIFAR10 as it is not clear that capturing equivariances helps to classify different instances of object categories. Wouldn’t action-recognition in videos, for example, not be a better illustrative dataset?
Output (English): en