Task: Language Identification
Input (Spanish, English): This paper proposes a design principle for computation blocks in convolutional networks based on repeated application of expand and join operations resulting in a fractal-like structure.   This paper is primarily about experimental evaluation, since the objective is to show that a residual formulation is not necessary to obtain good performance, at least on some tasks.  However, in my opinion the evaluations in the paper are not convincing. The primary issue is lack of a proper baseline, against which the improvements can be clearly demonstrated by making isolated changes. I understand that for this paper such a baseline is hard to construct, since it is about a novel architecture principle. This is why more effort should be put into this, so that core insights from this paper can be useful even after better performing architectures are discovered. The number of parameters and amount of computation should be used to indicate how fair the comparisons are between architectures. Some detailed comments:  - In Table 1 comparisons to Resnets, the resnets from He et al. 2016b and Wide Resnets should be compared to FractalNet (in lieu of a proper baseline). The first outperforms FractalNet on CIFAR-100 while the second outperforms it on both. The authors compare to other results without augmentation, but did not perform additional experiments without augmentation for these architectures.  - The 40 layer Fractal Net should not be compared to other models unless the parameter reduction tricks are utilized for the other models as well.  - A proper comparison to Inception networks should also be performed for these networks. My guess is that the reason behind a seemingly 'ad-hoc' design of Inception modules is to reduce the computational footprint of the model (which is not a central motivation of fractal nets). Since this model is directly related to the Inception module due to use of shorter and longer paths without shortcuts, one can easily simplify the Inception design to build a strong baseline e.g. by converting the concatenation operation to a mean operation among equally sized convolution outputs. As an aside, note that Inception networks have already shown that residual networks are not necessary to obtain the best performance [1].  - It should be noted that Residual/Highway architectures do have a type of anytime property, as shown by lesioning experiments in Srivastava et al and Viet et al.  - The architecture specific drop-path regularization is interesting, but is used along with other regularizers such as dropout, batch norm and weight decay and its benefit on its own is not clear.  Overall, it's not clear to me that the experiments clearly demonstrate the utility of the proposed architecture.   [1] Szegedy, Christian, Sergey Ioffe, and Vincent Vanhoucke. "Inception-v4, inception-resnet and the impact of residual connections on learning." arXiv preprint arXiv:1602.07261 (2016).
Output (English): en