Task: Language Identification
Input (Spanish, English): This paper proposes a method for generating datasets of pictures from simple building blocks, as well as corresponding logical forms and language descriptions. The goal seems to be to have a method where the complexity of pictures and corresponding desciptions can be controlled and parametrized.    - The biggest downside seems to be that the maximally achievable complexity is very limited, and way below the complexity typically faced with image-captioning and other multimodal tasks.   - The relative simplicity is also a big difference to the referenced bAbI tasks (which cover the whole qualitative spectrum of easy-to-hard reasoning tasks), whereas in the proposed method a (qualitatively) easy image reconition task can only be quantitatively made harder, by increasing the number of objects, noise etc in unnatural ways.  - This is also reflected in the experimental section. Whenever the experimental performance results are not satisfying, these cases seem like basic over/underfitting issues that may easily be tackled by restricting/extending the capacity of the networks or using more data. It is hard for me to spot any other qualitative insight.  - In the introduction it is stated that the "goal is not too achieve optimal performance" but to find out whether "architectures are able to successfully demonstrate the desired understanding" - there is a fundamental contradiction here, in that the proposed task on the one side is meant to provide a measure as to whether architectures demontrate "understanding", on the other hand the score is not supposed to be taken as meaningful/seriously.  General comments: The general approach should be made more tangible earlier (i.e. in the introction rather than in section 3)
Output (English): en