Question: How to test the models for linguistic generalization in this paper?
Evidence: It is well known that language has certain structural properties which allows natural language speakers to make “infinite use of finite means" BIBREF3 . This structure allows us to generalize beyond the typical machine learning definition of generalization BIBREF4 (which considers performance on the distribution that generated the training set), permitting the understanding of any utterance sharing the same structure, regardless of probability. For example, sentences of length 100 typically do not appear in natural text or speech (our personal 'training set'), but can be understood regardless due to their structure. We refer to this notion as linguistic generalization .
Many problems in NLP are treated as sequence to sequence tasks with solutions built on seq2seq-attention based models. While these models perform very well on standard datasets and also appear to capture some linguistic structure BIBREF5 , BIBREF6 , BIBREF7 , they also can be quite brittle, typically breaking on uncharacteristic inputs BIBREF8 , BIBREF1 , indicating that the extent of linguistic generalization these models achieve is still somewhat lacking.
Due to the high capacity of these models, it is not unreasonable to expect them to learn some structure from the data. However, learning structure is not a sufficient condition to achieving linguistic generalization. If this structure is to be usable on data outside the training distribution, the model must learn the structure without additionally learning (overfitting on) patterns specific to the training data. One may hope, given the right hyperparameter configuration and regularization, that a model converges to a solution that captures the reusable structure without overfitting too much on the training set. While this solution exists in theory, in practice, it may be difficult to find.
In this work, we look at the feasibility of training and tuning seq2seq-attention models towards a solution that generalizes in this linguistic sense. In particular, we train models on a symbol replacement task with a well defined generalizable structure. The task is simple enough that all models achieve near perfect accuracy on the standard test set, i.e., where the inputs are drawn from the same distribution as that of the training set. We then test these models for linguistic generalization by creating test sets of uncharacteristic inputs, i.e., inputs that are not typical in the training distribution but still solvable given that the generalizable structure was learned. Our results indicate that generalization is highly sensitive; such that even changes in the random seed can drastically affect the ability to generalize. This dependence on an element that is not (or ideally should not be) a hyperparameter suggests that the line between generalization and failure is quite fine, and may not be feasible to reach simply by hyperparameter tuning alone.
We then test these models for linguistic generalization by creating test sets of uncharacteristic inputs, i.e., inputs that are not typical in the training distribution but still solvable given that the generalizable structure was learned.
Answer: The authors create test sets of uncharacteristic inputs to test the models, i.e., inputs that are not typical in the training distribution but still solvable given that the generalizable structure was learned.