Before revision:  Training on artificial languages containing recursion (hierarchical structure) also improves performance on natural language, again with no vocabulary overlap . Surprisingly, training  on artificial languages consisting of sets of separated pairs of words, but with no recursion, improves performance on natural language as well as recursive languages do. Experiments on transfer between natural languages  show that zero-shot performance on a test language is highly correlated with typological syntactic similarity to the training language, suggesting that representations induced from natural languages correspond to the cross-linguistic syntactic properties studied in linguistic typology .
After revision:  Training on artificial languages containing recursion (hierarchical structure) also improves performance on natural language, again with no vocabulary overlap . Surprisingly, training on artificial languages consisting of sets of separated pairs of words, but with no recursion, improves performance on natural language . Further experiments on transfer between natural languages show that zero-shot performance on a test language is highly correlated with typological syntactic similarity to the training language, suggesting that representations induced from natural languages correspond to the cross-linguistic syntactic properties studied in linguistic typology .