Before revision: We propose a novel methodology for analyzing the encoding of grammatical structure in neural language models through transfer learning. We test how a language model can leverage its internal representations to transfer knowledge across languages and symbol systems. We train LSTMs on non-linguistic , structured data and test their performance on human language to assess which kinds of data induce generalizable encodings that LSTMs can use for natural language.
After revision: We propose a novel methodology for analyzing the encoding of grammatical structure in neural language models . We train LSTMs on non-linguistic , structured data and test their performance on human language to assess which kinds of data induce generalizable encodings that LSTMs can use for natural language.