Before revision:  English GPT-2 models with relearned lexical embeddings can generate realistic sentences in Italian and Dutch , but on average these sentences are still identifiable as artificial by humans . Based on perplexity scores and human judgements, we find that generated sentences become more realistic with some additional full model finetuning, especially for Dutch. For Italian, we see that they are evaluated on par with sentences generated by a GPT-2 model fully trained from scratch .
After revision:  English GPT-2 models with relearned lexical embeddings can generate realistic sentences in Italian and Dutch , but on average these sentences are still identifiable as artificial by humans , they are assessed on par with sentences generated by a GPT-2 model fully trained from scratch .