Before revision:  English GPT-2 models with relearned lexical embeddings can generate realistic sentences in Italian and Dutch , but on average these sentences are still identifiable as artificial by humans . Based on perplexity scores and human judgements, we find that generated sentences become more realistic with some additional full model finetuning, especially for Dutch.
After revision:  English GPT-2 models with relearned lexical embeddings can generate realistic sentences in Italian and Dutch . Though on average these sentences are still identifiable as artificial by humans . Based on perplexity scores and human judgements, we find that generated sentences become more realistic with some additional full model finetuning, especially for Dutch.