Question: What challenges this work presents that must be solved to build better language-neutral representations?
Answer: contextual embeddings do not represent similar semantic phenomena similarly and therefore they are not directly usable for zero-shot cross-lingual tasks