Before revision:  Our model does not encode the word order information of the input sequences, and the predictions for each token are based on the attention on the whole sequence . Experimental results on dialogue natural language understanding, part-of-speech tagging, and named entity recognition tasks show that getting rid of word order information is able to achieve better zero-shot cross-lingual performance than baseline models .
After revision:  Our model does not encode the word order information of the input sequences, and the predictions for each token are based on the attention on the whole sequence . Experimental results on dialogue natural language understanding, part-of-speech tagging, and named entity recognition tasks show that getting rid of word order information is able to achieve better zero-shot cross-lingual performance . Furthermore, our proposed methods can also be applied to strong cross-lingual baselines, and improve their performances .