Before revision:  We hypothesize that cross-lingual models that fit into the source language word order might fail to handle target languages whose word orders are different. To test our conjecture, we build an order-agnostic model for cross-lingual sequence labeling tasks. Our model does not encode the word order information of the input sequences, and the predictions for each token are based on the attention on the whole sequence . Experimental results on dialogue natural language understanding, part-of-speech tagging, and named entity recognition tasks show that getting rid of word order information is able to achieve better zero-shot cross-lingual performance than baseline models .
After revision:  We hypothesize that cross-lingual models that fit into the source language word order might fail to handle target languages . To verify this hypothesis, we investigate whether making models insensitive to the word order information of the input sequences, and the predictions for each token are based on the attention on the whole sequence . Experimental results on dialogue natural language understanding, part-of-speech tagging, and named entity recognition tasks show that getting rid of word order information is able to achieve better zero-shot cross-lingual performance than baseline models .