text: 262,95,1392737009601503235,2640805367,Milan Gritta 🇬🇧 🇸🇰 🇺🇦,Hey! New paper with @iiacobacNLP called 'XeroAlign: Zero-Shot Cross-lingual Transformer Alignment' to be published in Findings of ACL 2021 :) Read the preprint at <LINK> #huawei #NLProc <LINK> Use this simple technique to align the representations of XLM-RoBERTa (or other pretrained #multilingual transformers) across languages for strong zero-shot transfer. #noahsArkLab #Huawei <LINK> XeroAlign achieves SOTA scores on 3 cross-lingual task-oriented natural language understanding datasets. It also works well for text classification tasks like paraphrase detection. Simple and effective! #ACL2021 <LINK>,https://arxiv.org/abs/2105.02472,"The introduction of pretrained cross-lingual language models brought decisive improvements to multilingual NLP tasks. However, the lack of labelled task data necessitates a variety of methods aiming to close the gap to high-resource languages. Zero-shot methods in particular, often use translated task data as a training signal to bridge the performance gap between the source and target language(s). We introduce XeroAlign, a simple method for task-specific alignment of cross-lingual pretrained transformers such as XLM-R. XeroAlign uses translated task data to encourage the model to generate similar sentence embeddings for different languages. The XeroAligned XLM-R, called XLM-RA, shows strong improvements over the baseline models to achieve state-of-the-art zero-shot results on three multilingual natural language understanding tasks. XLM-RA's text classification accuracy exceeds that of XLM-R trained with labelled data and performs on par with state-of-the-art models on a cross-lingual adversarial paraphrasing task. ",XeroAlign: Zero-Shot Cross-lingual Transformer Alignment,3,"[""Hey! New paper with @iiacobacNLP called 'XeroAlign: Zero-Shot Cross-lingual Transformer Alignment' to be published in Findings of ACL 2021 :) Read the preprint at <LINK> #huawei #NLProc <LINK>"", 'Use this simple technique to align the representations of XLM-RoBERTa (or other pretrained #multilingual transformers) across languages for strong zero-shot transfer. #noahsArkLab #Huawei https://t.co/vIZ7iLZ6fC', 'XeroAlign achieves SOTA scores on 3 cross-lingual task-oriented natural language understanding datasets. It also works well for text classification tasks like paraphrase detection. Simple and effective! #ACL2021 https://t.co/XqYqUsmk9i']",21,05,606